{
  "id": 12417,
  "title": "Feature Extraction",
  "url": "/competitions/datasciencebowl/discussion/12417",
  "author_name": "",
  "post_date": "2015-02-05T13:45:13.107Z",
  "votes": null,
  "comment_count": 21,
  "views": 9824,
  "content": "<p>Hello all</p>\n<p>Please, I am very new to data science and image processing. I have very hard time with feature extraction. I would appreciate if you refer me any tutorial, book, etc that could help me.</p>\n<p>I read briefly&nbsp;opencv feature extraction and skimage features but to be honest didn't understand it and didn't get any idea on how to use them.</p>\n<p>Thanks for any help</p>",
  "messages": [
    {
      "id": "63636",
      "postDate": "02/05/2015 13:45:13",
      "content": "<p>Hello all</p>\n<p>Please, I am very new to data science and image processing. I have very hard time with feature extraction. I would appreciate if you refer me any tutorial, book, etc that could help me.</p>\n<p>I read briefly&nbsp;opencv feature extraction and skimage features but to be honest didn't understand it and didn't get any idea on how to use them.</p>\n<p>Thanks for any help</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "63638",
      "postDate": "02/05/2015 15:29:16",
      "content": "<p>Have you looked at the tutorial (http://www.kaggle.com/c/datasciencebowl/details/tutorial)?</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "63678",
      "postDate": "02/06/2015 06:44:06",
      "content": "<p>I looked at the tutorial and implemented it but the problem it is just one feature the ratio of length over width. I meant how to extract more advanced features like Fourier descriptors etc.</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "63695",
      "postDate": "02/06/2015 13:33:19",
      "content": "<p>Here are some things I tried (which helped a lot) when I was playing with feature engineering. I basically extracted a bunch more features from the largest region and then added additional features from the mahotas library.</p>\n<p><a href=\"https://gist.github.com/msegala/06d79d6c3678b97fe484\">https://gist.github.com/msegala/06d79d6c3678b97fe484</a></p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "63743",
      "postDate": "02/07/2015 10:00:58",
      "content": "<p>Thanks mike1886&nbsp;</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "63799",
      "postDate": "02/09/2015 04:42:33",
      "content": "<p>Hey Mike1886</p>\n<p>I am extracting a similar set of features as you are using--the largest region characteristics, the count of number of unconnected regions. &nbsp;Also, I rotate the thresholded 25x25 pixel image and then vectorize that. &nbsp;</p>\n<p>The result ends up being 637 columns per image (12 image property values and 625 0/1 pixel values.) &nbsp;I have tried to fit that with the the obvious models, RandomForest, Softmax, MultinomialNaiveBayes on only the 0/1 pixels.</p>\n<p>My best score is ~2.3. &nbsp;If it isn't too much to give away, do you think the mahotas library produces features which perform really well in the more basic models listed above (as opposed to CNN's) or does most of the value come from training a neural net? &nbsp;Not sure whether more work on feature selection is the way to go, or more work on the model fitting side.</p>\n<p>Thanks!</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "63808",
      "postDate": "02/09/2015 14:01:02",
      "content": "<p>hi emeschke,&nbsp;</p>\n<p>Using the features I described above&nbsp;and a RF with 2000 trees, my best score was 2.16. Maybe you can get it a bit better but after that I switched to CNN's and now have a score of ~0.76. There was a post of someone using SVMs and getting a score of ~1.1. I think they were doing some k-means feature generation.&nbsp;</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "63844",
      "postDate": "02/10/2015 00:02:35",
      "content": "<p>Hello Mike1886</p>\n<p>&nbsp; &nbsp;which program u use for CNN's ? and if you have any tutorials and guides it would be awesome.</p>\n<p>my goal is just to be in the first 50&nbsp;</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "63847",
      "postDate": "02/10/2015 00:25:04",
      "content": "<p>Hey Mike1886,</p>\n<p>The best score I was able to get with RF was about 1.17. I've since switched to CNNs and my first few attempts have gotten me to about 0.98. Might I ask how long your best CNN took to train?</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "63855",
      "postDate": "02/10/2015 01:05:07",
      "content": "<p>Data-Lover: I use <a href=\"https://github.com/benanne/Lasagne\">lasagne</a>&nbsp;and <a href=\"https://github.com/dnouri/nolearn\">nolearn</a>, you can find an awesome tutorial here&nbsp;http://danielnouri.org/notes/2014/12/17/using-convolutional-neural-nets-to-detect-facial-keypoints-tutorial/</p>\n\n<p>Zach: My CNN models (on a&nbsp;GeForce GTX TITAN Black GPU) takes about 3 hours. That is a pretty good card so I would suspect it is faster than others.&nbsp;</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "63856",
      "postDate": "02/10/2015 01:08:31",
      "content": "Thanks Mike 1886",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "63857",
      "postDate": "02/10/2015 01:12:17",
      "content": "<p>Wow, I would love to have that card. I've been doing mine on a GeForce GT 650M. If that took 3 hours for you, it's probably unfeasible on my card.&nbsp;</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "63859",
      "postDate": "02/10/2015 01:24:39",
      "content": "<p>Guys&nbsp;do you think if it is possible to run those programs on Linux cluster?</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "63860",
      "postDate": "02/10/2015 01:30:36",
      "content": "<p>Yes, I use it on Ubuntu. Just pip install and run.&nbsp;</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "63861",
      "postDate": "02/10/2015 01:39:34",
      "content": "<p>Mike I can't install the programs like python or scikit on this cluster. I just can send the executable program as a job. the problem my brain is exhausted from the much of information I have to learn in short time.</p>\n<p>What u suggest in this case? the cluster is so powerful about 5000 cores</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "63862",
      "postDate": "02/10/2015 01:41:50",
      "content": "<p>sorry I am not sure about this, best is to ask your system admin.&nbsp;</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "64088",
      "postDate": "02/12/2015 03:36:42",
      "content": "<p>Hey Mike/Zach</p>\n<p>Thanks for the thoughts. &nbsp;I have a couple of ideas about more features (hu moments) to try and model parameters to tweak (SVM)...but it is helpful to have some idea what the ceiling on some of these models is.</p>\n<p>Also, that Lasagne tutorial on CNN's is great, thanks for the link. &nbsp;That may be a project for another competition.</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "65510",
      "postDate": "03/05/2015 14:06:34",
      "content": "<p>[quote=mike1886;63855]</p>\n<p>Data-Lover: I use <a href=\"https://github.com/benanne/Lasagne\">lasagne</a>&nbsp;and <a href=\"https://github.com/dnouri/nolearn\">nolearn</a>, you can find an awesome tutorial here&nbsp;http://danielnouri.org/notes/2014/12/17/using-convolutional-neural-nets-to-detect-facial-keypoints-tutorial/</p>\n<p>Zach: My CNN models (on a&nbsp;GeForce GTX TITAN Black GPU) takes about 3 hours. That is a pretty good card so I would suspect it is faster than others.&nbsp;</p>\n<p>[/quote]</p>\n\n<p>Hello Mike,</p>\n<p>I'm trying to apply this great tutorial to this challenge too but I have an issue while fitting. Maybe you ran into the same issue. Something must be wrong in the regression part (I set regression to False):</p>\n\n<p><code>net2 = NeuralNet(<br> layers=[<br> ('input', layers.InputLayer),<br> ('conv1', layers.Conv2DLayer),<br> ('pool1', layers.MaxPool2DLayer),<br> ('conv2', layers.Conv2DLayer),<br> ('pool2', layers.MaxPool2DLayer),<br> ('conv3', layers.Conv2DLayer),<br> ('pool3', layers.MaxPool2DLayer),<br> ('hidden4', layers.DenseLayer),<br> ('hidden5', layers.DenseLayer),<br> ('output', layers.DenseLayer),<br> ],<br> input_shape=(None, 1, 64, 64),<br> conv1_num_filters=32, conv1_filter_size=(3, 3), pool1_ds=(2, 2),<br> conv2_num_filters=64, conv2_filter_size=(2, 2), pool2_ds=(2, 2),<br> conv3_num_filters=128, conv3_filter_size=(2, 2), pool3_ds=(2, 2),<br> hidden4_num_units=500, hidden5_num_units=500,<br> output_num_units=121, output_nonlinearity=None,</code><code>update_learning_rate=0.01,<br> update_momentum=0.9,</code><code>regression=False,<br> max_epochs=3,<br> verbose=1,<br> )</code></p>\n<p>and I get the following error:</p>\n\n<p><code>InputLayer (None, 1, 64, 64) produces 4096 outputs<br> Conv2DLayer (None, 32, 62, 62) produces 123008 outputs<br> MaxPool2DLayer (None, 32, 31, 31) produces 30752 outputs<br> Conv2DLayer (None, 64, 30, 30) produces 57600 outputs<br> MaxPool2DLayer (None, 64, 15, 15) produces 14400 outputs<br> Conv2DLayer (None, 128, 14, 14) produces 25088 outputs<br> MaxPool2DLayer (None, 128, 7, 7) produces 6272 outputs<br> DenseLayer (None, 500) produces 500 outputs<br> DenseLayer (None, 500) produces 500 outputs<br> DenseLayer (None, 121) produces 121 outputs<br>---------------------------------------------------------------------------<br>IndexError Traceback (most recent call last)<br>&lt;ipython-input-22-b6fcb0014c3b&gt; in &lt;module&gt;()<br>----&gt; 1 net2.fit(X, y)</code><code>/home/frank/anaconda/lib/python2.7/site-packages/nolearn/lasagne.pyc in fit(self, X, y)<br> 242 <br> 243 try:<br>--&gt; 244 self.train_loop(X, y)<br> 245 except KeyboardInterrupt:<br> 246 pass</code><code>/home/frank/anaconda/lib/python2.7/site-packages/nolearn/lasagne.pyc in train_loop(self, X, y)<br> 249 def train_loop(self, X, y):<br> 250 X_train, X_valid, y_train, y_valid = self.train_test_split(<br>--&gt; 251 X, y, self.eval_size)<br> 252 <br> 253 on_epoch_finished = self.on_epoch_finished</code><code>/home/frank/anaconda/lib/python2.7/site-packages/nolearn/lasagne.pyc in train_test_split(self, X, y, eval_size)<br> 355 kf = KFold(y.shape[0], round(1. / eval_size))<br> 356 else:<br>--&gt; 357 kf = StratifiedKFold(y, round(1. / eval_size))<br> 358 <br> 359 train_indices, valid_indices = next(iter(kf))</code><code>/home/frank/anaconda/lib/python2.7/site-packages/sklearn/cross_validation.pyc in __init__(self, y, n_folds, indices, shuffle, random_state)<br> 428 for test_fold_idx, per_label_splits in enumerate(zip(*per_label_cvs)):<br> 429 for label, (_, test_split) in zip(unique_labels, per_label_splits):<br>--&gt; 430 label_test_folds = test_folds[y == label]<br> 431 # the test split can be too big because we used<br> 432 # KFold(max(c, self.n_folds), self.n_folds) instead of</code><code>IndexError: too many indices for array</code></p>\n\n<p>Thanks for any help/hint.....</p>\n<p>Regards</p>\n<p>Frank</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "65511",
      "postDate": "03/05/2015 14:12:56",
      "content": "<p>You also need</p>\n<p>output_nonlinearity=nonlinearities.softmax,</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "65521",
      "postDate": "03/05/2015 15:40:17",
      "content": "<p>Thanks a lot Mike. I think your fix took me a step further. But now I'm stuck here:</p>\n<p><code> InputLayer (None, 1, 64, 64) produces 4096 outputs<br> Conv2DLayer (None, 32, 62, 62) produces 123008 outputs<br> MaxPool2DLayer (None, 32, 31, 31) produces 30752 outputs<br> Conv2DLayer (None, 64, 30, 30) produces 57600 outputs<br> MaxPool2DLayer (None, 64, 15, 15) produces 14400 outputs<br> Conv2DLayer (None, 128, 14, 14) produces 25088 outputs<br> MaxPool2DLayer (None, 128, 7, 7) produces 6272 outputs<br> DenseLayer (None, 500) produces 500 outputs<br> DenseLayer (None, 500) produces 500 outputs<br> DenseLayer (None, 121) produces 121 outputs<br>Traceback (most recent call last):<br> File &quot;mnist_NDSB.py&quot;, line 128, in module<br> net2.fit(X, y)<br> File &quot;/usr/local/lib/python2.7/dist-packages/nolearn-0.6adev-py2.7.egg/nolearn/lasagne.py&quot;, line 253, in fit<br> self.train_loop(X, y)<br> File &quot;/usr/local/lib/python2.7/dist-packages/nolearn-0.6adev-py2.7.egg/nolearn/lasagne.py&quot;, line 260, in train_loop<br> X, y, self.eval_size)<br> File &quot;/usr/local/lib/python2.7/dist-packages/nolearn-0.6adev-py2.7.egg/nolearn/lasagne.py&quot;, line 366, in train_test_split<br> kf = StratifiedKFold(y, round(1. / eval_size))<br> File &quot;/usr/local/lib/python2.7/dist-packages/scikit_learn-0.15.2-py2.7-linux-x86_64.egg/sklearn/cross_validation.py&quot;, line 430, in __init__<br> label_test_folds = test_folds[y == label]<br>ValueError: boolean index array should have 1 dimension<br>Exception TypeError: TypeError(&quot;'NoneType' object is not callable&quot;,) in &lt;bound method UmfpackContext.new_del of scipy.sparse.linalg.dsolve.umfpack.umfpack.UmfpackContext object at 0x7f27df87acd0 ignored</code></p>\n<p>Do you have an idea where this might come from?</p>\n<p>thanks again</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "65526",
      "postDate": "03/05/2015 17:39:07",
      "content": "<p>Basically you passed an empty array to this line&nbsp; test_folds[y == label] either your&nbsp; y vector or label vector are empty</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "71781",
      "postDate": "04/15/2015 20:34:09",
      "content": "<p>Did you solve your problem, knarfben?</p>",
      "rawMarkdown": "",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 63638,
      "author_name": "msegala",
      "author_url": "",
      "post_date": "02/05/2015 15:29:16",
      "content": "<p>Have you looked at the tutorial (http://www.kaggle.com/c/datasciencebowl/details/tutorial)?</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 63678,
      "author_name": "",
      "author_url": "",
      "post_date": "02/06/2015 06:44:06",
      "content": "<p>I looked at the tutorial and implemented it but the problem it is just one feature the ratio of length over width. I meant how to extract more advanced features like Fourier descriptors etc.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 63695,
      "author_name": "msegala",
      "author_url": "",
      "post_date": "02/06/2015 13:33:19",
      "content": "<p>Here are some things I tried (which helped a lot) when I was playing with feature engineering. I basically extracted a bunch more features from the largest region and then added additional features from the mahotas library.</p>\n<p><a href=\"https://gist.github.com/msegala/06d79d6c3678b97fe484\">https://gist.github.com/msegala/06d79d6c3678b97fe484</a></p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 63743,
      "author_name": "",
      "author_url": "",
      "post_date": "02/07/2015 10:00:58",
      "content": "<p>Thanks mike1886&nbsp;</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 63799,
      "author_name": "emeschke",
      "author_url": "",
      "post_date": "02/09/2015 04:42:33",
      "content": "<p>Hey Mike1886</p>\n<p>I am extracting a similar set of features as you are using--the largest region characteristics, the count of number of unconnected regions. &nbsp;Also, I rotate the thresholded 25x25 pixel image and then vectorize that. &nbsp;</p>\n<p>The result ends up being 637 columns per image (12 image property values and 625 0/1 pixel values.) &nbsp;I have tried to fit that with the the obvious models, RandomForest, Softmax, MultinomialNaiveBayes on only the 0/1 pixels.</p>\n<p>My best score is ~2.3. &nbsp;If it isn't too much to give away, do you think the mahotas library produces features which perform really well in the more basic models listed above (as opposed to CNN's) or does most of the value come from training a neural net? &nbsp;Not sure whether more work on feature selection is the way to go, or more work on the model fitting side.</p>\n<p>Thanks!</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 63808,
      "author_name": "msegala",
      "author_url": "",
      "post_date": "02/09/2015 14:01:02",
      "content": "<p>hi emeschke,&nbsp;</p>\n<p>Using the features I described above&nbsp;and a RF with 2000 trees, my best score was 2.16. Maybe you can get it a bit better but after that I switched to CNN's and now have a score of ~0.76. There was a post of someone using SVMs and getting a score of ~1.1. I think they were doing some k-means feature generation.&nbsp;</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 63844,
      "author_name": "",
      "author_url": "",
      "post_date": "02/10/2015 00:02:35",
      "content": "<p>Hello Mike1886</p>\n<p>&nbsp; &nbsp;which program u use for CNN's ? and if you have any tutorials and guides it would be awesome.</p>\n<p>my goal is just to be in the first 50&nbsp;</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 63847,
      "author_name": "thekannman",
      "author_url": "",
      "post_date": "02/10/2015 00:25:04",
      "content": "<p>Hey Mike1886,</p>\n<p>The best score I was able to get with RF was about 1.17. I've since switched to CNNs and my first few attempts have gotten me to about 0.98. Might I ask how long your best CNN took to train?</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 63855,
      "author_name": "msegala",
      "author_url": "",
      "post_date": "02/10/2015 01:05:07",
      "content": "<p>Data-Lover: I use <a href=\"https://github.com/benanne/Lasagne\">lasagne</a>&nbsp;and <a href=\"https://github.com/dnouri/nolearn\">nolearn</a>, you can find an awesome tutorial here&nbsp;http://danielnouri.org/notes/2014/12/17/using-convolutional-neural-nets-to-detect-facial-keypoints-tutorial/</p>\n\n<p>Zach: My CNN models (on a&nbsp;GeForce GTX TITAN Black GPU) takes about 3 hours. That is a pretty good card so I would suspect it is faster than others.&nbsp;</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 63856,
      "author_name": "",
      "author_url": "",
      "post_date": "02/10/2015 01:08:31",
      "content": "Thanks Mike 1886",
      "votes": null,
      "replies": []
    },
    {
      "id": 63857,
      "author_name": "thekannman",
      "author_url": "",
      "post_date": "02/10/2015 01:12:17",
      "content": "<p>Wow, I would love to have that card. I've been doing mine on a GeForce GT 650M. If that took 3 hours for you, it's probably unfeasible on my card.&nbsp;</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 63859,
      "author_name": "",
      "author_url": "",
      "post_date": "02/10/2015 01:24:39",
      "content": "<p>Guys&nbsp;do you think if it is possible to run those programs on Linux cluster?</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 63860,
      "author_name": "msegala",
      "author_url": "",
      "post_date": "02/10/2015 01:30:36",
      "content": "<p>Yes, I use it on Ubuntu. Just pip install and run.&nbsp;</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 63861,
      "author_name": "",
      "author_url": "",
      "post_date": "02/10/2015 01:39:34",
      "content": "<p>Mike I can't install the programs like python or scikit on this cluster. I just can send the executable program as a job. the problem my brain is exhausted from the much of information I have to learn in short time.</p>\n<p>What u suggest in this case? the cluster is so powerful about 5000 cores</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 63862,
      "author_name": "msegala",
      "author_url": "",
      "post_date": "02/10/2015 01:41:50",
      "content": "<p>sorry I am not sure about this, best is to ask your system admin.&nbsp;</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 64088,
      "author_name": "emeschke",
      "author_url": "",
      "post_date": "02/12/2015 03:36:42",
      "content": "<p>Hey Mike/Zach</p>\n<p>Thanks for the thoughts. &nbsp;I have a couple of ideas about more features (hu moments) to try and model parameters to tweak (SVM)...but it is helpful to have some idea what the ceiling on some of these models is.</p>\n<p>Also, that Lasagne tutorial on CNN's is great, thanks for the link. &nbsp;That may be a project for another competition.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 65510,
      "author_name": "knarfben",
      "author_url": "",
      "post_date": "03/05/2015 14:06:34",
      "content": "<p>[quote=mike1886;63855]</p>\n<p>Data-Lover: I use <a href=\"https://github.com/benanne/Lasagne\">lasagne</a>&nbsp;and <a href=\"https://github.com/dnouri/nolearn\">nolearn</a>, you can find an awesome tutorial here&nbsp;http://danielnouri.org/notes/2014/12/17/using-convolutional-neural-nets-to-detect-facial-keypoints-tutorial/</p>\n<p>Zach: My CNN models (on a&nbsp;GeForce GTX TITAN Black GPU) takes about 3 hours. That is a pretty good card so I would suspect it is faster than others.&nbsp;</p>\n<p>[/quote]</p>\n\n<p>Hello Mike,</p>\n<p>I'm trying to apply this great tutorial to this challenge too but I have an issue while fitting. Maybe you ran into the same issue. Something must be wrong in the regression part (I set regression to False):</p>\n\n<p><code>net2 = NeuralNet(<br> layers=[<br> ('input', layers.InputLayer),<br> ('conv1', layers.Conv2DLayer),<br> ('pool1', layers.MaxPool2DLayer),<br> ('conv2', layers.Conv2DLayer),<br> ('pool2', layers.MaxPool2DLayer),<br> ('conv3', layers.Conv2DLayer),<br> ('pool3', layers.MaxPool2DLayer),<br> ('hidden4', layers.DenseLayer),<br> ('hidden5', layers.DenseLayer),<br> ('output', layers.DenseLayer),<br> ],<br> input_shape=(None, 1, 64, 64),<br> conv1_num_filters=32, conv1_filter_size=(3, 3), pool1_ds=(2, 2),<br> conv2_num_filters=64, conv2_filter_size=(2, 2), pool2_ds=(2, 2),<br> conv3_num_filters=128, conv3_filter_size=(2, 2), pool3_ds=(2, 2),<br> hidden4_num_units=500, hidden5_num_units=500,<br> output_num_units=121, output_nonlinearity=None,</code><code>update_learning_rate=0.01,<br> update_momentum=0.9,</code><code>regression=False,<br> max_epochs=3,<br> verbose=1,<br> )</code></p>\n<p>and I get the following error:</p>\n\n<p><code>InputLayer (None, 1, 64, 64) produces 4096 outputs<br> Conv2DLayer (None, 32, 62, 62) produces 123008 outputs<br> MaxPool2DLayer (None, 32, 31, 31) produces 30752 outputs<br> Conv2DLayer (None, 64, 30, 30) produces 57600 outputs<br> MaxPool2DLayer (None, 64, 15, 15) produces 14400 outputs<br> Conv2DLayer (None, 128, 14, 14) produces 25088 outputs<br> MaxPool2DLayer (None, 128, 7, 7) produces 6272 outputs<br> DenseLayer (None, 500) produces 500 outputs<br> DenseLayer (None, 500) produces 500 outputs<br> DenseLayer (None, 121) produces 121 outputs<br>---------------------------------------------------------------------------<br>IndexError Traceback (most recent call last)<br>&lt;ipython-input-22-b6fcb0014c3b&gt; in &lt;module&gt;()<br>----&gt; 1 net2.fit(X, y)</code><code>/home/frank/anaconda/lib/python2.7/site-packages/nolearn/lasagne.pyc in fit(self, X, y)<br> 242 <br> 243 try:<br>--&gt; 244 self.train_loop(X, y)<br> 245 except KeyboardInterrupt:<br> 246 pass</code><code>/home/frank/anaconda/lib/python2.7/site-packages/nolearn/lasagne.pyc in train_loop(self, X, y)<br> 249 def train_loop(self, X, y):<br> 250 X_train, X_valid, y_train, y_valid = self.train_test_split(<br>--&gt; 251 X, y, self.eval_size)<br> 252 <br> 253 on_epoch_finished = self.on_epoch_finished</code><code>/home/frank/anaconda/lib/python2.7/site-packages/nolearn/lasagne.pyc in train_test_split(self, X, y, eval_size)<br> 355 kf = KFold(y.shape[0], round(1. / eval_size))<br> 356 else:<br>--&gt; 357 kf = StratifiedKFold(y, round(1. / eval_size))<br> 358 <br> 359 train_indices, valid_indices = next(iter(kf))</code><code>/home/frank/anaconda/lib/python2.7/site-packages/sklearn/cross_validation.pyc in __init__(self, y, n_folds, indices, shuffle, random_state)<br> 428 for test_fold_idx, per_label_splits in enumerate(zip(*per_label_cvs)):<br> 429 for label, (_, test_split) in zip(unique_labels, per_label_splits):<br>--&gt; 430 label_test_folds = test_folds[y == label]<br> 431 # the test split can be too big because we used<br> 432 # KFold(max(c, self.n_folds), self.n_folds) instead of</code><code>IndexError: too many indices for array</code></p>\n\n<p>Thanks for any help/hint.....</p>\n<p>Regards</p>\n<p>Frank</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 65511,
      "author_name": "msegala",
      "author_url": "",
      "post_date": "03/05/2015 14:12:56",
      "content": "<p>You also need</p>\n<p>output_nonlinearity=nonlinearities.softmax,</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 65521,
      "author_name": "knarfben",
      "author_url": "",
      "post_date": "03/05/2015 15:40:17",
      "content": "<p>Thanks a lot Mike. I think your fix took me a step further. But now I'm stuck here:</p>\n<p><code> InputLayer (None, 1, 64, 64) produces 4096 outputs<br> Conv2DLayer (None, 32, 62, 62) produces 123008 outputs<br> MaxPool2DLayer (None, 32, 31, 31) produces 30752 outputs<br> Conv2DLayer (None, 64, 30, 30) produces 57600 outputs<br> MaxPool2DLayer (None, 64, 15, 15) produces 14400 outputs<br> Conv2DLayer (None, 128, 14, 14) produces 25088 outputs<br> MaxPool2DLayer (None, 128, 7, 7) produces 6272 outputs<br> DenseLayer (None, 500) produces 500 outputs<br> DenseLayer (None, 500) produces 500 outputs<br> DenseLayer (None, 121) produces 121 outputs<br>Traceback (most recent call last):<br> File &quot;mnist_NDSB.py&quot;, line 128, in module<br> net2.fit(X, y)<br> File &quot;/usr/local/lib/python2.7/dist-packages/nolearn-0.6adev-py2.7.egg/nolearn/lasagne.py&quot;, line 253, in fit<br> self.train_loop(X, y)<br> File &quot;/usr/local/lib/python2.7/dist-packages/nolearn-0.6adev-py2.7.egg/nolearn/lasagne.py&quot;, line 260, in train_loop<br> X, y, self.eval_size)<br> File &quot;/usr/local/lib/python2.7/dist-packages/nolearn-0.6adev-py2.7.egg/nolearn/lasagne.py&quot;, line 366, in train_test_split<br> kf = StratifiedKFold(y, round(1. / eval_size))<br> File &quot;/usr/local/lib/python2.7/dist-packages/scikit_learn-0.15.2-py2.7-linux-x86_64.egg/sklearn/cross_validation.py&quot;, line 430, in __init__<br> label_test_folds = test_folds[y == label]<br>ValueError: boolean index array should have 1 dimension<br>Exception TypeError: TypeError(&quot;'NoneType' object is not callable&quot;,) in &lt;bound method UmfpackContext.new_del of scipy.sparse.linalg.dsolve.umfpack.umfpack.UmfpackContext object at 0x7f27df87acd0 ignored</code></p>\n<p>Do you have an idea where this might come from?</p>\n<p>thanks again</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 65526,
      "author_name": "tofighnaghibi",
      "author_url": "",
      "post_date": "03/05/2015 17:39:07",
      "content": "<p>Basically you passed an empty array to this line&nbsp; test_folds[y == label] either your&nbsp; y vector or label vector are empty</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 71781,
      "author_name": "nenadz",
      "author_url": "",
      "post_date": "04/15/2015 20:34:09",
      "content": "<p>Did you solve your problem, knarfben?</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "63636": "",
    "63638": "",
    "63678": "",
    "63695": "",
    "63743": "",
    "63799": "",
    "63808": "",
    "63844": "",
    "63847": "",
    "63855": "",
    "63856": "",
    "63857": "",
    "63859": "",
    "63860": "",
    "63861": "",
    "63862": "",
    "64088": "",
    "65510": "",
    "65511": "",
    "65521": "",
    "65526": "",
    "71781": ""
  },
  "source": "meta"
}