{
  "id": 11263,
  "title": "Code example for making a submission",
  "url": "/competitions/datasciencebowl/discussion/11263",
  "author_name": "udibr",
  "post_date": "2014-12-16T12:42:02.993000",
  "votes": 13,
  "comment_count": 13,
  "views": 5406,
  "content": "<p>The notebook at&nbsp;<a href=\"http://nbviewer.ipython.org/github/udibr/datasciencebowl/blob/master/141215-tutorial-submission.ipynb\">http://nbviewer.ipython.org/github/udibr/datasciencebowl/blob/master/141215-tutorial-submission.ipynb</a></p>\n<p>shows you how to take the classifier from the&nbsp;<a href=\"http://nbviewer.ipython.org/github/udibr/datasciencebowl/blob/master/141215-tutorial.ipynb\">tutorial</a>&nbsp;and turn it into a submission you can upload to the leaderboard.</p>\n<p>enjoy!</p>",
  "messages": [
    {
      "id": 60341,
      "postDate": "2014-12-16T12:42:02.993Z",
      "content": "<p>The notebook at&nbsp;<a href=\"http://nbviewer.ipython.org/github/udibr/datasciencebowl/blob/master/141215-tutorial-submission.ipynb\">http://nbviewer.ipython.org/github/udibr/datasciencebowl/blob/master/141215-tutorial-submission.ipynb</a></p>\n<p>shows you how to take the classifier from the&nbsp;<a href=\"http://nbviewer.ipython.org/github/udibr/datasciencebowl/blob/master/141215-tutorial.ipynb\">tutorial</a>&nbsp;and turn it into a submission you can upload to the leaderboard.</p>\n<p>enjoy!</p>",
      "votes": 13
    },
    {
      "id": 61454,
      "postDate": "2014-12-18T08:31:22.080Z",
      "content": "<p>if you are running on windows you need to use the following statements:</p>\n\n<p>...</p>\n<p>labels = map(lambda s: s.split('\\\\')[-1], namesClasses)</p>\n<p>...</p>\n<p>images = map(lambda fileName: fileName.split('\\\\')[-1], fnames)</p>\n<p>...</p>",
      "votes": 3
    },
    {
      "id": 61729,
      "postDate": "2014-12-23T04:36:51.650Z",
      "content": "<p><strong>My mistake: always do a &quot;conda update&quot; and restart IPython when in doubt</strong></p>\n<p>y_pred = clf.predict_proba(X_test)<br>---------------------------------------------------------------------------<br>ValueError Traceback (most recent call last)<br>&lt;ipython-input-424-5b7fb286b5f1&gt; in &lt;module&gt;()<br>----&gt; 1 y_pred = clf.predict_proba(X_test)</p>\n<p>//anaconda/lib/python2.7/site-packages/sklearn/qda.pyc in predict_proba(self, X)<br> 201 Posterior probabilities of classification per class.<br> 202 &quot;&quot;&quot;<br>--&gt; 203 values = self._decision_function(X)<br> 204 # compute the likelihood of the underlying gaussian models<br> 205 # up to a multiplicative constant.</p>\n<p>//anaconda/lib/python2.7/site-packages/sklearn/qda.pyc in _decision_function(self, X)<br> 143 R = self.rotations_[i]<br> 144 S = self.scalings_[i]<br>--&gt; 145 Xm = X - self.means_[i]<br> 146 X2 = np.dot(Xm, R * (S ** (-0.5)))<br> 147 norm2.append(np.sum(X2 ** 2, 1))</p>\n<p>ValueError: operands could not be broadcast together with shapes (130400,626) (2,)</p>"
    },
    {
      "id": 61727,
      "postDate": "2014-12-23T03:51:32.950Z",
      "content": "<p>Thank you&nbsp;</p>"
    },
    {
      "id": 61574,
      "postDate": "2014-12-20T14:45:56.437Z",
      "content": "<p>For those with limited machines. Reducing number of trees saves some memory. I was getting &quot;Segmentation fault&quot; with 100 trees on my laptop. With 70 trees it works without errors. Also some may need to predict probability on batches.</p>"
    },
    {
      "id": 61514,
      "postDate": "2014-12-19T03:08:53.953Z",
      "content": "<p>You can submit the zip file.</p>"
    },
    {
      "id": 61489,
      "postDate": "2014-12-18T15:17:32.127Z",
      "content": "<p>Vincent,&nbsp; That was it! Thanks so much for the help!</p>"
    },
    {
      "id": 61449,
      "postDate": "2014-12-18T07:05:10.550Z",
      "content": "<p>Trying to run this sample python code.&nbsp; I've downloaded anaconda (python 2.7 version) and I get to In 27:&nbsp; df = df[header]&nbsp; (Everything looks good up to that point!) But when I execute df = df[header] I get a very long error saying all the header names are not in the index.&nbsp; Prior to that I'm getting the output that I should be, and everything seems to be working ok.</p>\n<p>The specific error is:&nbsp; raise KeyError('%s not in index' % objarr[mask])</p>\n<p>That's followed by KeyError: &quot;['acantharia_protist_big_center' 'acantharia_protist_halo' &lt;rest of names from the header&gt; 'unknown_unclassified'] not in index&quot;</p>\n<p>Any idea what might be wrong?</p>\n<p>Thanks,&nbsp; </p>\n<p>Chip</p>"
    },
    {
      "id": 60495,
      "postDate": "2014-12-17T16:17:39.953Z",
      "content": "<p>Does anyone use Matlab for this challenge? Is there a similar tutorial that uses Matlab?</p>"
    },
    {
      "id": 60353,
      "postDate": "2014-12-16T14:22:06.030Z",
      "content": "<p>Nevermind. i=0 ruined my model</p>"
    },
    {
      "id": 60349,
      "postDate": "2014-12-16T13:57:56.573Z",
      "content": "<p>Is there any difference between using opencv imread and skimage.io.imread? It seems im unable to beat even 5.0 if i use opencv with a validation score of 4.17,</p>"
    },
    {
      "id": 60348,
      "postDate": "2014-12-16T13:37:20.687Z",
      "content": "<p>Many thanks! Great job!</p>"
    },
    {
      "id": 61532,
      "postDate": "2014-12-19T09:41:19.130Z",
      "rawMarkdown": "",
      "votes": 1,
      "isDeleted": true
    },
    {
      "id": 61513,
      "postDate": "2014-12-19T03:06:52.040Z",
      "rawMarkdown": "",
      "votes": 1,
      "isDeleted": true
    }
  ],
  "comments": [
    {
      "id": 61454,
      "author_name": "Vincent Wong",
      "author_url": "",
      "post_date": "2014-12-18T08:31:22.080000",
      "content": "<p>if you are running on windows you need to use the following statements:</p>\n\n<p>...</p>\n<p>labels = map(lambda s: s.split('\\\\')[-1], namesClasses)</p>\n<p>...</p>\n<p>images = map(lambda fileName: fileName.split('\\\\')[-1], fnames)</p>\n<p>...</p>",
      "votes": 3,
      "replies": []
    },
    {
      "id": 61729,
      "author_name": "Mark Conway",
      "author_url": "",
      "post_date": "2014-12-23T04:36:51.650000",
      "content": "<p><strong>My mistake: always do a &quot;conda update&quot; and restart IPython when in doubt</strong></p>\n<p>y_pred = clf.predict_proba(X_test)<br>---------------------------------------------------------------------------<br>ValueError Traceback (most recent call last)<br>&lt;ipython-input-424-5b7fb286b5f1&gt; in &lt;module&gt;()<br>----&gt; 1 y_pred = clf.predict_proba(X_test)</p>\n<p>//anaconda/lib/python2.7/site-packages/sklearn/qda.pyc in predict_proba(self, X)<br> 201 Posterior probabilities of classification per class.<br> 202 &quot;&quot;&quot;<br>--&gt; 203 values = self._decision_function(X)<br> 204 # compute the likelihood of the underlying gaussian models<br> 205 # up to a multiplicative constant.</p>\n<p>//anaconda/lib/python2.7/site-packages/sklearn/qda.pyc in _decision_function(self, X)<br> 143 R = self.rotations_[i]<br> 144 S = self.scalings_[i]<br>--&gt; 145 Xm = X - self.means_[i]<br> 146 X2 = np.dot(Xm, R * (S ** (-0.5)))<br> 147 norm2.append(np.sum(X2 ** 2, 1))</p>\n<p>ValueError: operands could not be broadcast together with shapes (130400,626) (2,)</p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 61727,
      "author_name": "Santhosh-ladalla",
      "author_url": "",
      "post_date": "2014-12-23T03:51:32.950000",
      "content": "<p>Thank you&nbsp;</p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 61574,
      "author_name": "Ivan Iordanov",
      "author_url": "",
      "post_date": "2014-12-20T14:45:56.437000",
      "content": "<p>For those with limited machines. Reducing number of trees saves some memory. I was getting &quot;Segmentation fault&quot; with 100 trees on my laptop. With 70 trees it works without errors. Also some may need to predict probability on batches.</p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 61514,
      "author_name": "Chip",
      "author_url": "",
      "post_date": "2014-12-19T03:08:53.953000",
      "content": "<p>You can submit the zip file.</p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 61489,
      "author_name": "Chip",
      "author_url": "",
      "post_date": "2014-12-18T15:17:32.127000",
      "content": "<p>Vincent,&nbsp; That was it! Thanks so much for the help!</p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 61449,
      "author_name": "Chip",
      "author_url": "",
      "post_date": "2014-12-18T07:05:10.550000",
      "content": "<p>Trying to run this sample python code.&nbsp; I've downloaded anaconda (python 2.7 version) and I get to In 27:&nbsp; df = df[header]&nbsp; (Everything looks good up to that point!) But when I execute df = df[header] I get a very long error saying all the header names are not in the index.&nbsp; Prior to that I'm getting the output that I should be, and everything seems to be working ok.</p>\n<p>The specific error is:&nbsp; raise KeyError('%s not in index' % objarr[mask])</p>\n<p>That's followed by KeyError: &quot;['acantharia_protist_big_center' 'acantharia_protist_halo' &lt;rest of names from the header&gt; 'unknown_unclassified'] not in index&quot;</p>\n<p>Any idea what might be wrong?</p>\n<p>Thanks,&nbsp; </p>\n<p>Chip</p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 60495,
      "author_name": "",
      "author_url": "",
      "post_date": "2014-12-17T16:17:39.953000",
      "content": "<p>Does anyone use Matlab for this challenge? Is there a similar tutorial that uses Matlab?</p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 60353,
      "author_name": "Abhishek Thakur",
      "author_url": "",
      "post_date": "2014-12-16T14:22:06.030000",
      "content": "<p>Nevermind. i=0 ruined my model</p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 60349,
      "author_name": "Abhishek Thakur",
      "author_url": "",
      "post_date": "2014-12-16T13:57:56.573000",
      "content": "<p>Is there any difference between using opencv imread and skimage.io.imread? It seems im unable to beat even 5.0 if i use opencv with a validation score of 4.17,</p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 60348,
      "author_name": "Jiwei Liu",
      "author_url": "",
      "post_date": "2014-12-16T13:37:20.687000",
      "content": "<p>Many thanks! Great job!</p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 61532,
      "author_name": "",
      "author_url": "",
      "post_date": "2014-12-19T09:41:19.130000",
      "content": "",
      "votes": 1,
      "replies": []
    },
    {
      "id": 61513,
      "author_name": "",
      "author_url": "",
      "post_date": "2014-12-19T03:06:52.040000",
      "content": "",
      "votes": 1,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "60341": "",
    "61454": "",
    "61729": "",
    "61727": "",
    "61574": "",
    "61514": "",
    "61489": "",
    "61449": "",
    "60495": "",
    "60353": "",
    "60349": "",
    "60348": "",
    "61532": "",
    "61513": ""
  }
}