{
  "id": 11242,
  "title": "Getting started - a tutorial",
  "url": "/competitions/datasciencebowl/discussion/11242",
  "author_name": "",
  "post_date": "2014-12-15T14:55:48.983Z",
  "votes": 29,
  "comment_count": 21,
  "views": 14444,
  "content": "<p>ok Kagglers, did you check out the getting started tutorial for the National Data Science Bowl?</p>\n<p>The tutorial at&nbsp;https://www.kaggle.com/c/datasciencebowl/details/tutorial gives you&nbsp;step-by-step through a simple model to distinguish different types of plankton and demonstrate some tools for exploring the image dataset.</p>\n\n<p>Check it out. &nbsp;&nbsp;</p>",
  "messages": [
    {
      "id": "60235",
      "postDate": "12/15/2014 14:55:48",
      "content": "<p>ok Kagglers, did you check out the getting started tutorial for the National Data Science Bowl?</p>\n<p>The tutorial at&nbsp;https://www.kaggle.com/c/datasciencebowl/details/tutorial gives you&nbsp;step-by-step through a simple model to distinguish different types of plankton and demonstrate some tools for exploring the image dataset.</p>\n\n<p>Check it out. &nbsp;&nbsp;</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "60250",
      "postDate": "12/15/2014 18:27:12",
      "content": "<p>Thanks! I've made an ipython notebook out of your tutorial</p>\n<p><a href=\"http://nbviewer.ipython.org/github/udibr/datasciencebowl/blob/master/141215-tutorial.ipynb\">http://nbviewer.ipython.org/github/udibr/datasciencebowl/blob/master/141215-tutorial.ipynb</a></p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "60264",
      "postDate": "12/15/2014 19:47:02",
      "content": "<p>Hey zzspar, that's great! I'm sure a lot of folks will find it useful</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "60287",
      "postDate": "12/15/2014 21:34:57",
      "content": "<p>Thank you very much guys. May I ask what is the version of skimage in the tutorial? I installed&nbsp;0.10.1 skimage and it gave the error for this one:</p>\n<p>im = imread(example_file, as_grey=True)</p>\n<p>as_grey is not defined parameter, or something like that.</p>\n<p>I can solve it by commenting out as_grey</p>\n<p>but&nbsp;the&nbsp;multiclass_log_loss in the last cell gave me 9.1, which is much larger than 3.7 in original tutorial.&nbsp;</p>\n<p>Is this degrade&nbsp;a problem due to the version? Anyone run this code, what score do you get?</p>\n<p>Thank you.</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "60288",
      "postDate": "12/15/2014 21:51:25",
      "content": "<p>Hi rcarson,</p>\n\n<p>Make sure that you haven't accidentally used the imread from matplotlib (it doesn't have an as_grey option).&nbsp;</p>\n<p>http://scikit-image.org/docs/dev/api/skimage.io.html#skimage.io.imread</p>\n<p>vs.</p>\n<p>http://matplotlib.org/api/pyplot_api.html#matplotlib.pyplot.imread</p>\n<p>you can test this out by importing skimage.io&nbsp;</p>\n<p>from skimage import io<br>io.imread(file, as_grey=True)</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "60290",
      "postDate": "12/15/2014 21:55:32",
      "content": "<p>Thank you very much! That's the problem!</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "60292",
      "postDate": "12/15/2014 22:29:04",
      "content": "<p>LB score is ~2.85 after doubling the number of trees.</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "60335",
      "postDate": "12/16/2014 10:43:19",
      "content": "<p>can anyone upload the full file for submission (just being lazy) :P</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "61498",
      "postDate": "12/18/2014 17:15:08",
      "content": "<p>Is it obligatory to use same image pre-processing as shown in the tutorial ?</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "61739",
      "postDate": "12/23/2014 09:37:53",
      "content": "<p>Thanks for the notebook!&nbsp;</p>\n<p>I tried running the code but I keep getting the following error:</p>\n<p>Traceback (most recent call last):<br> File &quot;data_science.py&quot;, line 145, in &lt;module&gt;<br> kf = KFold(y, n_folds=5)<br> File &quot;~/anaconda/lib/python2.7/site-packages/sklearn/cross_validation.py&quot;, line 402, in __init__<br> len(y), n_folds, indices, shuffle, random_state)<br> File &quot;~/anaconda/lib/python2.7/site-packages/sklearn/cross_validation.py&quot;, line 253, in __init__<br> &quot; than the number of samples: {1}.&quot;).format(n_folds, n))<br>ValueError: Cannot have number of folds n_folds=5 greater than the number of samples: 0.</p>\n<p>At this point, I'm wondering whether my version of anaconda is right.. Has anyone else run into this issue?</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "61740",
      "postDate": "12/23/2014 09:38:56",
      "content": "<p>The error got cut off- here it is again:</p>\n<p>Traceback (most recent call last):<br> File &quot;data_science.py&quot;, line 145, in module<br> kf = KFold(y, n_folds=5)<br> File &quot;/Users/aishwaryaafzulpurkar/anaconda/lib/python2.7/site-packages/sklearn/cross_validation.py&quot;, line 402, in __init__<br> len(y), n_folds, indices, shuffle, random_state)<br> File &quot;/Users/aishwaryaafzulpurkar/anaconda/lib/python2.7/site-packages/sklearn/cross_validation.py&quot;, line 253, in __init__<br> &quot; than the number of samples: {1}.&quot;).format(n_folds, n))<br>ValueError: Cannot have number of folds n_folds=5 greater than the number of samples: 0.</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "61784",
      "postDate": "12/23/2014 22:06:01",
      "content": "<p>@aishwarya</p>\n<p>It seems like a data read could be the problem (number of samples is zero).</p>\n<p>Can you check that the input file was read correctly? Do you have the training data in the directory expected by the script? Hope that helps. Good luck!</p>\n<p>ValueError: <strong>Cannot have number of folds n_folds=5 greater than the number of samples: 0.</strong></p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "61795",
      "postDate": "12/24/2014 00:24:12",
      "content": "<p>Thanks! Turns out I just had to change the directory name that was being searched..</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "61845",
      "postDate": "12/24/2014 23:20:16",
      "content": "<p>Glad to know that you were able to resolve the problem. Best of luck!</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "61927",
      "postDate": "12/27/2014 08:24:04",
      "content": "<p>while running top few lines ( till<a href=\"https://www.kaggle.com/c/datasciencebowl/details/tutorial\" target=\"_blank\"> [6] </a>in tutorial) , i am facing this error</p>\n\n<p>&quot; File &quot;C:\\Continuum\\Anaconda\\lib\\site-packages\\PIL\\Image.py&quot;, line 1956, in open</p>\n<p>prefix = fp.read(16)</p>\n<p>AttributeError: 'list' object has no attribute 'read' &quot;</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "61932",
      "postDate": "12/27/2014 14:48:58",
      "content": "<p>Your 'example_file' object is a list returned from glob. You'll need to subset it.&nbsp;</p>\n<p>im = imread(example_file[0], as_grey=True)</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "61963",
      "postDate": "12/28/2014 00:20:15",
      "content": "<p>@zzspar - thanks again for the notebook!</p>\n<p>anyone else getting the following error? thanks y'all!</p>\n<p>IndexError Traceback (most recent call last)<br>&lt;ipython-input-4-ff2a4c5acd3d&gt; in &lt;module&gt;()<br> 2 # This example was chosen for because it has two noncontinguous pieces<br> 3 # that will make the segmentation example more illustrative<br>----&gt; 4 example_file = glob.glob(os.path.join(directory_names[5],&quot;*.jpg&quot;))[9]<br> 5 print example_file<br> 6 im = imread(example_file, as_grey=True)</p>\n<p>IndexError: list index out of range</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "61965",
      "postDate": "12/28/2014 00:41:01",
      "content": "<p>oops error got cufoff:</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "62085",
      "postDate": "12/30/2014 05:03:42",
      "content": "<p>@ <a href=\"http://www.kaggle.com/users/88875/aj-buckeye\">Aj_Buckeye</a>&nbsp;Thanks got it working, @&nbsp;&nbsp;<a href=\"http://www.kaggle.com/users/209396/ytk9000\">YTK9000</a>&nbsp;see&nbsp;Aj_Buckeye answer</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "62522",
      "postDate": "01/09/2015 19:25:06",
      "content": "<p>Hey, guys!<br>I've tried to launch tutorial script several times and in the moment of RF-training kernel of ipython notebook was dying. I'm working in Windows 7, hardware: 8 GB RAM, core i7.</p>\n<p>Task manager showed in moment of RF-training python obtained ~1,1 gb of RAM and 4 gb was free. I've no idea what the reason of happening.</p>\n<p>Could anyone suggest solution&nbsp;of the trouble?</p>\n<p>Thank you for attention!</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "62560",
      "postDate": "01/10/2015 13:15:53",
      "content": "<p>I had the same problem. I switched from the python xy packet which uses the 32 bit python 2.7 version to Winpython for which you can get a 64 bit version of python (in my case I went for&nbsp;WinPython-64bit-3.3.5.4). I know that 32 bit python 2.7 has a limit of 2 gb, not quite sure why the problem occurs at 1.1 gb (was exactly the same for me). But the 64 bit version of python can use more memory. After that everything worked fine for me. &nbsp;</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "62567",
      "postDate": "01/10/2015 15:19:40",
      "content": "<p>[quote=SteveNL;62560]</p>\n<p>I had the same problem. I switched from the python xy packet which uses the 32 bit python 2.7 version to Winpython for which you can get a 64 bit version of python (in my case I went for&nbsp;WinPython-64bit-3.3.5.4). I know that 32 bit python 2.7 has a limit of 2 gb, not quite sure why the problem occurs at 1.1 gb (was exactly the same for me). But the 64 bit version of python can use more memory. After that everything worked fine for me. &nbsp;</p>\n<p>[/quote]</p>\n\n<p>Maybe I am wrong but is it because the windows task manager displays the physical memory rather than the virtual memory of the process ?</p>",
      "rawMarkdown": "",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 60250,
      "author_name": "udibr1",
      "author_url": "",
      "post_date": "12/15/2014 18:27:12",
      "content": "<p>Thanks! I've made an ipython notebook out of your tutorial</p>\n<p><a href=\"http://nbviewer.ipython.org/github/udibr/datasciencebowl/blob/master/141215-tutorial.ipynb\">http://nbviewer.ipython.org/github/udibr/datasciencebowl/blob/master/141215-tutorial.ipynb</a></p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 60264,
      "author_name": "angelazutavern",
      "author_url": "",
      "post_date": "12/15/2014 19:47:02",
      "content": "<p>Hey zzspar, that's great! I'm sure a lot of folks will find it useful</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 60287,
      "author_name": "jiweiliu",
      "author_url": "",
      "post_date": "12/15/2014 21:34:57",
      "content": "<p>Thank you very much guys. May I ask what is the version of skimage in the tutorial? I installed&nbsp;0.10.1 skimage and it gave the error for this one:</p>\n<p>im = imread(example_file, as_grey=True)</p>\n<p>as_grey is not defined parameter, or something like that.</p>\n<p>I can solve it by commenting out as_grey</p>\n<p>but&nbsp;the&nbsp;multiclass_log_loss in the last cell gave me 9.1, which is much larger than 3.7 in original tutorial.&nbsp;</p>\n<p>Is this degrade&nbsp;a problem due to the version? Anyone run this code, what score do you get?</p>\n<p>Thank you.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 60288,
      "author_name": "sander31",
      "author_url": "",
      "post_date": "12/15/2014 21:51:25",
      "content": "<p>Hi rcarson,</p>\n\n<p>Make sure that you haven't accidentally used the imread from matplotlib (it doesn't have an as_grey option).&nbsp;</p>\n<p>http://scikit-image.org/docs/dev/api/skimage.io.html#skimage.io.imread</p>\n<p>vs.</p>\n<p>http://matplotlib.org/api/pyplot_api.html#matplotlib.pyplot.imread</p>\n<p>you can test this out by importing skimage.io&nbsp;</p>\n<p>from skimage import io<br>io.imread(file, as_grey=True)</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 60290,
      "author_name": "jiweiliu",
      "author_url": "",
      "post_date": "12/15/2014 21:55:32",
      "content": "<p>Thank you very much! That's the problem!</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 60292,
      "author_name": "devinanzelmo",
      "author_url": "",
      "post_date": "12/15/2014 22:29:04",
      "content": "<p>LB score is ~2.85 after doubling the number of trees.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 60335,
      "author_name": "abhishek",
      "author_url": "",
      "post_date": "12/16/2014 10:43:19",
      "content": "<p>can anyone upload the full file for submission (just being lazy) :P</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 61498,
      "author_name": "pzpzpz",
      "author_url": "",
      "post_date": "12/18/2014 17:15:08",
      "content": "<p>Is it obligatory to use same image pre-processing as shown in the tutorial ?</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 61739,
      "author_name": "aishwarya229018",
      "author_url": "",
      "post_date": "12/23/2014 09:37:53",
      "content": "<p>Thanks for the notebook!&nbsp;</p>\n<p>I tried running the code but I keep getting the following error:</p>\n<p>Traceback (most recent call last):<br> File &quot;data_science.py&quot;, line 145, in &lt;module&gt;<br> kf = KFold(y, n_folds=5)<br> File &quot;~/anaconda/lib/python2.7/site-packages/sklearn/cross_validation.py&quot;, line 402, in __init__<br> len(y), n_folds, indices, shuffle, random_state)<br> File &quot;~/anaconda/lib/python2.7/site-packages/sklearn/cross_validation.py&quot;, line 253, in __init__<br> &quot; than the number of samples: {1}.&quot;).format(n_folds, n))<br>ValueError: Cannot have number of folds n_folds=5 greater than the number of samples: 0.</p>\n<p>At this point, I'm wondering whether my version of anaconda is right.. Has anyone else run into this issue?</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 61740,
      "author_name": "aishwarya229018",
      "author_url": "",
      "post_date": "12/23/2014 09:38:56",
      "content": "<p>The error got cut off- here it is again:</p>\n<p>Traceback (most recent call last):<br> File &quot;data_science.py&quot;, line 145, in module<br> kf = KFold(y, n_folds=5)<br> File &quot;/Users/aishwaryaafzulpurkar/anaconda/lib/python2.7/site-packages/sklearn/cross_validation.py&quot;, line 402, in __init__<br> len(y), n_folds, indices, shuffle, random_state)<br> File &quot;/Users/aishwaryaafzulpurkar/anaconda/lib/python2.7/site-packages/sklearn/cross_validation.py&quot;, line 253, in __init__<br> &quot; than the number of samples: {1}.&quot;).format(n_folds, n))<br>ValueError: Cannot have number of folds n_folds=5 greater than the number of samples: 0.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 61784,
      "author_name": "asaini",
      "author_url": "",
      "post_date": "12/23/2014 22:06:01",
      "content": "<p>@aishwarya</p>\n<p>It seems like a data read could be the problem (number of samples is zero).</p>\n<p>Can you check that the input file was read correctly? Do you have the training data in the directory expected by the script? Hope that helps. Good luck!</p>\n<p>ValueError: <strong>Cannot have number of folds n_folds=5 greater than the number of samples: 0.</strong></p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 61795,
      "author_name": "aishwarya229018",
      "author_url": "",
      "post_date": "12/24/2014 00:24:12",
      "content": "<p>Thanks! Turns out I just had to change the directory name that was being searched..</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 61845,
      "author_name": "asaini",
      "author_url": "",
      "post_date": "12/24/2014 23:20:16",
      "content": "<p>Glad to know that you were able to resolve the problem. Best of luck!</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 61927,
      "author_name": "rastaman",
      "author_url": "",
      "post_date": "12/27/2014 08:24:04",
      "content": "<p>while running top few lines ( till<a href=\"https://www.kaggle.com/c/datasciencebowl/details/tutorial\" target=\"_blank\"> [6] </a>in tutorial) , i am facing this error</p>\n\n<p>&quot; File &quot;C:\\Continuum\\Anaconda\\lib\\site-packages\\PIL\\Image.py&quot;, line 1956, in open</p>\n<p>prefix = fp.read(16)</p>\n<p>AttributeError: 'list' object has no attribute 'read' &quot;</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 61932,
      "author_name": "sander31",
      "author_url": "",
      "post_date": "12/27/2014 14:48:58",
      "content": "<p>Your 'example_file' object is a list returned from glob. You'll need to subset it.&nbsp;</p>\n<p>im = imread(example_file[0], as_grey=True)</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 61963,
      "author_name": "ytk9000",
      "author_url": "",
      "post_date": "12/28/2014 00:20:15",
      "content": "<p>@zzspar - thanks again for the notebook!</p>\n<p>anyone else getting the following error? thanks y'all!</p>\n<p>IndexError Traceback (most recent call last)<br>&lt;ipython-input-4-ff2a4c5acd3d&gt; in &lt;module&gt;()<br> 2 # This example was chosen for because it has two noncontinguous pieces<br> 3 # that will make the segmentation example more illustrative<br>----&gt; 4 example_file = glob.glob(os.path.join(directory_names[5],&quot;*.jpg&quot;))[9]<br> 5 print example_file<br> 6 im = imread(example_file, as_grey=True)</p>\n<p>IndexError: list index out of range</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 61965,
      "author_name": "ytk9000",
      "author_url": "",
      "post_date": "12/28/2014 00:41:01",
      "content": "<p>oops error got cufoff:</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 62085,
      "author_name": "rastaman",
      "author_url": "",
      "post_date": "12/30/2014 05:03:42",
      "content": "<p>@ <a href=\"http://www.kaggle.com/users/88875/aj-buckeye\">Aj_Buckeye</a>&nbsp;Thanks got it working, @&nbsp;&nbsp;<a href=\"http://www.kaggle.com/users/209396/ytk9000\">YTK9000</a>&nbsp;see&nbsp;Aj_Buckeye answer</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 62522,
      "author_name": "zeworst",
      "author_url": "",
      "post_date": "01/09/2015 19:25:06",
      "content": "<p>Hey, guys!<br>I've tried to launch tutorial script several times and in the moment of RF-training kernel of ipython notebook was dying. I'm working in Windows 7, hardware: 8 GB RAM, core i7.</p>\n<p>Task manager showed in moment of RF-training python obtained ~1,1 gb of RAM and 4 gb was free. I've no idea what the reason of happening.</p>\n<p>Could anyone suggest solution&nbsp;of the trouble?</p>\n<p>Thank you for attention!</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 62560,
      "author_name": "kiwisteve",
      "author_url": "",
      "post_date": "01/10/2015 13:15:53",
      "content": "<p>I had the same problem. I switched from the python xy packet which uses the 32 bit python 2.7 version to Winpython for which you can get a 64 bit version of python (in my case I went for&nbsp;WinPython-64bit-3.3.5.4). I know that 32 bit python 2.7 has a limit of 2 gb, not quite sure why the problem occurs at 1.1 gb (was exactly the same for me). But the 64 bit version of python can use more memory. After that everything worked fine for me. &nbsp;</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 62567,
      "author_name": "bluetrin",
      "author_url": "",
      "post_date": "01/10/2015 15:19:40",
      "content": "<p>[quote=SteveNL;62560]</p>\n<p>I had the same problem. I switched from the python xy packet which uses the 32 bit python 2.7 version to Winpython for which you can get a 64 bit version of python (in my case I went for&nbsp;WinPython-64bit-3.3.5.4). I know that 32 bit python 2.7 has a limit of 2 gb, not quite sure why the problem occurs at 1.1 gb (was exactly the same for me). But the 64 bit version of python can use more memory. After that everything worked fine for me. &nbsp;</p>\n<p>[/quote]</p>\n\n<p>Maybe I am wrong but is it because the windows task manager displays the physical memory rather than the virtual memory of the process ?</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "60235": "",
    "60250": "",
    "60264": "",
    "60287": "",
    "60288": "",
    "60290": "",
    "60292": "",
    "60335": "",
    "61498": "",
    "61739": "",
    "61740": "",
    "61784": "",
    "61795": "",
    "61845": "",
    "61927": "",
    "61932": "",
    "61963": "",
    "61965": "",
    "62085": "",
    "62522": "",
    "62560": "",
    "62567": ""
  },
  "source": "meta"
}