{
  "id": 12816,
  "title": "Please delete.",
  "url": "/competitions/datasciencebowl/discussion/12816",
  "author_name": "",
  "post_date": "2015-03-15T07:24:32.230Z",
  "votes": 7,
  "comment_count": 30,
  "views": 5885,
  "content": "<p>Admins please delete.</p>",
  "messages": [
    {
      "id": "66227",
      "postDate": "03/15/2015 07:24:32",
      "content": "<p>Admins please delete.</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "66230",
      "postDate": "03/15/2015 12:10:52",
      "content": "<p>About how much improvement is due to the extra morphological channels you think? It was an idea I had, but did have time to implement.</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "66231",
      "postDate": "03/15/2015 12:21:51",
      "content": "<p>[quote=npetitclerc;66230]</p>\n<p>About how much improvement is due to the extra morphological channels you think? It was an idea I had, but did have time to implement.</p>\n<p>[/quote]</p>\n<p>Much&nbsp;of the improvement in this case is from the extra Convolutional layer, but I'm not exactly sure as I wrote this particular implementation to share only yesterday. So I have not done comparisons to ascertain the contributions of the extra Convolutional layer&nbsp;and the morphological enhancement with this particular shared implementation.</p>\n<p>Although, in my own previous implementations the morphological enhancement did in fact improve performance with statistical significance.&nbsp;I have compared different morphological operations and tophat and bottomhat operations seem to do comparatively well since it highlights features like tentacles and appendages that are affected easily by erosion etc.</p>\n<p>I've also used 4-channel morphologically enhanced images and achieved 79% training accuracy, which is notable since 3-channel images only achieves about 75% training accuracy.</p>\n<p>It overfit with 4-channel images to achieve only a 72% validation accuracy / 79% training accuracy ratio, and I couldn't tune the architecture to hit a sweetspot that beats my own personal 0.79 (74%) Public Leaderboard score. A big limitation of the Dato graphlab deeplearning implementation is that you can have at most 4-channel inputs, so it precludes the simultaneous use of more morphological operations.</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "66258",
      "postDate": "03/15/2015 18:29:10",
      "content": "<p>Thanks for the code. I got an error:</p>\n<p>### end network parameters ###</p>\n<p>PROGRESS: Computing mean image...<br>[ERROR] Toolkit error: Cannot perform sum or average over images of different sizes. Found images of total size (ie. width * height * channels) of both 12288and 4096. Please use graplab.image_analysis.resize() to make images a uniform size.</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "66259",
      "postDate": "03/15/2015 18:36:58",
      "content": "<p>[quote=Steven Du;66258]</p>\n<p>Thanks for the code. I got an error:</p>\n<p>### end network parameters ###</p>\n<p>PROGRESS: Computing mean image...<br>[ERROR] Toolkit error: Cannot perform sum or average over images of different sizes. Found images of total size (ie. width * height * channels) of both 12288and 4096. Please use graplab.image_analysis.resize() to make images a uniform size.</p>\n<p>[/quote]</p>\n<p>If you run gen_train.py and gen_test.py to create a new dataset in Data_morph/ then you should have images that are all sized to 64x64. This can be confirmed by looking at the final command in those preprocessing files.</p>\n<p>In my personal attempts I've used ImageMagick convert to do the morphology operations and image creation, but it seems like skimage.io.imsave compresses the '.jpg' more than ImageMagick does as there is notable mosquito noise with skimage. This is probably not doing the classifier any favors, so you can change gen_train.py and gen_test.py to output '.png' instead of '.jpg' and it should get you closer to 0.80.</p>\n<p>EDIT: I updated the repository with new gen_train.py and gen_test.py files that generate PNG instead of JPG to remove noise. I am testing for new accuracy, but will take +3h so will update score when finished. Remember to delete Data_morph/ before generating with the new gen_*.py files.</p>\n<p>EDIT2: The newest repository generates PNG files instead of noisy JPG, which improved Validation to 0.775 from 0.84. Just note that if you've been running it before 1am UTC 16 March that the submission.csv contains &quot;.jpg.png&quot; row names, which has to be replaced globally with &quot;.jpg&quot;. I have fixed the make_submission.py file to do that automatically since 1am.</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "66320",
      "postDate": "03/16/2015 03:24:25",
      "content": "<p>Thanks, i will test the new code now.</p>\n\n<p>[quote=Unas;66259]</p>\n<p>[quote=Steven Du;66258]</p>\n<p>Thanks for the code. I got an error:</p>\n<p>### end network parameters ###</p>\n<p>PROGRESS: Computing mean image...<br>[ERROR] Toolkit error: Cannot perform sum or average over images of different sizes. Found images of total size (ie. width * height * channels) of both 12288and 4096. Please use graplab.image_analysis.resize() to make images a uniform size.</p>\n<p>[/quote]</p>\n<p>If you run gen_train.py and gen_test.py to create a new dataset in Data_morph/ then you should have images that are all sized to 64x64. This can be confirmed by looking at the final command in those preprocessing files.</p>\n<p>In my personal attempts I've used ImageMagick convert to do the morphology operations and image creation, but it seems like skimage.io.imsave compresses the '.jpg' more than ImageMagick does as there is notable mosquito noise with skimage. This is probably not doing the classifier any favors, so you can change gen_train.py and gen_test.py to output '.png' instead of '.jpg' and it should get you closer to 0.80.</p>\n<p>EDIT: I updated the repository with new gen_train.py and gen_test.py files that generate PNG instead of JPG to remove noise. I am testing for new accuracy, but will take +3h so will update score when finished. Remember to delete Data_morph/ before generating with the new gen_*.py files.</p>\n<p>EDIT2: The newest repository generates PNG files instead of noisy JPG, which improved Validation to 0.775 from 0.84. Just note that if you've been running it before 1am UTC 16 March that the submission.csv contains &quot;.jpg.png&quot; row names, which has to be replaced globally with &quot;.jpg&quot;. I have fixed the make_submission.py file to do that automatically since 1am.</p>\n<p>[/quote]</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "66370",
      "postDate": "03/16/2015 14:32:18",
      "content": "<p>um... &nbsp;I'm worried that deleting a post is not a good way. Somebody has already used that code, like me. And there are others who haven't got a chance to look at&nbsp;it. So it may be unfair for those people. Although I agree late sharing is controversial, personally I may be against it or be silent about it depending on whether it can improve my ensemble.&nbsp;But technically, late sharing doesn't violate any rule. And technically, it is fair since we all can use it and ensemble it.</p>\n<p>But stop sharing it&nbsp;may change things&nbsp;since it is no longer 'public' shared. Will my score&nbsp;be removed because of this? I don't violate any rule but I did take an advantage.</p>\n<p>So I'm&nbsp;just going to share it here. It has nothing to do with Unas. All credits of creating this wonderful code&nbsp;go to Unas&nbsp;of course. Just blame me for&nbsp;sharing it. If this post is indeed deleted, I will post another one.</p>\n<p>Edit: this is the 0.77 one.</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "66378",
      "postDate": "03/16/2015 15:07:33",
      "content": "<p>I also downloaded and&nbsp;forked this code.&nbsp; </p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "66384",
      "postDate": "03/16/2015 15:25:22",
      "content": "<p>Using publicly shared code is within the rules and will not be used against you.</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "66386",
      "postDate": "03/16/2015 15:38:27",
      "content": "<p>Why don't you guys who have already used this code post your submission file? So that people without GPU can also get your scores.</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "66387",
      "postDate": "03/16/2015 15:41:06",
      "content": "<p>[quote=Sachin;66386]</p>\n<p>Why don't you guys who have already used this code post your submission file? So that people without GPU can also get your scores.</p>\n<p>[/quote]</p>\n<p>make the code run is not trivial to me. it's part of the learning process.</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "66391",
      "postDate": "03/16/2015 15:53:52",
      "content": "<p>[quote=rcarson;66387]</p>\n<p>[quote=Sachin;66386]</p>\n<p>Why don't you guys who have already used this code post your submission file? So that people without GPU can also get your scores.</p>\n<p>[/quote]</p>\n<p>make the code run is not trivial to me. it's part of the learning process.</p>\n<p>[/quote]</p>\n<p>I respect the author for the wonderful code he has written. The point here is that the time when this code was shared, just a day left before the close of the competition! But this is really disappointing for people who have worked for several days on this project.</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "66393",
      "postDate": "03/16/2015 15:58:25",
      "content": "<p>[quote=Sachin;66386]</p>\n<p>Why don't you guys who have already used this code post your submission file? So that people without GPU can also get your scores.</p>\n<p>[/quote]</p>\n<p>Perhaps @Sachin is being sarcastic?</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "66394",
      "postDate": "03/16/2015 16:01:37",
      "content": "<p>[quote=Sachin;66391]</p>\n<p>I respect the author for the wonderful code he has written. The point here is that the time when this code was shared, just a day left before the close of the competition! But this is really disappointing for people who have worked for several days on this project.</p>\n<p>[/quote]</p>\n<p>All credit to&nbsp;Srikrishna Sridhar, I just made very basic modifications to his excellent code.</p>\n<p>Suffice it to say, I won't share code in this manner again because I see how it&nbsp;can be unfair. Apologies.</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "66395",
      "postDate": "03/16/2015 16:03:34",
      "content": "<p>@inversion</p>\n<p>You are right, I am being sarcastic because I have spent so much time on this competition not having managed to get a GPU.</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "66408",
      "postDate": "03/16/2015 17:24:07",
      "content": "<p>Although Sachin's comment seems silly, sharing this code is only a few steps away from simply sharing the submission file.</p>\n<p>I ran the code myself, and it was easy. &nbsp;This competition isn't about getting GraphLab to install. &nbsp;You might as well have just handed me the submission file.</p>\n<p>@rcarson - it sounds like you still value the idea that we have to do some work in order to generate a submission; but it seems as if you've conveniently been dropped off 3 steps from your destination by Unas' helicopter, then said &quot;I know most of you have no helicopter, but those last 3 steps are some work too, so we should not share the results of those last 3 steps in order to preserve the integrity of the competition&quot;</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "66413",
      "postDate": "03/16/2015 17:26:35",
      "content": "<p>[quote=Unas;66394]</p>\n<p>[quote=Sachin;66391]</p>\n<p>I respect the author for the wonderful code he has written. The point here is that the time when this code was shared, just a day left before the close of the competition! But this is really disappointing for people who have worked for several days on this project.</p>\n<p>[/quote]</p>\n<p>All credit to&nbsp;Srikrishna Sridhar, I just made very basic modifications to his excellent code.</p>\n<p>Suffice it to say, I won't share code in this manner again because I see how it&nbsp;can be unfair. Apologies.</p>\n<p>[/quote]</p>\n\n<p>Honestly Unas, I'm personally not blaming you - I thought it appeared that you just found Srikri's post and wanted to share back an improvement you found. &nbsp;</p>\n<p>The main problem is Srikri's original post just a few days before the deadline. &nbsp;I think yours, Unas, was just in the spirit of the community given Sriki's post.</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "66414",
      "postDate": "03/16/2015 17:32:00",
      "content": "<p>[quote=Sachin;66395]</p>\n<p>@inversion</p>\n<p>You are right, I am being sarcastic because I have spent so much time on this competition not having managed to get a GPU.</p>\n<p>[/quote]</p>\n<p>I bought 2 GPUs because of the contest. I spent more time getting things to run on them than I did actually running code.</p>\n<p>It was worth the time though.</p>\n<p>I went from MLP on the CPU (using this <a href=\"https://github.com/IssamLaradji/NeuralNetworks\" target=\"_blank\">toolbox</a>), to MLP on the GPU (using <a href=\"https://github.com/hannes-brt/hebel\" target=\"_blank\">Hebel</a>), to using CNN on the GPU with <a href=\"https://github.com/dnouri/nolearn\" target=\"_blank\">NoLearn/Lasagne</a>.</p>\n<p>There's a lot I need to learn regarding CNNs. But, they look like a lot of fun.</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "66417",
      "postDate": "03/16/2015 17:46:04",
      "content": "<p>[quote=Phillip Chilton Adkins;66408]</p>\n<p>@rcarson - it sounds like you still value the idea that we have to do some work in order to generate a submission; but it seems as if you've conveniently been dropped off 3 steps from your destination by Unas' helicopter, then said &quot;I know most of you have no helicopter, but those last 3 steps are some work too, so we should not share the results of those last 3 steps in order to preserve the integrity of the competition&quot;</p>\n<p>[/quote]</p>\n<p>So I should share the submission?</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "66419",
      "postDate": "03/16/2015 17:51:08",
      "content": "<p>@inversion, A ha, nice to meet you here again : ). It seems that we are in the same favor of Nolearn/Lasagne. Besides you just mentioned Hebel and I check it in the Github. I have never heard of such Lib before, not to mention use it. So would you like to simply point out the advantages/disadvantages between hebel and nolearn ?</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "66428",
      "postDate": "03/16/2015 18:03:06",
      "content": "<p>[quote=inversion;66414]</p>\n<p>[quote=Sachin;66395]</p>\n<p>@inversion</p>\n<p>You are right, I am being sarcastic because I have spent so much time on this competition not having managed to get a GPU.</p>\n<p>[/quote]</p>\n<p>I bought 2 GPUs because of the contest. I spent more time getting things to run on them than I did actually running code.</p>\n<p>It was worth the time though.</p>\n<p>I went from MLP on the CPU (using this <a href=\"https://github.com/IssamLaradji/NeuralNetworks\" target=\"_blank\">toolbox</a>), to MLP on the GPU (using <a href=\"https://github.com/hannes-brt/hebel\" target=\"_blank\">Hebel</a>), to using CNN on the GPU with <a href=\"https://github.com/dnouri/nolearn\" target=\"_blank\">NoLearn/Lasagne</a>.</p>\n<p>There's a lot I need to learn regarding CNNs. But, they look like a lot of fun.</p>\n<p>[/quote]</p>\n<p>True said, this is my fist competition where I have put good amount of effort and I have learnt a lot.</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "66433",
      "postDate": "03/16/2015 18:35:49",
      "content": "<p>[quote=PengPai;66419]</p>\n<p>@inversion, A ha, nice to meet you here again : ). It seems that we are in the same favor of Nolearn/Lasagne. Besides you just mentioned Hebel and I check it in the Github. I have never heard of such Lib before, not to mention use it. So would you like to simply point out the advantages/disadvantages between hebel and nolearn ?</p>\n<p>[/quote]</p>\n<p>Hebel doesn't have any advantages over nolearn, except it is a bit less complicated. You can set up nolearn to have the exact same architecture as what hebel provides.&nbsp;</p>\n<p>Although, now that I think about it, Hebel does have Multi Task learning. I haven't tried it, although it might have been useful for this competition.</p>\n<p>EDIT: The only reason I used Hebel is because it was a &quot;baby step&quot; for me to go to NoLearn/Lasagne.</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "66438",
      "postDate": "03/16/2015 18:50:25",
      "content": "<p>@rcarson</p>\n<p>if some people only cannot run the code because they have no GPU, then in the spirit of cooperation: maybe</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "66439",
      "postDate": "03/16/2015 18:50:55",
      "content": "<p>[quote=inversion;66433]</p>\n<p>[quote=PengPai;66419]</p>\n<p>@inversion, A ha, nice to meet you here again : ). It seems that we are in the same favor of Nolearn/Lasagne. Besides you just mentioned Hebel and I check it in the Github. I have never heard of such Lib before, not to mention use it. So would you like to simply point out the advantages/disadvantages between hebel and nolearn ?</p>\n<p>[/quote]</p>\n<p>Hebel doesn't have any advantages over nolearn, except it is a bit less complicated. You can set up nolearn to have the exact same architecture as what hebel provides.&nbsp;</p>\n<p>Although, now that I think about it, Hebel does have Multi Task learning. I haven't tried it, although it might have been useful for this competition.</p>\n<p>EDIT: The only reason I used Hebel is because it was a &quot;baby step&quot; for me to go to NoLearn/Lasagne.</p>\n<p>[/quote]</p>\n<p>Thank you for your sharing, inversion. Then, I will stick with Nolearn. I think one disadvantage of Nolearn is: when the data size it too large to load into memory, how could we invoke net.fit() in a right way? Theoretically, we could split training data into multiple shares and call fit function multiple times. But when I tried in practice, my validation accuracy keeps decreasing, starts from the second calling.</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "66444",
      "postDate": "03/16/2015 19:02:07",
      "content": "<p>[quote=rcarson;66417]</p>\n<p>[quote=Phillip Chilton Adkins;66408]</p>\n<p>@rcarson - it sounds like you still value the idea that we have to do some work in order to generate a submission; but it seems as if you've conveniently been dropped off 3 steps from your destination by Unas' helicopter, then said &quot;I know most of you have no helicopter, but those last 3 steps are some work too, so we should not share the results of those last 3 steps in order to preserve the integrity of the competition&quot;</p>\n<p>[/quote]</p>\n<p>So I should share the submission?</p>\n<p>[/quote]</p>\n\n<p>You would just be penalizing everyone with a higher score that cannot submit within the last few hours. There is a reason there is a clear timeline for the competition, don't ruin it for others.</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "66445",
      "postDate": "03/16/2015 19:09:28",
      "content": "<p>@Phillip Chilton Adkins @ dkay</p>\n<p>I won't share it then. Thank you. You all made good points.</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "66453",
      "postDate": "03/16/2015 19:19:02",
      "content": "<p>@rcarson</p>\n\n<p>If you think dkay has a point about not sharing the submission so close to the deadline, then you must be against sharing the code that generates such a submission so close to the deadline for just the same reason. &nbsp;</p>\n<p>The difference between getting the csv to upload and typing &quot;python make_submission.py&quot; in the console and THEN getting the csv to upload is trivial in most cases. &nbsp;</p>\n<p>Unas / Shri??? essentially shared the .csv</p>\n<p>And the only reason we don't have a flood of &nbsp;.8 and .7 on the LB is that most people probably aren't reading this forum - just as was argued, penalizing everyone that cannot submit within the last few hours.</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "66454",
      "postDate": "03/16/2015 19:20:31",
      "content": "<p>[quote=PengPai;66439]</p>\n<p>Thank you for your sharing, inversion. Then, I will stick with Nolearn. I think one disadvantage of Nolearn is: when the data size it too large to load into memory, how could we invoke net.fit() in a right way? Theoretically, we could split training data into multiple shares and call fit function multiple times. But when I tried in practice, my validation accuracy keeps decreasing, starts from the second calling.</p>\n<p>[/quote]</p>\n<p>The BatchIterator should take care of that (in your neural network definition).</p>\n<p><code>batch_iterator_train=BatchIterator(batch_size=128),</code></p>\n\n<p>Also . . . have you tried reducing your learning rate?</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "66464",
      "postDate": "03/16/2015 19:47:07",
      "content": "<p>[quote=Phillip Chilton Adkins;66453]</p>\n<p>@rcarson</p>\n<p>If you think dkay has a point about not sharing the submission so close to the deadline, then you must be against sharing the code that generates such a submission so close to the deadline for just the same reason. &nbsp;</p>\n<p>The difference between getting the csv to upload and typing &quot;python make_submission.py&quot; in the console and THEN getting the csv to upload is trivial in most cases. &nbsp;</p>\n<p>Unas / Shri??? essentially shared the .csv</p>\n<p>And the only reason we don't have a flood of &nbsp;.8 and .7 on the LB is that most people probably aren't reading this forum - just as was argued, penalizing everyone that cannot submit within the last few hours.</p>\n<p>[/quote]</p>\n<p>What I shared was there before.&nbsp;The code&nbsp;is there for two days now. Not a couple of hours. For a player who don't use GPU, I can't see the difference it is posted&nbsp;two days or two months before the deadline, since everyone is well aware of the performance difference of GPU and CPU from the cxxnet benchmark. For a player with GPU, two days is enough to make some tuning and ensemble. It is even enough for someone to order a gpu card from amazon, choose one day shipping, install it and set up everything from scratch&nbsp;and still has time to&nbsp;further improve the score&nbsp;based on this code.</p>\n<p>For those who don't look at forum, I feel sorry.</p>\n\n<p>So there is a difference between sharing a code yesterday and sharing a submission now. I'm just keeping that code shared, but not the submission.</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "66486",
      "postDate": "03/16/2015 20:30:00",
      "content": "<p>@rcarson</p>\n<p>I'm not actually arguing for you to share it. &nbsp;In fact, it's totally dumb to share it.&nbsp;</p>\n<p>I just want it acknowledged that it's about 95% the same&nbsp;to share the code that generates the submission as sharing the submission itself; and if you feel that one is fine, that it would be somewhat inconsistent to say that the other is not. &nbsp;It's a little disingenuous / symptomatic of a conflict of interest to say that that 5% is the difference between a great idea and a terrible idea, when that 5% difference might just be installing a package and typing &quot;python make_submission.py&quot; into the console.</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "66497",
      "postDate": "03/16/2015 20:58:01",
      "content": "<p>@Phillip Chilton Adkins. You are right. I'm sorry. I fully agree&nbsp;that it's almost same except that&nbsp;code has more potential to exploit if someone want to. I'm not&nbsp;even sure I'm making more damage or less. Anyway, good luck to everyone. As one of many, I really want to learn&nbsp;deep learning and look forward to&nbsp;the post contest solution sharing.&nbsp;</p>",
      "rawMarkdown": "",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 66230,
      "author_name": "npetitclerc",
      "author_url": "",
      "post_date": "03/15/2015 12:10:52",
      "content": "<p>About how much improvement is due to the extra morphological channels you think? It was an idea I had, but did have time to implement.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 66231,
      "author_name": "unas252547",
      "author_url": "",
      "post_date": "03/15/2015 12:21:51",
      "content": "<p>[quote=npetitclerc;66230]</p>\n<p>About how much improvement is due to the extra morphological channels you think? It was an idea I had, but did have time to implement.</p>\n<p>[/quote]</p>\n<p>Much&nbsp;of the improvement in this case is from the extra Convolutional layer, but I'm not exactly sure as I wrote this particular implementation to share only yesterday. So I have not done comparisons to ascertain the contributions of the extra Convolutional layer&nbsp;and the morphological enhancement with this particular shared implementation.</p>\n<p>Although, in my own previous implementations the morphological enhancement did in fact improve performance with statistical significance.&nbsp;I have compared different morphological operations and tophat and bottomhat operations seem to do comparatively well since it highlights features like tentacles and appendages that are affected easily by erosion etc.</p>\n<p>I've also used 4-channel morphologically enhanced images and achieved 79% training accuracy, which is notable since 3-channel images only achieves about 75% training accuracy.</p>\n<p>It overfit with 4-channel images to achieve only a 72% validation accuracy / 79% training accuracy ratio, and I couldn't tune the architecture to hit a sweetspot that beats my own personal 0.79 (74%) Public Leaderboard score. A big limitation of the Dato graphlab deeplearning implementation is that you can have at most 4-channel inputs, so it precludes the simultaneous use of more morphological operations.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 66258,
      "author_name": "stevendu",
      "author_url": "",
      "post_date": "03/15/2015 18:29:10",
      "content": "<p>Thanks for the code. I got an error:</p>\n<p>### end network parameters ###</p>\n<p>PROGRESS: Computing mean image...<br>[ERROR] Toolkit error: Cannot perform sum or average over images of different sizes. Found images of total size (ie. width * height * channels) of both 12288and 4096. Please use graplab.image_analysis.resize() to make images a uniform size.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 66259,
      "author_name": "unas252547",
      "author_url": "",
      "post_date": "03/15/2015 18:36:58",
      "content": "<p>[quote=Steven Du;66258]</p>\n<p>Thanks for the code. I got an error:</p>\n<p>### end network parameters ###</p>\n<p>PROGRESS: Computing mean image...<br>[ERROR] Toolkit error: Cannot perform sum or average over images of different sizes. Found images of total size (ie. width * height * channels) of both 12288and 4096. Please use graplab.image_analysis.resize() to make images a uniform size.</p>\n<p>[/quote]</p>\n<p>If you run gen_train.py and gen_test.py to create a new dataset in Data_morph/ then you should have images that are all sized to 64x64. This can be confirmed by looking at the final command in those preprocessing files.</p>\n<p>In my personal attempts I've used ImageMagick convert to do the morphology operations and image creation, but it seems like skimage.io.imsave compresses the '.jpg' more than ImageMagick does as there is notable mosquito noise with skimage. This is probably not doing the classifier any favors, so you can change gen_train.py and gen_test.py to output '.png' instead of '.jpg' and it should get you closer to 0.80.</p>\n<p>EDIT: I updated the repository with new gen_train.py and gen_test.py files that generate PNG instead of JPG to remove noise. I am testing for new accuracy, but will take +3h so will update score when finished. Remember to delete Data_morph/ before generating with the new gen_*.py files.</p>\n<p>EDIT2: The newest repository generates PNG files instead of noisy JPG, which improved Validation to 0.775 from 0.84. Just note that if you've been running it before 1am UTC 16 March that the submission.csv contains &quot;.jpg.png&quot; row names, which has to be replaced globally with &quot;.jpg&quot;. I have fixed the make_submission.py file to do that automatically since 1am.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 66320,
      "author_name": "stevendu",
      "author_url": "",
      "post_date": "03/16/2015 03:24:25",
      "content": "<p>Thanks, i will test the new code now.</p>\n\n<p>[quote=Unas;66259]</p>\n<p>[quote=Steven Du;66258]</p>\n<p>Thanks for the code. I got an error:</p>\n<p>### end network parameters ###</p>\n<p>PROGRESS: Computing mean image...<br>[ERROR] Toolkit error: Cannot perform sum or average over images of different sizes. Found images of total size (ie. width * height * channels) of both 12288and 4096. Please use graplab.image_analysis.resize() to make images a uniform size.</p>\n<p>[/quote]</p>\n<p>If you run gen_train.py and gen_test.py to create a new dataset in Data_morph/ then you should have images that are all sized to 64x64. This can be confirmed by looking at the final command in those preprocessing files.</p>\n<p>In my personal attempts I've used ImageMagick convert to do the morphology operations and image creation, but it seems like skimage.io.imsave compresses the '.jpg' more than ImageMagick does as there is notable mosquito noise with skimage. This is probably not doing the classifier any favors, so you can change gen_train.py and gen_test.py to output '.png' instead of '.jpg' and it should get you closer to 0.80.</p>\n<p>EDIT: I updated the repository with new gen_train.py and gen_test.py files that generate PNG instead of JPG to remove noise. I am testing for new accuracy, but will take +3h so will update score when finished. Remember to delete Data_morph/ before generating with the new gen_*.py files.</p>\n<p>EDIT2: The newest repository generates PNG files instead of noisy JPG, which improved Validation to 0.775 from 0.84. Just note that if you've been running it before 1am UTC 16 March that the submission.csv contains &quot;.jpg.png&quot; row names, which has to be replaced globally with &quot;.jpg&quot;. I have fixed the make_submission.py file to do that automatically since 1am.</p>\n<p>[/quote]</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 66370,
      "author_name": "jiweiliu",
      "author_url": "",
      "post_date": "03/16/2015 14:32:18",
      "content": "<p>um... &nbsp;I'm worried that deleting a post is not a good way. Somebody has already used that code, like me. And there are others who haven't got a chance to look at&nbsp;it. So it may be unfair for those people. Although I agree late sharing is controversial, personally I may be against it or be silent about it depending on whether it can improve my ensemble.&nbsp;But technically, late sharing doesn't violate any rule. And technically, it is fair since we all can use it and ensemble it.</p>\n<p>But stop sharing it&nbsp;may change things&nbsp;since it is no longer 'public' shared. Will my score&nbsp;be removed because of this? I don't violate any rule but I did take an advantage.</p>\n<p>So I'm&nbsp;just going to share it here. It has nothing to do with Unas. All credits of creating this wonderful code&nbsp;go to Unas&nbsp;of course. Just blame me for&nbsp;sharing it. If this post is indeed deleted, I will post another one.</p>\n<p>Edit: this is the 0.77 one.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 66378,
      "author_name": "stevendu",
      "author_url": "",
      "post_date": "03/16/2015 15:07:33",
      "content": "<p>I also downloaded and&nbsp;forked this code.&nbsp; </p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 66384,
      "author_name": "wcukierski",
      "author_url": "",
      "post_date": "03/16/2015 15:25:22",
      "content": "<p>Using publicly shared code is within the rules and will not be used against you.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 66386,
      "author_name": "sachinvernekar",
      "author_url": "",
      "post_date": "03/16/2015 15:38:27",
      "content": "<p>Why don't you guys who have already used this code post your submission file? So that people without GPU can also get your scores.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 66387,
      "author_name": "jiweiliu",
      "author_url": "",
      "post_date": "03/16/2015 15:41:06",
      "content": "<p>[quote=Sachin;66386]</p>\n<p>Why don't you guys who have already used this code post your submission file? So that people without GPU can also get your scores.</p>\n<p>[/quote]</p>\n<p>make the code run is not trivial to me. it's part of the learning process.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 66391,
      "author_name": "sachinvernekar",
      "author_url": "",
      "post_date": "03/16/2015 15:53:52",
      "content": "<p>[quote=rcarson;66387]</p>\n<p>[quote=Sachin;66386]</p>\n<p>Why don't you guys who have already used this code post your submission file? So that people without GPU can also get your scores.</p>\n<p>[/quote]</p>\n<p>make the code run is not trivial to me. it's part of the learning process.</p>\n<p>[/quote]</p>\n<p>I respect the author for the wonderful code he has written. The point here is that the time when this code was shared, just a day left before the close of the competition! But this is really disappointing for people who have worked for several days on this project.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 66393,
      "author_name": "inversion",
      "author_url": "",
      "post_date": "03/16/2015 15:58:25",
      "content": "<p>[quote=Sachin;66386]</p>\n<p>Why don't you guys who have already used this code post your submission file? So that people without GPU can also get your scores.</p>\n<p>[/quote]</p>\n<p>Perhaps @Sachin is being sarcastic?</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 66394,
      "author_name": "unas252547",
      "author_url": "",
      "post_date": "03/16/2015 16:01:37",
      "content": "<p>[quote=Sachin;66391]</p>\n<p>I respect the author for the wonderful code he has written. The point here is that the time when this code was shared, just a day left before the close of the competition! But this is really disappointing for people who have worked for several days on this project.</p>\n<p>[/quote]</p>\n<p>All credit to&nbsp;Srikrishna Sridhar, I just made very basic modifications to his excellent code.</p>\n<p>Suffice it to say, I won't share code in this manner again because I see how it&nbsp;can be unfair. Apologies.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 66395,
      "author_name": "sachinvernekar",
      "author_url": "",
      "post_date": "03/16/2015 16:03:34",
      "content": "<p>@inversion</p>\n<p>You are right, I am being sarcastic because I have spent so much time on this competition not having managed to get a GPU.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 66408,
      "author_name": "phillipadkins",
      "author_url": "",
      "post_date": "03/16/2015 17:24:07",
      "content": "<p>Although Sachin's comment seems silly, sharing this code is only a few steps away from simply sharing the submission file.</p>\n<p>I ran the code myself, and it was easy. &nbsp;This competition isn't about getting GraphLab to install. &nbsp;You might as well have just handed me the submission file.</p>\n<p>@rcarson - it sounds like you still value the idea that we have to do some work in order to generate a submission; but it seems as if you've conveniently been dropped off 3 steps from your destination by Unas' helicopter, then said &quot;I know most of you have no helicopter, but those last 3 steps are some work too, so we should not share the results of those last 3 steps in order to preserve the integrity of the competition&quot;</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 66413,
      "author_name": "phillipadkins",
      "author_url": "",
      "post_date": "03/16/2015 17:26:35",
      "content": "<p>[quote=Unas;66394]</p>\n<p>[quote=Sachin;66391]</p>\n<p>I respect the author for the wonderful code he has written. The point here is that the time when this code was shared, just a day left before the close of the competition! But this is really disappointing for people who have worked for several days on this project.</p>\n<p>[/quote]</p>\n<p>All credit to&nbsp;Srikrishna Sridhar, I just made very basic modifications to his excellent code.</p>\n<p>Suffice it to say, I won't share code in this manner again because I see how it&nbsp;can be unfair. Apologies.</p>\n<p>[/quote]</p>\n\n<p>Honestly Unas, I'm personally not blaming you - I thought it appeared that you just found Srikri's post and wanted to share back an improvement you found. &nbsp;</p>\n<p>The main problem is Srikri's original post just a few days before the deadline. &nbsp;I think yours, Unas, was just in the spirit of the community given Sriki's post.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 66414,
      "author_name": "inversion",
      "author_url": "",
      "post_date": "03/16/2015 17:32:00",
      "content": "<p>[quote=Sachin;66395]</p>\n<p>@inversion</p>\n<p>You are right, I am being sarcastic because I have spent so much time on this competition not having managed to get a GPU.</p>\n<p>[/quote]</p>\n<p>I bought 2 GPUs because of the contest. I spent more time getting things to run on them than I did actually running code.</p>\n<p>It was worth the time though.</p>\n<p>I went from MLP on the CPU (using this <a href=\"https://github.com/IssamLaradji/NeuralNetworks\" target=\"_blank\">toolbox</a>), to MLP on the GPU (using <a href=\"https://github.com/hannes-brt/hebel\" target=\"_blank\">Hebel</a>), to using CNN on the GPU with <a href=\"https://github.com/dnouri/nolearn\" target=\"_blank\">NoLearn/Lasagne</a>.</p>\n<p>There's a lot I need to learn regarding CNNs. But, they look like a lot of fun.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 66417,
      "author_name": "jiweiliu",
      "author_url": "",
      "post_date": "03/16/2015 17:46:04",
      "content": "<p>[quote=Phillip Chilton Adkins;66408]</p>\n<p>@rcarson - it sounds like you still value the idea that we have to do some work in order to generate a submission; but it seems as if you've conveniently been dropped off 3 steps from your destination by Unas' helicopter, then said &quot;I know most of you have no helicopter, but those last 3 steps are some work too, so we should not share the results of those last 3 steps in order to preserve the integrity of the competition&quot;</p>\n<p>[/quote]</p>\n<p>So I should share the submission?</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 66419,
      "author_name": "pengpai",
      "author_url": "",
      "post_date": "03/16/2015 17:51:08",
      "content": "<p>@inversion, A ha, nice to meet you here again : ). It seems that we are in the same favor of Nolearn/Lasagne. Besides you just mentioned Hebel and I check it in the Github. I have never heard of such Lib before, not to mention use it. So would you like to simply point out the advantages/disadvantages between hebel and nolearn ?</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 66428,
      "author_name": "sachinvernekar",
      "author_url": "",
      "post_date": "03/16/2015 18:03:06",
      "content": "<p>[quote=inversion;66414]</p>\n<p>[quote=Sachin;66395]</p>\n<p>@inversion</p>\n<p>You are right, I am being sarcastic because I have spent so much time on this competition not having managed to get a GPU.</p>\n<p>[/quote]</p>\n<p>I bought 2 GPUs because of the contest. I spent more time getting things to run on them than I did actually running code.</p>\n<p>It was worth the time though.</p>\n<p>I went from MLP on the CPU (using this <a href=\"https://github.com/IssamLaradji/NeuralNetworks\" target=\"_blank\">toolbox</a>), to MLP on the GPU (using <a href=\"https://github.com/hannes-brt/hebel\" target=\"_blank\">Hebel</a>), to using CNN on the GPU with <a href=\"https://github.com/dnouri/nolearn\" target=\"_blank\">NoLearn/Lasagne</a>.</p>\n<p>There's a lot I need to learn regarding CNNs. But, they look like a lot of fun.</p>\n<p>[/quote]</p>\n<p>True said, this is my fist competition where I have put good amount of effort and I have learnt a lot.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 66433,
      "author_name": "inversion",
      "author_url": "",
      "post_date": "03/16/2015 18:35:49",
      "content": "<p>[quote=PengPai;66419]</p>\n<p>@inversion, A ha, nice to meet you here again : ). It seems that we are in the same favor of Nolearn/Lasagne. Besides you just mentioned Hebel and I check it in the Github. I have never heard of such Lib before, not to mention use it. So would you like to simply point out the advantages/disadvantages between hebel and nolearn ?</p>\n<p>[/quote]</p>\n<p>Hebel doesn't have any advantages over nolearn, except it is a bit less complicated. You can set up nolearn to have the exact same architecture as what hebel provides.&nbsp;</p>\n<p>Although, now that I think about it, Hebel does have Multi Task learning. I haven't tried it, although it might have been useful for this competition.</p>\n<p>EDIT: The only reason I used Hebel is because it was a &quot;baby step&quot; for me to go to NoLearn/Lasagne.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 66438,
      "author_name": "phillipadkins",
      "author_url": "",
      "post_date": "03/16/2015 18:50:25",
      "content": "<p>@rcarson</p>\n<p>if some people only cannot run the code because they have no GPU, then in the spirit of cooperation: maybe</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 66439,
      "author_name": "pengpai",
      "author_url": "",
      "post_date": "03/16/2015 18:50:55",
      "content": "<p>[quote=inversion;66433]</p>\n<p>[quote=PengPai;66419]</p>\n<p>@inversion, A ha, nice to meet you here again : ). It seems that we are in the same favor of Nolearn/Lasagne. Besides you just mentioned Hebel and I check it in the Github. I have never heard of such Lib before, not to mention use it. So would you like to simply point out the advantages/disadvantages between hebel and nolearn ?</p>\n<p>[/quote]</p>\n<p>Hebel doesn't have any advantages over nolearn, except it is a bit less complicated. You can set up nolearn to have the exact same architecture as what hebel provides.&nbsp;</p>\n<p>Although, now that I think about it, Hebel does have Multi Task learning. I haven't tried it, although it might have been useful for this competition.</p>\n<p>EDIT: The only reason I used Hebel is because it was a &quot;baby step&quot; for me to go to NoLearn/Lasagne.</p>\n<p>[/quote]</p>\n<p>Thank you for your sharing, inversion. Then, I will stick with Nolearn. I think one disadvantage of Nolearn is: when the data size it too large to load into memory, how could we invoke net.fit() in a right way? Theoretically, we could split training data into multiple shares and call fit function multiple times. But when I tried in practice, my validation accuracy keeps decreasing, starts from the second calling.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 66444,
      "author_name": "dkaylor",
      "author_url": "",
      "post_date": "03/16/2015 19:02:07",
      "content": "<p>[quote=rcarson;66417]</p>\n<p>[quote=Phillip Chilton Adkins;66408]</p>\n<p>@rcarson - it sounds like you still value the idea that we have to do some work in order to generate a submission; but it seems as if you've conveniently been dropped off 3 steps from your destination by Unas' helicopter, then said &quot;I know most of you have no helicopter, but those last 3 steps are some work too, so we should not share the results of those last 3 steps in order to preserve the integrity of the competition&quot;</p>\n<p>[/quote]</p>\n<p>So I should share the submission?</p>\n<p>[/quote]</p>\n\n<p>You would just be penalizing everyone with a higher score that cannot submit within the last few hours. There is a reason there is a clear timeline for the competition, don't ruin it for others.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 66445,
      "author_name": "jiweiliu",
      "author_url": "",
      "post_date": "03/16/2015 19:09:28",
      "content": "<p>@Phillip Chilton Adkins @ dkay</p>\n<p>I won't share it then. Thank you. You all made good points.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 66453,
      "author_name": "phillipadkins",
      "author_url": "",
      "post_date": "03/16/2015 19:19:02",
      "content": "<p>@rcarson</p>\n\n<p>If you think dkay has a point about not sharing the submission so close to the deadline, then you must be against sharing the code that generates such a submission so close to the deadline for just the same reason. &nbsp;</p>\n<p>The difference between getting the csv to upload and typing &quot;python make_submission.py&quot; in the console and THEN getting the csv to upload is trivial in most cases. &nbsp;</p>\n<p>Unas / Shri??? essentially shared the .csv</p>\n<p>And the only reason we don't have a flood of &nbsp;.8 and .7 on the LB is that most people probably aren't reading this forum - just as was argued, penalizing everyone that cannot submit within the last few hours.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 66454,
      "author_name": "inversion",
      "author_url": "",
      "post_date": "03/16/2015 19:20:31",
      "content": "<p>[quote=PengPai;66439]</p>\n<p>Thank you for your sharing, inversion. Then, I will stick with Nolearn. I think one disadvantage of Nolearn is: when the data size it too large to load into memory, how could we invoke net.fit() in a right way? Theoretically, we could split training data into multiple shares and call fit function multiple times. But when I tried in practice, my validation accuracy keeps decreasing, starts from the second calling.</p>\n<p>[/quote]</p>\n<p>The BatchIterator should take care of that (in your neural network definition).</p>\n<p><code>batch_iterator_train=BatchIterator(batch_size=128),</code></p>\n\n<p>Also . . . have you tried reducing your learning rate?</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 66464,
      "author_name": "jiweiliu",
      "author_url": "",
      "post_date": "03/16/2015 19:47:07",
      "content": "<p>[quote=Phillip Chilton Adkins;66453]</p>\n<p>@rcarson</p>\n<p>If you think dkay has a point about not sharing the submission so close to the deadline, then you must be against sharing the code that generates such a submission so close to the deadline for just the same reason. &nbsp;</p>\n<p>The difference between getting the csv to upload and typing &quot;python make_submission.py&quot; in the console and THEN getting the csv to upload is trivial in most cases. &nbsp;</p>\n<p>Unas / Shri??? essentially shared the .csv</p>\n<p>And the only reason we don't have a flood of &nbsp;.8 and .7 on the LB is that most people probably aren't reading this forum - just as was argued, penalizing everyone that cannot submit within the last few hours.</p>\n<p>[/quote]</p>\n<p>What I shared was there before.&nbsp;The code&nbsp;is there for two days now. Not a couple of hours. For a player who don't use GPU, I can't see the difference it is posted&nbsp;two days or two months before the deadline, since everyone is well aware of the performance difference of GPU and CPU from the cxxnet benchmark. For a player with GPU, two days is enough to make some tuning and ensemble. It is even enough for someone to order a gpu card from amazon, choose one day shipping, install it and set up everything from scratch&nbsp;and still has time to&nbsp;further improve the score&nbsp;based on this code.</p>\n<p>For those who don't look at forum, I feel sorry.</p>\n\n<p>So there is a difference between sharing a code yesterday and sharing a submission now. I'm just keeping that code shared, but not the submission.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 66486,
      "author_name": "phillipadkins",
      "author_url": "",
      "post_date": "03/16/2015 20:30:00",
      "content": "<p>@rcarson</p>\n<p>I'm not actually arguing for you to share it. &nbsp;In fact, it's totally dumb to share it.&nbsp;</p>\n<p>I just want it acknowledged that it's about 95% the same&nbsp;to share the code that generates the submission as sharing the submission itself; and if you feel that one is fine, that it would be somewhat inconsistent to say that the other is not. &nbsp;It's a little disingenuous / symptomatic of a conflict of interest to say that that 5% is the difference between a great idea and a terrible idea, when that 5% difference might just be installing a package and typing &quot;python make_submission.py&quot; into the console.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 66497,
      "author_name": "jiweiliu",
      "author_url": "",
      "post_date": "03/16/2015 20:58:01",
      "content": "<p>@Phillip Chilton Adkins. You are right. I'm sorry. I fully agree&nbsp;that it's almost same except that&nbsp;code has more potential to exploit if someone want to. I'm not&nbsp;even sure I'm making more damage or less. Anyway, good luck to everyone. As one of many, I really want to learn&nbsp;deep learning and look forward to&nbsp;the post contest solution sharing.&nbsp;</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "66227": "",
    "66230": "",
    "66231": "",
    "66258": "",
    "66259": "",
    "66320": "",
    "66370": "",
    "66378": "",
    "66384": "",
    "66386": "",
    "66387": "",
    "66391": "",
    "66393": "",
    "66394": "",
    "66395": "",
    "66408": "",
    "66413": "",
    "66414": "",
    "66417": "",
    "66419": "",
    "66428": "",
    "66433": "",
    "66438": "",
    "66439": "",
    "66444": "",
    "66445": "",
    "66453": "",
    "66454": "",
    "66464": "",
    "66486": "",
    "66497": ""
  },
  "source": "meta"
}