{ "id": 38097, "title": "3rd-Place Solution Overview", "content": "

Hello everyone,

\n\n

First off, thanks to the competition organizers for a clean, interesting, and leak-free dataset and congrats to the winners!

\n\n

My solution was heavily dependent upon the use of neural networks for feature extraction. I decided not to add any manual feature engineering since I thought it would be cool to try to win a competition with pure deep learning. In retrospect, this wasn't one of my best ideas, and I probably could have benefited from some additional features. Anyways, I managed to keep my codebase clean throughout the competition, and I've posted all the code on github in addition to a slightly more in-depth description of my approach.

\n\n

Here's the link - https://github.com/sjvasquez/instacart-basket-prediction

", "comments": [ { "id": 213564, "author_name": "tour1st", "author_url": "", "post_date": "2017-08-15T00:57:14.167000", "content": "

For us mere mortals, would you be kind enough to write a blog post on the process you went though when you were eating lightning and crapping thunder.

\n\n

And why you chose to write a custom class for training, fitting. Why something like keras was not suitable?

", "votes": 15, "replies": [ { "id": 213623, "author_name": "sjv", "author_url": "", "post_date": "2017-08-15T02:03:55.673000", "content": "

All the networks could have been implemented in Keras, and with less code. There's still a lot of redundant code outside of the network itself. I've written the code for the training loop, logging, checkpointing, early stopping, etc. many many times so I decided it would be worthwhile to abstract some of it.

", "votes": 9, "replies": [] }, { "id": 213634, "author_name": "DatumX", "author_url": "", "post_date": "2017-08-15T02:26:20.267000", "content": "

Would it be possible for you to publish the code in keras sometime in future?

", "votes": 1, "replies": [] }, { "id": 213823, "author_name": "tour1st", "author_url": "", "post_date": "2017-08-15T10:43:04.107000", "content": "

Now I don't feel like looking at your code until I give it a shot myself. Great job!

", "votes": 0, "replies": [] } ] }, { "id": 213562, "author_name": "Jing Lu", "author_url": "", "post_date": "2017-08-15T00:51:14.763000", "content": "

Congratualation! That's pretty impressive. How do you achieve such deep understanding on NN? I usually feel NN is a beast, that is hard to control well.

", "votes": 5, "replies": [] }, { "id": 217581, "author_name": "EricYu", "author_url": "", "post_date": "2017-08-31T07:05:08.143000", "content": "

Out of curiosity, how long did it take to train the neural network on this data with your specs?

", "votes": 3, "replies": [] }, { "id": 214022, "author_name": "YaGana Sheriff-Hussaini", "author_url": "", "post_date": "2017-08-15T18:29:11.213000", "content": "

Thanks for sharing @SJV and well done putting on such a strong finishing with a deep learning solution. This really is what Kaggle is all about to me, i.e. coming at an ML problem from different angles in order to arrive at the answer.

\n\n

My current labtop prevents me from implementing a NN solution for large data problems. I will definitely take a look at your code but only when I get my new hardware and tried it myself. I am not done with the instacart data as I feel it is a very good ground to play and learn much more for this kind of problem.

", "votes": 3, "replies": [] }, { "id": 213792, "author_name": "Aljaž", "author_url": "", "post_date": "2017-08-15T09:06:37.160000", "content": "

Congrats! Looks like something from the future. I hope I'll get there soon ;)

", "votes": 3, "replies": [] }, { "id": 213655, "author_name": "Ravi Teja Gutta", "author_url": "", "post_date": "2017-08-15T03:07:35.210000", "content": "

Really impressive!!, my prediction came true, I knew that someone with pure deep learning with minimal feature engineering would be able to get in top 10 and that was you .It would be really interesting to see the vector representations of products and users. Congrats! and also thank you for sharing your codebase

", "votes": 3, "replies": [] }, { "id": 213573, "author_name": "🐳鲲(China)", "author_url": "", "post_date": "2017-08-15T01:09:28.140000", "content": "

congratulations! damn cool! you light my fire to learn enjoy NN again.

", "votes": 3, "replies": [] }, { "id": 213545, "author_name": "happycube", "author_url": "", "post_date": "2017-08-15T00:35:41.297000", "content": "

Anything that gets 3rd place is a pretty damn good idea, I'd say ;) Congrats (and thanks for posting source :) )

", "votes": 3, "replies": [] }, { "id": 215874, "author_name": "Diego", "author_url": "", "post_date": "2017-08-23T13:54:49.107000", "content": "

Just a quick tip: If you are trying to run the code on python 3.4+ and having issues with the reload command on tf_base_model, just import:

\n\n
from importlib import reload\n
\n\n

that should solve your problem

", "votes": 1, "replies": [] }, { "id": 213608, "author_name": "Yihong Chen", "author_url": "", "post_date": "2017-08-15T01:50:39.180000", "content": "

Cool stuff ! I also tried to build a rnn model and it gave me a few good features. And then I messed it up with lightgbm/xgboost .... I should have more confidence in neural networks.

", "votes": 1, "replies": [] }, { "id": 214338, "author_name": "Jon", "author_url": "", "post_date": "2017-08-16T15:25:49.453000", "content": "

I am not sure what kind of machine can run this script.\n16GB ram took long long time in RNN training even 100 user_id filtered only.

", "votes": 2, "replies": [] }, { "id": 213732, "author_name": "Sameh Faidi", "author_url": "", "post_date": "2017-08-15T06:08:33.487000", "content": "

congrats on an excellent job. i wonder what kind of logloss and F1 score did you get with the base models ?

", "votes": 2, "replies": [ { "id": 214007, "author_name": "sjv", "author_url": "", "post_date": "2017-08-15T17:51:38.230000", "content": "

The best CNN/RNN gets logloss around .241, with public LB of about .407. The lgb model reduced logloss to .239.

", "votes": 3, "replies": [] } ] }, { "id": 213719, "author_name": "paulantoine", "author_url": "", "post_date": "2017-08-15T05:33:17.957000", "content": "

Congrats and thanks for sharing !

\n\n

The doc + codebase is nice to read ans i'm impressed by the use of various dnns \"at the right place\"

\n\n

Two questions :\nhow important is the final lgbm part ?\nhow much computing powerdid you use ?

\n\n

cheers

", "votes": 2, "replies": [] }, { "id": 213550, "author_name": "AlphaMeow", "author_url": "", "post_date": "2017-08-15T00:38:53.050000", "content": "

Congratulations and many thanks for the sharing.

\n\n

I was really curious about whether someone managed to do a neural network during this competition. This is wonderful. : )

", "votes": 1, "replies": [] }, { "id": 311882, "author_name": "mroman09", "author_url": "", "post_date": "2018-04-10T20:48:12.937000", "content": "

Very nicely done. Lots to learn from here.

", "votes": 0, "replies": [] }, { "id": 300582, "author_name": "Ben Sprecher", "author_url": "", "post_date": "2018-03-21T18:30:33.040000", "content": "

Very interesting work! Thanks for sharing!

", "votes": 0, "replies": [] }, { "id": 225938, "author_name": "SeanZhang", "author_url": "", "post_date": "2017-09-30T13:14:42.937000", "content": "

I learn a lot from your simple and elegant code, thank you very much

", "votes": 0, "replies": [] }, { "id": 221498, "author_name": "yy1165", "author_url": "", "post_date": "2017-09-15T13:25:57.870000", "content": "

Congrats and the brilliant code!

", "votes": 0, "replies": [] }, { "id": 216582, "author_name": "rainbow36", "author_url": "", "post_date": "2017-08-26T18:13:24.483000", "content": "

@sjv i want to gain a complete understanding of how neural networks work and how to build them using code...can you please tell me how should i start learning

", "votes": 0, "replies": [] }, { "id": 216580, "author_name": "rainbow36", "author_url": "", "post_date": "2017-08-26T18:00:53.840000", "content": "

@sjv can you please tell me how to get such a solid understanding of Neural Networks...i am able to understand theory but not able to do anything in the coding part...can you please give me some advice regarding how to learn to program neural networks to solve such problems

", "votes": 0, "replies": [] }, { "id": 214112, "author_name": "LivingProgram", "author_url": "", "post_date": "2017-08-16T01:58:34.673000", "content": "

Thanks for sharing @sjv, I was curious how did you perform the hyper-parameter tuning for all of your different models?

", "votes": 0, "replies": [] }, { "id": 213950, "author_name": "John Duffy", "author_url": "", "post_date": "2017-08-15T15:04:36.807000", "content": "

WOW. I think you ushered in a great milestone.

\n\n

Onto Passenger Screening...

", "votes": 0, "replies": [] }, { "id": 213923, "author_name": "Bradley Mitchell", "author_url": "", "post_date": "2017-08-15T14:36:19.213000", "content": "

Truly impressive work - thank you for sharing this with the community!!

", "votes": 0, "replies": [] }, { "id": 213883, "author_name": "ZZ", "author_url": "", "post_date": "2017-08-15T13:34:05.867000", "content": "

Very impressive NN solution! This is definitely a must-see resource after competition.

", "votes": 0, "replies": [] }, { "id": 213779, "author_name": "Daniel Burkhardt Cerigo", "author_url": "", "post_date": "2017-08-15T08:34:48.960000", "content": "

Thanks for sharing! \"I decided not to add any manual feature engineering since I thought it would be cool\" -> badass...!

", "votes": 0, "replies": [] }, { "id": 213759, "author_name": "Kent Chiu", "author_url": "", "post_date": "2017-08-15T07:45:10.033000", "content": "

Congrats, it is exactly what I want to do!

\n\n

Thanks for sharing the code, for us to learn.

", "votes": 0, "replies": [] }, { "id": 213726, "author_name": "AnilPandey", "author_url": "", "post_date": "2017-08-15T05:52:51.893000", "content": "

Congrats. Impressive Deep learning solution...

", "votes": 0, "replies": [] }, { "id": 213708, "author_name": "amasl", "author_url": "", "post_date": "2017-08-15T05:07:39.897000", "content": "

Congrats and many thanks for sharing RNN solution! Did you use a products embedding in the solution?

", "votes": 0, "replies": [ { "id": 213715, "author_name": "sjv", "author_url": "", "post_date": "2017-08-15T05:17:58.270000", "content": "

Yes, all high cardinality features were embedded (user_id, product_id, aisle_id, etc.)

", "votes": 3, "replies": [] }, { "id": 214810, "author_name": "ZJU-PANDA", "author_url": "", "post_date": "2017-08-18T11:24:39.977000", "content": "

Have you tried Google Wide&Deep Network? I tried it but found it can not converge in terms of wide module.\nThe product_id is such a huge dimension feature, how do you deal with embedding of product_id?

", "votes": 0, "replies": [] } ] }, { "id": 213650, "author_name": "Smilence", "author_url": "", "post_date": "2017-08-15T02:58:56.700000", "content": "

congratulation! That's super cool! I will read it tmr

", "votes": 0, "replies": [] }, { "id": 213640, "author_name": "", "author_url": "", "post_date": "2017-08-15T02:35:19.640000", "content": "", "votes": 0, "replies": [] }, { "id": 213637, "author_name": "", "author_url": "", "post_date": "2017-08-15T02:32:57.063000", "content": "", "votes": 0, "replies": [] }, { "id": 213636, "author_name": "", "author_url": "", "post_date": "2017-08-15T02:29:56.653000", "content": "", "votes": 0, "replies": [] }, { "id": 213627, "author_name": "", "author_url": "", "post_date": "2017-08-15T02:15:13.590000", "content": "", "votes": 0, "replies": [] }, { "id": 213607, "author_name": "", "author_url": "", "post_date": "2017-08-15T01:45:59.060000", "content": "", "votes": 0, "replies": [] }, { "id": 213604, "author_name": "", "author_url": "", "post_date": "2017-08-15T01:42:49.720000", "content": "", "votes": 0, "replies": [] }, { "id": 213603, "author_name": "", "author_url": "", "post_date": "2017-08-15T01:42:26.130000", "content": "", "votes": 0, "replies": [] }, { "id": 213602, "author_name": "", "author_url": "", "post_date": "2017-08-15T01:40:50.087000", "content": "", "votes": 0, "replies": [ { "id": 213625, "author_name": "", "author_url": "", "post_date": "2017-08-15T02:10:18", "content": "", "votes": 8, "replies": [] }, { "id": 213865, "author_name": "", "author_url": "", "post_date": "2017-08-15T12:37:07.893000", "content": "", "votes": 0, "replies": [] } ] }, { "id": 213595, "author_name": "", "author_url": "", "post_date": "2017-08-15T01:29:23.457000", "content": "", "votes": 0, "replies": [] }, { "id": 213588, "author_name": "", "author_url": "", "post_date": "2017-08-15T01:21:10.117000", "content": "", "votes": 0, "replies": [] }, { "id": 213581, "author_name": "", "author_url": "", "post_date": "2017-08-15T01:16:21.510000", "content": "", "votes": 0, "replies": [] }, { "id": 213577, "author_name": "", "author_url": "", "post_date": "2017-08-15T01:12:25.870000", "content": "", "votes": 0, "replies": [] }, { "id": 213566, "author_name": "", "author_url": "", "post_date": "2017-08-15T00:59:15.087000", "content": "", "votes": 0, "replies": [] }, { "id": 213558, "author_name": "", "author_url": "", "post_date": "2017-08-15T00:44:44.353000", "content": "", "votes": 0, "replies": [] }, { "id": 213556, "author_name": "", "author_url": "", "post_date": "2017-08-15T00:41:52.820000", "content": "", "votes": 0, "replies": [] }, { "id": 213551, "author_name": "", "author_url": "", "post_date": "2017-08-15T00:38:58.270000", "content": "", "votes": 0, "replies": [] }, { "id": 213548, "author_name": "", "author_url": "", "post_date": "2017-08-15T00:37:59.170000", "content": "", "votes": 0, "replies": [] }, { "id": 213547, "author_name": "", "author_url": "", "post_date": "2017-08-15T00:37:31.693000", "content": "", "votes": 0, "replies": [] }, { "id": 334875, "author_name": "", "author_url": "", "post_date": "2018-05-28T16:24:41.403000", "content": "", "votes": 0, "replies": [] }, { "id": 237665, "author_name": "", "author_url": "", "post_date": "2017-10-30T17:51:34.733000", "content": "", "votes": 2, "replies": [] }, { "id": 213721, "author_name": "", "author_url": "", "post_date": "2017-08-15T05:40:58.457000", "content": "", "votes": 0, "replies": [] }, { "id": 1602102, "author_name": "", "author_url": "", "post_date": "2021-12-01T19:11:09.503000", "content": "", "votes": 0, "replies": [] }, { "id": 213880, "author_name": "", "author_url": "", "post_date": "2017-08-15T13:23:19.133000", "content": "", "votes": 0, "replies": [] }, { "id": 213728, "author_name": "", "author_url": "", "post_date": "2017-08-15T05:54:41.607000", "content": "", "votes": 0, "replies": [] }, { "id": 213654, "author_name": "", "author_url": "", "post_date": "2017-08-15T03:07:21.760000", "content": "", "votes": 0, "replies": [] }, { "id": 213647, "author_name": "", "author_url": "", "post_date": "2017-08-15T02:52:21.743000", "content": "", "votes": 0, "replies": [] }, { "id": 213635, "author_name": "", "author_url": "", "post_date": "2017-08-15T02:29:43.130000", "content": "", "votes": 0, "replies": [] }, { "id": 213619, "author_name": "", "author_url": "", "post_date": "2017-08-15T02:01:26.227000", "content": "", "votes": 0, "replies": [] }, { "id": 213611, "author_name": "", "author_url": "", "post_date": "2017-08-15T01:53:41.650000", "content": "", "votes": 0, "replies": [] }, { "id": 213609, "author_name": "", "author_url": "", "post_date": "2017-08-15T01:51:27.510000", "content": "", "votes": 0, "replies": [] }, { "id": 213593, "author_name": "", "author_url": "", "post_date": "2017-08-15T01:28:34.317000", "content": "", "votes": 0, "replies": [] }, { "id": 213572, "author_name": "", "author_url": "", "post_date": "2017-08-15T01:08:10.637000", "content": "", "votes": 0, "replies": [] }, { "id": 213557, "author_name": "", "author_url": "", "post_date": "2017-08-15T00:42:02.523000", "content": "", "votes": 0, "replies": [] }, { "id": 213555, "author_name": "", "author_url": "", "post_date": "2017-08-15T00:40:31.013000", "content": "", "votes": 0, "replies": [] } ], "comment_count": 71, "messages": [ { "id": 213543, "postDate": "2017-08-15T00:32:17.750Z", "content": "

Hello everyone,

\n\n

First off, thanks to the competition organizers for a clean, interesting, and leak-free dataset and congrats to the winners!

\n\n

My solution was heavily dependent upon the use of neural networks for feature extraction. I decided not to add any manual feature engineering since I thought it would be cool to try to win a competition with pure deep learning. In retrospect, this wasn't one of my best ideas, and I probably could have benefited from some additional features. Anyways, I managed to keep my codebase clean throughout the competition, and I've posted all the code on github in addition to a slightly more in-depth description of my approach.

\n\n

Here's the link - https://github.com/sjvasquez/instacart-basket-prediction

", "rawMarkdown": "Hello everyone,\n\nFirst off, thanks to the competition organizers for a clean, interesting, and leak-free dataset and congrats to the winners!\n\nMy solution was heavily dependent upon the use of neural networks for feature extraction. I decided not to add any manual feature engineering since I thought it would be cool to try to win a competition with pure deep learning. In retrospect, this wasn't one of my best ideas, and I probably could have benefited from some additional features. Anyways, I managed to keep my codebase clean throughout the competition, and I've posted all the code on github in addition to a slightly more in-depth description of my approach.\n\nHere's the link - https://github.com/sjvasquez/instacart-basket-prediction", "votes": 235 }, { "id": 213564, "postDate": "2017-08-15T00:57:14.167Z", "content": "

For us mere mortals, would you be kind enough to write a blog post on the process you went though when you were eating lightning and crapping thunder.

\n\n

And why you chose to write a custom class for training, fitting. Why something like keras was not suitable?

", "rawMarkdown": "For us mere mortals, would you be kind enough to write a blog post on the process you went though when you were eating lightning and crapping thunder.\n\nAnd why you chose to write a custom class for training, fitting. Why something like keras was not suitable?", "votes": 15, "replies": [ { "id": 213623, "postDate": "2017-08-15T02:03:55.673Z", "content": "

All the networks could have been implemented in Keras, and with less code. There's still a lot of redundant code outside of the network itself. I've written the code for the training loop, logging, checkpointing, early stopping, etc. many many times so I decided it would be worthwhile to abstract some of it.

", "rawMarkdown": "All the networks could have been implemented in Keras, and with less code. There's still a lot of redundant code outside of the network itself. I've written the code for the training loop, logging, checkpointing, early stopping, etc. many many times so I decided it would be worthwhile to abstract some of it.", "votes": 9 }, { "id": 213634, "postDate": "2017-08-15T02:26:20.267Z", "content": "

Would it be possible for you to publish the code in keras sometime in future?

", "rawMarkdown": "Would it be possible for you to publish the code in keras sometime in future?", "votes": 1 }, { "id": 213823, "postDate": "2017-08-15T10:43:04.107Z", "content": "

Now I don't feel like looking at your code until I give it a shot myself. Great job!

", "rawMarkdown": "Now I don't feel like looking at your code until I give it a shot myself. Great job!" } ] }, { "id": 213562, "postDate": "2017-08-15T00:51:14.763Z", "content": "

Congratualation! That's pretty impressive. How do you achieve such deep understanding on NN? I usually feel NN is a beast, that is hard to control well.

", "rawMarkdown": "Congratualation! That's pretty impressive. How do you achieve such deep understanding on NN? I usually feel NN is a beast, that is hard to control well.", "votes": 5 }, { "id": 217581, "postDate": "2017-08-31T07:05:08.143Z", "content": "

Out of curiosity, how long did it take to train the neural network on this data with your specs?

", "rawMarkdown": "Out of curiosity, how long did it take to train the neural network on this data with your specs?", "votes": 3 }, { "id": 214022, "postDate": "2017-08-15T18:29:11.213Z", "content": "

Thanks for sharing @SJV and well done putting on such a strong finishing with a deep learning solution. This really is what Kaggle is all about to me, i.e. coming at an ML problem from different angles in order to arrive at the answer.

\n\n

My current labtop prevents me from implementing a NN solution for large data problems. I will definitely take a look at your code but only when I get my new hardware and tried it myself. I am not done with the instacart data as I feel it is a very good ground to play and learn much more for this kind of problem.

", "rawMarkdown": "Thanks for sharing @SJV and well done putting on such a strong finishing with a deep learning solution. This really is what Kaggle is all about to me, i.e. coming at an ML problem from different angles in order to arrive at the answer.\n\nMy current labtop prevents me from implementing a NN solution for large data problems. I will definitely take a look at your code but only when I get my new hardware and tried it myself. I am not done with the instacart data as I feel it is a very good ground to play and learn much more for this kind of problem.", "votes": 3 }, { "id": 213792, "postDate": "2017-08-15T09:06:37.160Z", "content": "

Congrats! Looks like something from the future. I hope I'll get there soon ;)

", "rawMarkdown": "Congrats! Looks like something from the future. I hope I'll get there soon ;)", "votes": 3 }, { "id": 213655, "postDate": "2017-08-15T03:07:35.210Z", "content": "

Really impressive!!, my prediction came true, I knew that someone with pure deep learning with minimal feature engineering would be able to get in top 10 and that was you .It would be really interesting to see the vector representations of products and users. Congrats! and also thank you for sharing your codebase

", "rawMarkdown": "Really impressive!!, my prediction came true, I knew that someone with pure deep learning with minimal feature engineering would be able to get in top 10 and that was you .It would be really interesting to see the vector representations of products and users. Congrats! and also thank you for sharing your codebase", "votes": 3 }, { "id": 213573, "postDate": "2017-08-15T01:09:28.140Z", "content": "

congratulations! damn cool! you light my fire to learn enjoy NN again.

", "rawMarkdown": "congratulations! damn cool! you light my fire to learn enjoy NN again.", "votes": 3 }, { "id": 213545, "postDate": "2017-08-15T00:35:41.297Z", "content": "

Anything that gets 3rd place is a pretty damn good idea, I'd say ;) Congrats (and thanks for posting source :) )

", "rawMarkdown": "Anything that gets 3rd place is a pretty damn good idea, I'd say ;) Congrats (and thanks for posting source :) )", "votes": 3 }, { "id": 215874, "postDate": "2017-08-23T13:54:49.107Z", "content": "

Just a quick tip: If you are trying to run the code on python 3.4+ and having issues with the reload command on tf_base_model, just import:

\n\n
from importlib import reload\n
\n\n

that should solve your problem

", "rawMarkdown": "Just a quick tip: If you are trying to run the code on python 3.4+ and having issues with the reload command on tf_base_model, just import:\n\n from importlib import reload\n\nthat should solve your problem", "votes": 1 }, { "id": 213608, "postDate": "2017-08-15T01:50:39.180Z", "content": "

Cool stuff ! I also tried to build a rnn model and it gave me a few good features. And then I messed it up with lightgbm/xgboost .... I should have more confidence in neural networks.

", "rawMarkdown": "Cool stuff ! I also tried to build a rnn model and it gave me a few good features. And then I messed it up with lightgbm/xgboost .... I should have more confidence in neural networks.", "votes": 1 }, { "id": 214338, "postDate": "2017-08-16T15:25:49.453Z", "content": "

I am not sure what kind of machine can run this script.\n16GB ram took long long time in RNN training even 100 user_id filtered only.

", "rawMarkdown": "I am not sure what kind of machine can run this script.\n16GB ram took long long time in RNN training even 100 user_id filtered only.", "votes": 2 }, { "id": 213732, "postDate": "2017-08-15T06:08:33.487Z", "content": "

congrats on an excellent job. i wonder what kind of logloss and F1 score did you get with the base models ?

", "rawMarkdown": "congrats on an excellent job. i wonder what kind of logloss and F1 score did you get with the base models ?", "votes": 2, "replies": [ { "id": 214007, "postDate": "2017-08-15T17:51:38.230Z", "content": "

The best CNN/RNN gets logloss around .241, with public LB of about .407. The lgb model reduced logloss to .239.

", "rawMarkdown": "The best CNN/RNN gets logloss around .241, with public LB of about .407. The lgb model reduced logloss to .239.", "votes": 3 } ] }, { "id": 213719, "postDate": "2017-08-15T05:33:17.957Z", "content": "

Congrats and thanks for sharing !

\n\n

The doc + codebase is nice to read ans i'm impressed by the use of various dnns \"at the right place\"

\n\n

Two questions :\nhow important is the final lgbm part ?\nhow much computing powerdid you use ?

\n\n

cheers

", "rawMarkdown": "Congrats and thanks for sharing !\n\nThe doc + codebase is nice to read ans i'm impressed by the use of various dnns \"at the right place\"\n\nTwo questions :\nhow important is the final lgbm part ?\nhow much computing powerdid you use ?\n\ncheers\n\n", "votes": 2 }, { "id": 213550, "postDate": "2017-08-15T00:38:53.050Z", "content": "

Congratulations and many thanks for the sharing.

\n\n

I was really curious about whether someone managed to do a neural network during this competition. This is wonderful. : )

", "rawMarkdown": "Congratulations and many thanks for the sharing. \n\nI was really curious about whether someone managed to do a neural network during this competition. This is wonderful. : )", "votes": 1 }, { "id": 311882, "postDate": "2018-04-10T20:48:12.937Z", "content": "

Very nicely done. Lots to learn from here.

", "rawMarkdown": "Very nicely done. Lots to learn from here. " }, { "id": 300582, "postDate": "2018-03-21T18:30:33.040Z", "content": "

Very interesting work! Thanks for sharing!

", "rawMarkdown": "Very interesting work! Thanks for sharing!" }, { "id": 225938, "postDate": "2017-09-30T13:14:42.937Z", "content": "

I learn a lot from your simple and elegant code, thank you very much

", "rawMarkdown": "I learn a lot from your simple and elegant code, thank you very much" }, { "id": 221498, "postDate": "2017-09-15T13:25:57.870Z", "content": "

Congrats and the brilliant code!

", "rawMarkdown": "Congrats and the brilliant code!" }, { "id": 216582, "postDate": "2017-08-26T18:13:24.483Z", "content": "

@sjv i want to gain a complete understanding of how neural networks work and how to build them using code...can you please tell me how should i start learning

", "rawMarkdown": "@sjv i want to gain a complete understanding of how neural networks work and how to build them using code...can you please tell me how should i start learning" }, { "id": 216580, "postDate": "2017-08-26T18:00:53.840Z", "content": "

@sjv can you please tell me how to get such a solid understanding of Neural Networks...i am able to understand theory but not able to do anything in the coding part...can you please give me some advice regarding how to learn to program neural networks to solve such problems

", "rawMarkdown": "@sjv can you please tell me how to get such a solid understanding of Neural Networks...i am able to understand theory but not able to do anything in the coding part...can you please give me some advice regarding how to learn to program neural networks to solve such problems" }, { "id": 214112, "postDate": "2017-08-16T01:58:34.673Z", "content": "

Thanks for sharing @sjv, I was curious how did you perform the hyper-parameter tuning for all of your different models?

", "rawMarkdown": "Thanks for sharing @sjv, I was curious how did you perform the hyper-parameter tuning for all of your different models?" }, { "id": 213950, "postDate": "2017-08-15T15:04:36.807Z", "content": "

WOW. I think you ushered in a great milestone.

\n\n

Onto Passenger Screening...

", "rawMarkdown": "WOW. I think you ushered in a great milestone.\n\nOnto Passenger Screening..." }, { "id": 213923, "postDate": "2017-08-15T14:36:19.213Z", "content": "

Truly impressive work - thank you for sharing this with the community!!

", "rawMarkdown": "Truly impressive work - thank you for sharing this with the community!!" }, { "id": 213883, "postDate": "2017-08-15T13:34:05.867Z", "content": "

Very impressive NN solution! This is definitely a must-see resource after competition.

", "rawMarkdown": "Very impressive NN solution! This is definitely a must-see resource after competition. \n" }, { "id": 213779, "postDate": "2017-08-15T08:34:48.960Z", "content": "

Thanks for sharing! \"I decided not to add any manual feature engineering since I thought it would be cool\" -> badass...!

", "rawMarkdown": "Thanks for sharing! \"I decided not to add any manual feature engineering since I thought it would be cool\" -> badass...!" }, { "id": 213759, "postDate": "2017-08-15T07:45:10.033Z", "content": "

Congrats, it is exactly what I want to do!

\n\n

Thanks for sharing the code, for us to learn.

", "rawMarkdown": "Congrats, it is exactly what I want to do!\n\nThanks for sharing the code, for us to learn. \n" }, { "id": 213726, "postDate": "2017-08-15T05:52:51.893Z", "content": "

Congrats. Impressive Deep learning solution...

", "rawMarkdown": "Congrats. Impressive Deep learning solution..." }, { "id": 213708, "postDate": "2017-08-15T05:07:39.897Z", "content": "

Congrats and many thanks for sharing RNN solution! Did you use a products embedding in the solution?

", "rawMarkdown": "Congrats and many thanks for sharing RNN solution! Did you use a products embedding in the solution?", "replies": [ { "id": 213715, "postDate": "2017-08-15T05:17:58.270Z", "content": "

Yes, all high cardinality features were embedded (user_id, product_id, aisle_id, etc.)

", "rawMarkdown": "Yes, all high cardinality features were embedded (user_id, product_id, aisle_id, etc.)", "votes": 3 }, { "id": 214810, "postDate": "2017-08-18T11:24:39.977Z", "content": "

Have you tried Google Wide&Deep Network? I tried it but found it can not converge in terms of wide module.\nThe product_id is such a huge dimension feature, how do you deal with embedding of product_id?

", "rawMarkdown": "Have you tried Google Wide&Deep Network? I tried it but found it can not converge in terms of wide module.\nThe product_id is such a huge dimension feature, how do you deal with embedding of product_id? " } ] }, { "id": 213650, "postDate": "2017-08-15T02:58:56.700Z", "content": "

congratulation! That's super cool! I will read it tmr

", "rawMarkdown": "congratulation! That's super cool! I will read it tmr" }, { "id": 213640, "postDate": "2017-08-15T02:35:19.640Z", "content": "

Congrats and thank you so much for sharing your impressive solution!

", "rawMarkdown": "Congrats and thank you so much for sharing your impressive solution!" }, { "id": 213637, "postDate": "2017-08-15T02:32:57.063Z", "content": "

Congratulations !!!!

", "rawMarkdown": "Congratulations !!!!" }, { "id": 213636, "postDate": "2017-08-15T02:29:56.653Z", "content": "

Congrats. Impressive solution. Thanks for sharing.

", "rawMarkdown": "Congrats. Impressive solution. Thanks for sharing. " }, { "id": 213627, "postDate": "2017-08-15T02:15:13.590Z", "content": "

Cool. The solution can be applied to real business.\nCongrats!

", "rawMarkdown": "Cool. The solution can be applied to real business.\nCongrats!" }, { "id": 213607, "postDate": "2017-08-15T01:45:59.060Z", "content": "

Thank you for sharing, and congratulations!

", "rawMarkdown": "Thank you for sharing, and congratulations!" }, { "id": 213604, "postDate": "2017-08-15T01:42:49.720Z", "content": "

Congrats! It is impresive to implement this large deep learning framework.

", "rawMarkdown": "Congrats! It is impresive to implement this large deep learning framework." }, { "id": 213603, "postDate": "2017-08-15T01:42:26.130Z", "content": "

congratulations! Thank you for your sharing.

", "rawMarkdown": "congratulations! Thank you for your sharing." }, { "id": 213602, "postDate": "2017-08-15T01:40:50.087Z", "content": "

Just wondering how do you determine the network structures?

", "rawMarkdown": "Just wondering how do you determine the network structures?", "replies": [ { "id": 213625, "postDate": "2017-08-15T02:10:18Z", "content": "

It's an empirical process for the most part. The CNN was inspired by an existing successful architecture for sequence processing (WaveNet) and the LSTM was a vanilla LSTM. The most important architectural parameters are the number of layers in the CNN/LSTM and the hidden state sizes, so those are the only ones I really played around with.

", "rawMarkdown": "It's an empirical process for the most part. The CNN was inspired by an existing successful architecture for sequence processing (WaveNet) and the LSTM was a vanilla LSTM. The most important architectural parameters are the number of layers in the CNN/LSTM and the hidden state sizes, so those are the only ones I really played around with.", "votes": 8 }, { "id": 213865, "postDate": "2017-08-15T12:37:07.893Z", "content": "

Have you ever thought about applying Bi-LSTM?

", "rawMarkdown": "Have you ever thought about applying Bi-LSTM?" } ] }, { "id": 213595, "postDate": "2017-08-15T01:29:23.457Z", "content": "

This is really impressive! Thank you for sharing!

", "rawMarkdown": "This is really impressive! Thank you for sharing!" }, { "id": 213588, "postDate": "2017-08-15T01:21:10.117Z", "content": "

Thanks for your sharing. I will read and learn it later!!

", "rawMarkdown": "Thanks for your sharing. I will read and learn it later!!" }, { "id": 213581, "postDate": "2017-08-15T01:16:21.510Z", "content": "

Cool！

", "rawMarkdown": "Cool！" }, { "id": 213577, "postDate": "2017-08-15T01:12:25.870Z", "content": "

Amazing ! I really enjoy your solution.

", "rawMarkdown": "Amazing ! I really enjoy your solution." }, { "id": 213566, "postDate": "2017-08-15T00:59:15.087Z", "content": "

Congratulations! and thank you for your sharing!

", "rawMarkdown": "Congratulations! and thank you for your sharing!" }, { "id": 213558, "postDate": "2017-08-15T00:44:44.353Z", "content": "

Congratulations! Fantastic!

", "rawMarkdown": "Congratulations! Fantastic!" }, { "id": 213556, "postDate": "2017-08-15T00:41:52.820Z", "content": "

Congratulations!...Very impressive

", "rawMarkdown": "Congratulations!...Very impressive" }, { "id": 213551, "postDate": "2017-08-15T00:38:58.270Z", "content": "

Congrats.

", "rawMarkdown": "Congrats." }, { "id": 213548, "postDate": "2017-08-15T00:37:59.170Z", "content": "

You are amazing!!!

", "rawMarkdown": "You are amazing!!!" }, { "id": 213547, "postDate": "2017-08-15T00:37:31.693Z", "content": "

Pure deep learning solution ! Very impressive

\n\n

Congrats Sean !!!

\n\n

And thanks for sharing your code ! Hope to learn from it :-)

", "rawMarkdown": "Pure deep learning solution ! Very impressive \n\nCongrats Sean !!!\n\nAnd thanks for sharing your code ! Hope to learn from it :-)" }, { "id": 334875, "postDate": "2018-05-28T16:24:41.403Z", "rawMarkdown": "", "isDeleted": true }, { "id": 237665, "postDate": "2017-10-30T17:51:34.733Z", "rawMarkdown": "", "votes": 2, "isDeleted": true }, { "id": 213721, "postDate": "2017-08-15T05:40:58.457Z", "rawMarkdown": "", "isDeleted": true }, { "id": 1602102, "postDate": "2021-12-01T19:11:09.503Z", "content": "

Thanks for sharing!

", "rawMarkdown": "Thanks for sharing!" }, { "id": 213880, "postDate": "2017-08-15T13:23:19.133Z", "content": "

Thanks a lot for sharing this!

", "rawMarkdown": "Thanks a lot for sharing this!" }, { "id": 213728, "postDate": "2017-08-15T05:54:41.607Z", "content": "

Thanks for sharing, very impressive!

", "rawMarkdown": "Thanks for sharing, very impressive!" }, { "id": 213654, "postDate": "2017-08-15T03:07:21.760Z", "content": "

This is cool, thanks for sharing!

", "rawMarkdown": "This is cool, thanks for sharing!" }, { "id": 213647, "postDate": "2017-08-15T02:52:21.743Z", "content": "

Thanks for sharing!

", "rawMarkdown": "Thanks for sharing!" }, { "id": 213635, "postDate": "2017-08-15T02:29:43.130Z", "content": "

Cool stuff ! thanks for sharing.

", "rawMarkdown": "Cool stuff ! thanks for sharing." }, { "id": 213619, "postDate": "2017-08-15T02:01:26.227Z", "content": "

Thanks.

", "rawMarkdown": "Thanks." }, { "id": 213611, "postDate": "2017-08-15T01:53:41.650Z", "content": "

Congrats,thanks for your sharing!

", "rawMarkdown": "Congrats,thanks for your sharing!" }, { "id": 213609, "postDate": "2017-08-15T01:51:27.510Z", "content": "

Congrats,thanks for your sharing!

", "rawMarkdown": "Congrats,thanks for your sharing!" }, { "id": 213593, "postDate": "2017-08-15T01:28:34.317Z", "content": "

Thanks for your sharing.

", "rawMarkdown": "Thanks for your sharing." }, { "id": 213572, "postDate": "2017-08-15T01:08:10.637Z", "content": "

Excellent! Thanks for sharing!

", "rawMarkdown": "Excellent! Thanks for sharing!" }, { "id": 213557, "postDate": "2017-08-15T00:42:02.523Z", "content": "

Congrats, and thanks for sharing.

", "rawMarkdown": "Congrats, and thanks for sharing." }, { "id": 213555, "postDate": "2017-08-15T00:40:31.013Z", "content": "

Congrats, and thanks for sharing.

", "rawMarkdown": "Congrats, and thanks for sharing." } ], "complete": true, "source": "live", "fetched_at": "2026-10-20T02:03:38Z", "competition_id": 6644, "competition_slug": "basket-analysis", "team_id": 722719, "solution_url": "/competitions/basket-analysis/writeups/sjv-3rd-place-solution-overview" }