{ "id": 127210, "title": "4th private (7th public) place writeup, link to code", "content": "

Many thanks Booz Allen Hamilton for the great competitions and all participants for interesting ideas and discussions!

\n\n

We jumped from 7 place public LB on 4th place on private. Best submit we selected is 0.561 private, 0.572 on public LB. It was a blend of neural networks. The second one was a 3-level stack 0.560 private LB, 0.566 public LB.

\n\n

Some ideas first

\n\n
    \n
  1. The test set can be labeled with the same procedure as a train one and can be used in training.
    2. \n
  2. TfIdf on sequence of events. Each event_id can be coded as title + event_code + correct_flag + incorrect_flag. After that we can look at installation_id history as a sequence of tokens and train tfidf on them. For training procedure we apply transformer on actual sequence of tokens before the current assessment.
    3. \n
  3. Some clips and other titles are very important for estimation of accuracy on the assessment. Maybe order of them is not so important, but RNN architecture can handle their presence in a user history good enough.
    4. \n
  4. We have a small amount of data and stability is much more important than the actual score. If changing column order makes score worse than we are doing something wrong.
    5. \n
\n\n

Models

\n\n

Neural network

\n\n

Tfidf features + RNN on title sequence (last 64) with some additional features:\n1. Embedding of title dim=7.\n2. Number of correct attempts during the title.\n3. Number of incorrect attempts during the title.\n4. Ratio of 2 and 3.\n5. Log of time in seconds between starts of the titles.\n6. Number of correct attempts in the previous title like that.\n7. Number of incorrect attempts in the previous title like that.\n8. Ratio of 7 and 6.\n(9) I've tried a lot to include counters as is in the model. Finally, I gave up, but those models peformed better on the private LB.\nMost of single networks were on 0.56+ zone. The most interesting one for me has equal public and private scores:\nhttps://www.kaggle.com/sergeifironov/bowl-stabilize-coefs-cntrs-all5

\n\n

Tree based models

\n\n

Lightgbm,Xgb, Catboost. (will be soon)

\n\n

Stack

\n\n

0 level) NN folds in folds model (5 outer folds, 5 inner folds), lgbm, catboost.\n1st level) MLP, Lightgbm.\n2nd level) Ridge.\nhttps://www.kaggle.com/c/data-science-bowl-2019/discussion/127312

\n\n

Validation

\n\n

I wrote a lot in this topic: https://www.kaggle.com/c/data-science-bowl-2019/discussion/125001, but near the end of the competition I gave up to make it correlated with public LB and use the very simple one without installation id groups at all.

\n\n

What doesn’t work for us

\n\n
    \n
  1. Transformers, GPT-2 and BERT vectors trained on predict event_id, title, title+accuracy_group and so on. They are useless.
    2. \n
  2. Graph NN.
    3. \n
  3. Transformer on a sequence of events. It’s too fat for this small amount of data.
    4. \n
", "comments": [ { "id": 727034, "author_name": "Psi", "author_url": "", "post_date": "2020-01-23T11:57:36.853000", "content": "

Impressive, when I read about your success with NN I was sure that you will have something stable.

", "votes": 4, "replies": [] }, { "id": 726308, "author_name": "Hidehisa Arai", "author_url": "", "post_date": "2020-01-23T00:57:35.463000", "content": "

Congratulations @sergeifironov and thank you for your writeup!\nWe believed you wouldn't shake down, and it appears to be true!

", "votes": 3, "replies": [] }, { "id": 726257, "author_name": "Mukharbek Organokov", "author_url": "", "post_date": "2020-01-23T00:24:15.777000", "content": "

Good job! Поздравляю! It's amazing how small is your shakeup, which makes the achievement even greater.

", "votes": 3, "replies": [ { "id": 726260, "author_name": "Sergei Fironov", "author_url": "", "post_date": "2020-01-23T00:26:42.150000", "content": "

Thanks! We've tried to stabilise our solution as much as possible to avoid the pain of choice. The most stable submits were clear winners.

", "votes": 1, "replies": [] } ] }, { "id": 728732, "author_name": "toshi_k", "author_url": "", "post_date": "2020-01-25T06:50:04.020000", "content": "

Thank you for sharing ! My approach was similar with yours. I counted the number of event code for each sessions, and trained Transformers on sequence of sessions. I wonder why my NN models went to shake-down 😭 \nhttps://www.kaggle.com/toshik/37th-place-solution

", "votes": 2, "replies": [ { "id": 728967, "author_name": "Sergei Fironov", "author_url": "", "post_date": "2020-01-25T14:12:58.693000", "content": "

I'm really sorry to hear that. I guess the difference between public and private (and public-CV as well) would be smaller without some counters (like 4070, 3020, 3121), but I don't really sure that this is issue. Thanks for sharing your kernel, it's great!

", "votes": 1, "replies": [] } ] }, { "id": 726256, "author_name": "Massoud Hosseinali", "author_url": "", "post_date": "2020-01-23T00:23:22.553000", "content": "

Congratulations.

", "votes": 2, "replies": [] }, { "id": 728827, "author_name": "Shruti_Iyyer", "author_url": "", "post_date": "2020-01-25T10:49:11.613000", "content": "

Congrats on winning! Thanks for sharing the approach.\nI didn't get this point by the way \"The test set can be labeled with the same procedure as a train one and can be used in training\". Could you help me on this?

", "votes": 1, "replies": [ { "id": 728970, "author_name": "Sergei Fironov", "author_url": "", "post_date": "2020-01-25T14:15:12.663000", "content": "

Sure. I mean a lot of installation id (~50%) in the test have long history including assessments with attempts. So we can calculate accuracy for those assessments and use those parts of history as additional training samples. Hope it helps.

", "votes": 1, "replies": [] } ] }, { "id": 727058, "author_name": "Sam Wang", "author_url": "", "post_date": "2020-01-23T12:26:55.743000", "content": "

Congratulations! We can learn lots of knowledge and experience from your work！Thanks for sharing！

", "votes": 1, "replies": [] }, { "id": 726465, "author_name": "Jorge Pineda", "author_url": "", "post_date": "2020-01-23T02:48:39.847000", "content": "

I knew you were gonna win, congrats!

", "votes": 1, "replies": [] }, { "id": 726449, "author_name": "davidlee0314", "author_url": "", "post_date": "2020-01-23T02:33:02.970000", "content": "

There are some good codes in the kernel! Thank you for sharing

", "votes": 1, "replies": [] }, { "id": 726432, "author_name": "narsil (jobs-in-data.com)", "author_url": "", "post_date": "2020-01-23T02:24:12.907000", "content": "

Congrats @sergeifironov ! Your ideas inspired me and gave me confidence throughout this competition.

", "votes": 1, "replies": [] }, { "id": 726420, "author_name": "KhanhVD", "author_url": "", "post_date": "2020-01-23T02:14:28.337000", "content": "

Congrats @sergeifironov and thanks for sharing your solution!

", "votes": 1, "replies": [] }, { "id": 726359, "author_name": "Boris Latch", "author_url": "", "post_date": "2020-01-23T01:26:43.147000", "content": "

Thank you so much for posting this!!! This is a super interesting approach and I look forward to digging through it in more detail!

", "votes": 1, "replies": [] }, { "id": 726311, "author_name": "Patrick Chan", "author_url": "", "post_date": "2020-01-23T00:59:54.073000", "content": "

Great share and thanks for the insights. Congrats Sergei!

", "votes": 1, "replies": [] }, { "id": 726300, "author_name": "FGPC", "author_url": "", "post_date": "2020-01-23T00:52:29.917000", "content": "

Amazing solutions! Congrats!

", "votes": 1, "replies": [] }, { "id": 726290, "author_name": "Nima Shahbazi", "author_url": "", "post_date": "2020-01-23T00:46:10.260000", "content": "

Great job Sergei!

", "votes": 1, "replies": [] }, { "id": 735068, "author_name": "NorwayPing", "author_url": "", "post_date": "2020-02-02T14:05:58.197000", "content": "

Congrats!\nInteresting to learn the approach by stacking and using of NN model.\nCould you share with me how much performance increased by stacking? \nIn your understanding, what is the most important factors (features? validation? NN model? or stacking?) you won?

", "votes": 0, "replies": [] }, { "id": 727247, "author_name": "James Ingram", "author_url": "", "post_date": "2020-01-23T15:16:04.243000", "content": "

Congratulations and thanks for sharing your experience and insights especially about correlation with LB. Brilliant!

", "votes": 0, "replies": [] }, { "id": 727031, "author_name": "Ivan Bagmut", "author_url": "", "post_date": "2020-01-23T11:50:06.933000", "content": "

Congrats, great kernel, great work, very useful.

", "votes": 0, "replies": [] }, { "id": 726882, "author_name": "", "author_url": "", "post_date": "2020-01-23T09:48:51.840000", "content": "

Congratulations! Amazing solutions!

", "votes": 0, "replies": [] }, { "id": 726820, "author_name": "Mohammed Tayor", "author_url": "", "post_date": "2020-01-23T08:56:36.533000", "content": "

Congratulations!

", "votes": 0, "replies": [] }, { "id": 726561, "author_name": "Kent Chiu", "author_url": "", "post_date": "2020-01-23T04:43:07.790000", "content": "

Great job! Congratulations!

", "votes": 0, "replies": [] }, { "id": 726469, "author_name": "siddharth", "author_url": "", "post_date": "2020-01-23T02:52:43.687000", "content": "

Congratulations

", "votes": 0, "replies": [] }, { "id": 726302, "author_name": "Weiquan", "author_url": "", "post_date": "2020-01-23T00:53:48.050000", "content": "

I see that most of your features are different from public kernels. How did you arrive at them?

", "votes": 0, "replies": [ { "id": 726314, "author_name": "Sergei Fironov", "author_url": "", "post_date": "2020-01-23T01:02:12.447000", "content": "

My features are very simple and intuitive (from my point of view) for NN. We can't train on events, so we train on titles and use the most important information about it (number of attempts, ratio, time). Most public kernels contained feature preparation for tree-based models. Those features aren't totally impossible to incorporate in NN, but don't look so useful there. My teammate @simakov will write about tree-based models and feature engineering for them.

", "votes": 2, "replies": [] }, { "id": 726445, "author_name": "RDizzl3", "author_url": "", "post_date": "2020-01-23T02:29:47.510000", "content": "

That’s awesome @sergeifironov. The crazy thing is that I had a lot of these same features, even TFIDF on added columns, but I was not able to try a lot because of travel and I had a really bad bug in my code that I could not hunt down. I wanted to try the RNN on those sequences but didn’t have the time, seems like it was very fruitful. Congrats on the great finish!

", "votes": 1, "replies": [] }, { "id": 726447, "author_name": "RDizzl3", "author_url": "", "post_date": "2020-01-23T02:30:52.337000", "content": "

Oh also @sergeifironov thank you for sharing the code :) I always learn a lot from you

", "votes": 0, "replies": [] }, { "id": 726473, "author_name": "Sergei Fironov", "author_url": "", "post_date": "2020-01-23T02:59:07.680000", "content": "

@rdizzl3 thanks! Some of ideas were inspired from your great work in the mercari competition, you see ;) And I thought at least twice that this competition fits you much. Sorry to hear about the problems.

", "votes": 2, "replies": [] }, { "id": 726474, "author_name": "RDizzl3", "author_url": "", "post_date": "2020-01-23T03:01:30.563000", "content": "

Thanks @sergeifironov! It's no worries :) I am always glad to see a friend do well in these competitions! But it is great to know that a lot of the techniques that worked for us there inspired a win here! I will spend some time to go through the code and pull all the gold nuggets out of it :) Congrats again on the great finish.

", "votes": 1, "replies": [] } ] }, { "id": 726509, "author_name": "", "author_url": "", "post_date": "2020-01-23T03:25:25.743000", "content": "", "votes": 0, "replies": [] }, { "id": 733267, "author_name": "corochann", "author_url": "", "post_date": "2020-01-30T22:55:11.887000", "content": "

Congrats and thank you for sharing.

", "votes": 0, "replies": [] }, { "id": 727625, "author_name": "Gustavo T Fontes", "author_url": "", "post_date": "2020-01-23T22:51:27.280000", "content": "

Thanks for sharing!

", "votes": 0, "replies": [] }, { "id": 726467, "author_name": "Hanjoon Choe", "author_url": "", "post_date": "2020-01-23T02:50:14.720000", "content": "

Wow thank you for sharing.

", "votes": 0, "replies": [] } ], "comment_count": 35, "messages": [ { "id": 726249, "postDate": "2020-01-23T00:18:49.547Z", "content": "

Many thanks Booz Allen Hamilton for the great competitions and all participants for interesting ideas and discussions!

\n\n

We jumped from 7 place public LB on 4th place on private. Best submit we selected is 0.561 private, 0.572 on public LB. It was a blend of neural networks. The second one was a 3-level stack 0.560 private LB, 0.566 public LB.

\n\n

Some ideas first

\n\n
    \n
  1. The test set can be labeled with the same procedure as a train one and can be used in training.
    2. \n
  2. TfIdf on sequence of events. Each event_id can be coded as title + event_code + correct_flag + incorrect_flag. After that we can look at installation_id history as a sequence of tokens and train tfidf on them. For training procedure we apply transformer on actual sequence of tokens before the current assessment.
    3. \n
  3. Some clips and other titles are very important for estimation of accuracy on the assessment. Maybe order of them is not so important, but RNN architecture can handle their presence in a user history good enough.
    4. \n
  4. We have a small amount of data and stability is much more important than the actual score. If changing column order makes score worse than we are doing something wrong.
    5. \n
\n\n

Models

\n\n

Neural network

\n\n

Tfidf features + RNN on title sequence (last 64) with some additional features:\n1. Embedding of title dim=7.\n2. Number of correct attempts during the title.\n3. Number of incorrect attempts during the title.\n4. Ratio of 2 and 3.\n5. Log of time in seconds between starts of the titles.\n6. Number of correct attempts in the previous title like that.\n7. Number of incorrect attempts in the previous title like that.\n8. Ratio of 7 and 6.\n(9) I've tried a lot to include counters as is in the model. Finally, I gave up, but those models peformed better on the private LB.\nMost of single networks were on 0.56+ zone. The most interesting one for me has equal public and private scores:\nhttps://www.kaggle.com/sergeifironov/bowl-stabilize-coefs-cntrs-all5

\n\n

Tree based models

\n\n

Lightgbm,Xgb, Catboost. (will be soon)

\n\n

Stack

\n\n

0 level) NN folds in folds model (5 outer folds, 5 inner folds), lgbm, catboost.\n1st level) MLP, Lightgbm.\n2nd level) Ridge.\nhttps://www.kaggle.com/c/data-science-bowl-2019/discussion/127312

\n\n

Validation

\n\n

I wrote a lot in this topic: https://www.kaggle.com/c/data-science-bowl-2019/discussion/125001, but near the end of the competition I gave up to make it correlated with public LB and use the very simple one without installation id groups at all.

\n\n

What doesn’t work for us

\n\n
    \n
  1. Transformers, GPT-2 and BERT vectors trained on predict event_id, title, title+accuracy_group and so on. They are useless.
    2. \n
  2. Graph NN.
    3. \n
  3. Transformer on a sequence of events. It’s too fat for this small amount of data.
    4. \n
", "rawMarkdown": "Many thanks Booz Allen Hamilton for the great competitions and all participants for interesting ideas and discussions!\n\nWe jumped from 7 place public LB on 4th place on private. Best submit we selected is 0.561 private, 0.572 on public LB. It was a blend of neural networks. The second one was a 3-level stack 0.560 private LB, 0.566 public LB.\n\n# Some ideas first\n1. The test set can be labeled with the same procedure as a train one and can be used in training.\n2. TfIdf on sequence of events. Each event_id can be coded as title + event_code + correct_flag + incorrect_flag. After that we can look at installation_id history as a sequence of tokens and train tfidf on them. For training procedure we apply transformer on actual sequence of tokens before the current assessment.\n3. Some clips and other titles are very important for estimation of accuracy on the assessment. Maybe order of them is not so important, but RNN architecture can handle their presence in a user history good enough.\n4. We have a small amount of data and stability is much more important than the actual score. If changing column order makes score worse than we are doing something wrong.\n\n# Models\n## Neural network\nTfidf features + RNN on title sequence (last 64) with some additional features:\n1. Embedding of title dim=7.\n2. Number of correct attempts during the title.\n3. Number of incorrect attempts during the title.\n4. Ratio of 2 and 3.\n5. Log of time in seconds between starts of the titles.\n6. Number of correct attempts in the previous title like that.\n7. Number of incorrect attempts in the previous title like that.\n8. Ratio of 7 and 6.\n(9) I've tried a lot to include counters as is in the model. Finally, I gave up, but those models peformed better on the private LB.\nMost of single networks were on 0.56+ zone. The most interesting one for me has equal public and private scores:\nhttps://www.kaggle.com/sergeifironov/bowl-stabilize-coefs-cntrs-all5\n\n## Tree based models\nLightgbm,Xgb, Catboost. (will be soon)\n\n# Stack\n0 level) NN folds in folds model (5 outer folds, 5 inner folds), lgbm, catboost.\n1st level) MLP, Lightgbm.\n2nd level) Ridge.\nhttps://www.kaggle.com/c/data-science-bowl-2019/discussion/127312\n\n# Validation\nI wrote a lot in this topic: https://www.kaggle.com/c/data-science-bowl-2019/discussion/125001, but near the end of the competition I gave up to make it correlated with public LB and use the very simple one without installation id groups at all.\n\n# What doesn’t work for us\n1. Transformers, GPT-2 and BERT vectors trained on predict event_id, title, title+accuracy_group and so on. They are useless.\n2. Graph NN. \n3. Transformer on a sequence of events. It’s too fat for this small amount of data.\n", "votes": 138 }, { "id": 727034, "postDate": "2020-01-23T11:57:36.853Z", "content": "

Impressive, when I read about your success with NN I was sure that you will have something stable.

", "rawMarkdown": "Impressive, when I read about your success with NN I was sure that you will have something stable.", "votes": 4 }, { "id": 726308, "postDate": "2020-01-23T00:57:35.463Z", "content": "

Congratulations @sergeifironov and thank you for your writeup!\nWe believed you wouldn't shake down, and it appears to be true!

", "rawMarkdown": "Congratulations @sergeifironov and thank you for your writeup!\nWe believed you wouldn't shake down, and it appears to be true!", "votes": 3 }, { "id": 726257, "postDate": "2020-01-23T00:24:15.777Z", "content": "

Good job! Поздравляю! It's amazing how small is your shakeup, which makes the achievement even greater.

", "rawMarkdown": "Good job! Поздравляю! It's amazing how small is your shakeup, which makes the achievement even greater. ", "votes": 3, "replies": [ { "id": 726260, "postDate": "2020-01-23T00:26:42.150Z", "content": "

Thanks! We've tried to stabilise our solution as much as possible to avoid the pain of choice. The most stable submits were clear winners.

", "rawMarkdown": "Thanks! We've tried to stabilise our solution as much as possible to avoid the pain of choice. The most stable submits were clear winners.", "votes": 1 } ] }, { "id": 728732, "postDate": "2020-01-25T06:50:04.020Z", "content": "

Thank you for sharing ! My approach was similar with yours. I counted the number of event code for each sessions, and trained Transformers on sequence of sessions. I wonder why my NN models went to shake-down 😭 \nhttps://www.kaggle.com/toshik/37th-place-solution

", "rawMarkdown": "Thank you for sharing ! My approach was similar with yours. I counted the number of event code for each sessions, and trained Transformers on sequence of sessions. I wonder why my NN models went to shake-down 😭 \nhttps://www.kaggle.com/toshik/37th-place-solution", "votes": 2, "replies": [ { "id": 728967, "postDate": "2020-01-25T14:12:58.693Z", "content": "

I'm really sorry to hear that. I guess the difference between public and private (and public-CV as well) would be smaller without some counters (like 4070, 3020, 3121), but I don't really sure that this is issue. Thanks for sharing your kernel, it's great!

", "rawMarkdown": "I'm really sorry to hear that. I guess the difference between public and private (and public-CV as well) would be smaller without some counters (like 4070, 3020, 3121), but I don't really sure that this is issue. Thanks for sharing your kernel, it's great! ", "votes": 1 } ] }, { "id": 726256, "postDate": "2020-01-23T00:23:22.553Z", "content": "

Congratulations.

", "rawMarkdown": "Congratulations.", "votes": 2 }, { "id": 728827, "postDate": "2020-01-25T10:49:11.613Z", "content": "

Congrats on winning! Thanks for sharing the approach.\nI didn't get this point by the way \"The test set can be labeled with the same procedure as a train one and can be used in training\". Could you help me on this?

", "rawMarkdown": "Congrats on winning! Thanks for sharing the approach.\nI didn't get this point by the way \"The test set can be labeled with the same procedure as a train one and can be used in training\". Could you help me on this?", "votes": 1, "replies": [ { "id": 728970, "postDate": "2020-01-25T14:15:12.663Z", "content": "

Sure. I mean a lot of installation id (~50%) in the test have long history including assessments with attempts. So we can calculate accuracy for those assessments and use those parts of history as additional training samples. Hope it helps.

", "rawMarkdown": "Sure. I mean a lot of installation id (~50%) in the test have long history including assessments with attempts. So we can calculate accuracy for those assessments and use those parts of history as additional training samples. Hope it helps.", "votes": 1 } ] }, { "id": 727058, "postDate": "2020-01-23T12:26:55.743Z", "content": "

Congratulations! We can learn lots of knowledge and experience from your work！Thanks for sharing！

", "rawMarkdown": "Congratulations! We can learn lots of knowledge and experience from your work！Thanks for sharing！", "votes": 1 }, { "id": 726465, "postDate": "2020-01-23T02:48:39.847Z", "content": "

I knew you were gonna win, congrats!

", "rawMarkdown": "I knew you were gonna win, congrats!", "votes": 1 }, { "id": 726449, "postDate": "2020-01-23T02:33:02.970Z", "content": "

There are some good codes in the kernel! Thank you for sharing

", "rawMarkdown": "There are some good codes in the kernel! Thank you for sharing", "votes": 1 }, { "id": 726432, "postDate": "2020-01-23T02:24:12.907Z", "content": "

Congrats @sergeifironov ! Your ideas inspired me and gave me confidence throughout this competition.

", "rawMarkdown": "Congrats @sergeifironov ! Your ideas inspired me and gave me confidence throughout this competition.", "votes": 1 }, { "id": 726420, "postDate": "2020-01-23T02:14:28.337Z", "content": "

Congrats @sergeifironov and thanks for sharing your solution!

", "rawMarkdown": "Congrats @sergeifironov and thanks for sharing your solution!", "votes": 1 }, { "id": 726359, "postDate": "2020-01-23T01:26:43.147Z", "content": "

Thank you so much for posting this!!! This is a super interesting approach and I look forward to digging through it in more detail!

", "rawMarkdown": "Thank you so much for posting this!!! This is a super interesting approach and I look forward to digging through it in more detail!", "votes": 1 }, { "id": 726311, "postDate": "2020-01-23T00:59:54.073Z", "content": "

Great share and thanks for the insights. Congrats Sergei!

", "rawMarkdown": "Great share and thanks for the insights. Congrats Sergei!", "votes": 1 }, { "id": 726300, "postDate": "2020-01-23T00:52:29.917Z", "content": "

Amazing solutions! Congrats!

", "rawMarkdown": "Amazing solutions! Congrats!", "votes": 1 }, { "id": 726290, "postDate": "2020-01-23T00:46:10.260Z", "content": "

Great job Sergei!

", "rawMarkdown": "Great job Sergei! ", "votes": 1 }, { "id": 735068, "postDate": "2020-02-02T14:05:58.197Z", "content": "

Congrats!\nInteresting to learn the approach by stacking and using of NN model.\nCould you share with me how much performance increased by stacking? \nIn your understanding, what is the most important factors (features? validation? NN model? or stacking?) you won?

", "rawMarkdown": "Congrats!\nInteresting to learn the approach by stacking and using of NN model.\nCould you share with me how much performance increased by stacking? \nIn your understanding, what is the most important factors (features? validation? NN model? or stacking?) you won? " }, { "id": 727247, "postDate": "2020-01-23T15:16:04.243Z", "content": "

Congratulations and thanks for sharing your experience and insights especially about correlation with LB. Brilliant!

", "rawMarkdown": "Congratulations and thanks for sharing your experience and insights especially about correlation with LB. Brilliant!" }, { "id": 727031, "postDate": "2020-01-23T11:50:06.933Z", "content": "

Congrats, great kernel, great work, very useful.

", "rawMarkdown": "Congrats, great kernel, great work, very useful." }, { "id": 726882, "postDate": "2020-01-23T09:48:51.840Z", "content": "

Congratulations! Amazing solutions!

", "rawMarkdown": "Congratulations! Amazing solutions!" }, { "id": 726820, "postDate": "2020-01-23T08:56:36.533Z", "content": "

Congratulations!

", "rawMarkdown": "Congratulations!" }, { "id": 726561, "postDate": "2020-01-23T04:43:07.790Z", "content": "

Great job! Congratulations!

", "rawMarkdown": "Great job! Congratulations! \n" }, { "id": 726469, "postDate": "2020-01-23T02:52:43.687Z", "content": "

Congratulations

", "rawMarkdown": "Congratulations\n" }, { "id": 726302, "postDate": "2020-01-23T00:53:48.050Z", "content": "

I see that most of your features are different from public kernels. How did you arrive at them?

", "rawMarkdown": "I see that most of your features are different from public kernels. How did you arrive at them?", "replies": [ { "id": 726314, "postDate": "2020-01-23T01:02:12.447Z", "content": "

My features are very simple and intuitive (from my point of view) for NN. We can't train on events, so we train on titles and use the most important information about it (number of attempts, ratio, time). Most public kernels contained feature preparation for tree-based models. Those features aren't totally impossible to incorporate in NN, but don't look so useful there. My teammate @simakov will write about tree-based models and feature engineering for them.

", "rawMarkdown": "My features are very simple and intuitive (from my point of view) for NN. We can't train on events, so we train on titles and use the most important information about it (number of attempts, ratio, time). Most public kernels contained feature preparation for tree-based models. Those features aren't totally impossible to incorporate in NN, but don't look so useful there. My teammate @simakov will write about tree-based models and feature engineering for them.", "votes": 2 }, { "id": 726445, "postDate": "2020-01-23T02:29:47.510Z", "content": "

That’s awesome @sergeifironov. The crazy thing is that I had a lot of these same features, even TFIDF on added columns, but I was not able to try a lot because of travel and I had a really bad bug in my code that I could not hunt down. I wanted to try the RNN on those sequences but didn’t have the time, seems like it was very fruitful. Congrats on the great finish!

", "rawMarkdown": "That’s awesome @sergeifironov. The crazy thing is that I had a lot of these same features, even TFIDF on added columns, but I was not able to try a lot because of travel and I had a really bad bug in my code that I could not hunt down. I wanted to try the RNN on those sequences but didn’t have the time, seems like it was very fruitful. Congrats on the great finish!", "votes": 1 }, { "id": 726447, "postDate": "2020-01-23T02:30:52.337Z", "content": "

Oh also @sergeifironov thank you for sharing the code :) I always learn a lot from you

", "rawMarkdown": "Oh also @sergeifironov thank you for sharing the code :) I always learn a lot from you " }, { "id": 726473, "postDate": "2020-01-23T02:59:07.680Z", "content": "

@rdizzl3 thanks! Some of ideas were inspired from your great work in the mercari competition, you see ;) And I thought at least twice that this competition fits you much. Sorry to hear about the problems.

", "rawMarkdown": "@rdizzl3 thanks! Some of ideas were inspired from your great work in the mercari competition, you see ;) And I thought at least twice that this competition fits you much. Sorry to hear about the problems.", "votes": 2 }, { "id": 726474, "postDate": "2020-01-23T03:01:30.563Z", "content": "

Thanks @sergeifironov! It's no worries :) I am always glad to see a friend do well in these competitions! But it is great to know that a lot of the techniques that worked for us there inspired a win here! I will spend some time to go through the code and pull all the gold nuggets out of it :) Congrats again on the great finish.

", "rawMarkdown": "Thanks @sergeifironov! It's no worries :) I am always glad to see a friend do well in these competitions! But it is great to know that a lot of the techniques that worked for us there inspired a win here! I will spend some time to go through the code and pull all the gold nuggets out of it :) Congrats again on the great finish.", "votes": 1 } ] }, { "id": 726509, "postDate": "2020-01-23T03:25:25.743Z", "rawMarkdown": "", "isDeleted": true }, { "id": 733267, "postDate": "2020-01-30T22:55:11.887Z", "content": "

Congrats and thank you for sharing.

", "rawMarkdown": "Congrats and thank you for sharing." }, { "id": 727625, "postDate": "2020-01-23T22:51:27.280Z", "content": "

Thanks for sharing!

", "rawMarkdown": "Thanks for sharing!" }, { "id": 726467, "postDate": "2020-01-23T02:50:14.720Z", "content": "

Wow thank you for sharing.

", "rawMarkdown": "Wow thank you for sharing." } ], "complete": true, "source": "live", "fetched_at": "2026-10-24T13:49:34Z", "competition_id": 16531, "competition_slug": "data-science-bowl-2019", "team_id": 3855216, "solution_url": "/competitions/data-science-bowl-2019/writeups/crystal-cave-miners-4th-private-7th-public-place-w" }