Congrats to all the participants in this great and challenging tabular competition! Thank you Kaggle for organizing this competition.
\nMy solution is very simple, NN plays a key role. The categorical features, especially with high cardinality are very suitable for neural network to exert its power.
\n4x NN, 1x Catboost, and blend of some of public kernels. All NN models on same features, and one of them resulted in 0.78672 on the public LB. Thanks to @sergey and @siavash for sharing the public kernels, I used them in blending.
\nThat’s all. I have also tried using various other methods to process the features but they all lead to overfitting.
\nNN uses several state-of-the-art models for CTR prediction, including CIN in xDeepFM, PNN, Cross in DCN, AutoInt, etc.
\n1.Linear+DNN+CIN (0.78672 public LB)
\n2.FM+Cross+PNN (0.78655 public LB)
\n3.FM+DCN+DNN (0.78652 public LB)
\n4.Linear+DNN+AutoInt (0.78665 public LB)
There are many components available for feature extraction on tabular data and they can be combined with various ways. It is a huge workload to trail by coding from scratch every time. Deeptables greatly simplifies this job with only a few lines of code.
\nhttps://github.com/DataCanvasIO/deeptables.
\nI have shared the NN code here.
", "comments": [ { "id": 794955, "author_name": "KRKirov", "author_url": "", "post_date": "2020-04-02T08:44:11.627000", "content": "Congratulations on your 1st place! It strikes me that your solution is only \"simple\" because it employs existing libraries and the code is very high-level. Otherwise it is very complicated and so much algorithmic power leads to a negligible improvement. The difference between my best private score based on logistic regression and the winning solution is 0.78820 - 0.78721 = 0.00099. ;-)
", "votes": 3, "replies": [] }, { "id": 794578, "author_name": "MICADEE", "author_url": "", "post_date": "2020-04-01T23:00:33.643000", "content": "@jackguagua Wow....... Not a small effort most of us must have put on this competition i must say and to have found out this very awesome and easy approach is very shocking and surprising. Congratulations. All your efforts, cracking of brain, rewriting codes all over again, manipulating and blending of some public kernels paid off for you eventually. \nCongrats👍
", "votes": 3, "replies": [ { "id": 795492, "author_name": "Solyoh21", "author_url": "", "post_date": "2020-04-02T18:55:27.630000", "content": "Thanks
", "votes": 1, "replies": [] } ] }, { "id": 817796, "author_name": "Usama Imran", "author_url": "", "post_date": "2020-04-23T12:26:42.777000", "content": "Nice
", "votes": 1, "replies": [] }, { "id": 825843, "author_name": "Jagdish Mirchandani", "author_url": "", "post_date": "2020-04-29T09:21:48.793000", "content": "excellent notebook, you made it look like so easy!
", "votes": 0, "replies": [] }, { "id": 814546, "author_name": "anirudha16101", "author_url": "", "post_date": "2020-04-20T18:51:55.800000", "content": "Nice work,very well explained.Thanks for sharing 👍
", "votes": 0, "replies": [] }, { "id": 813542, "author_name": "Adeyoyin Temidayo", "author_url": "", "post_date": "2020-04-19T18:41:52.307000", "content": "Nice Work!
", "votes": 0, "replies": [] }, { "id": 805785, "author_name": "Anatidae", "author_url": "", "post_date": "2020-04-13T05:01:39.990000", "content": "Nice elegant solution! Love your approach and your model plots!
", "votes": 0, "replies": [] }, { "id": 797236, "author_name": "Jeebz", "author_url": "", "post_date": "2020-04-04T10:54:58.293000", "content": "Very good solution!
", "votes": 0, "replies": [] }, { "id": 796679, "author_name": "LJL", "author_url": "", "post_date": "2020-04-03T20:00:10.330000", "content": "Congratulations first! It is a very simple and elegant approach. I spent a lot of time on imputing missing value and exploring feature interaction. They seems not working well.
\n\nJust curious what did you use for your highest score on public leaderboard.
\n\nBest regards
", "votes": 0, "replies": [] }, { "id": 794775, "author_name": "Stephen Lau", "author_url": "", "post_date": "2020-04-02T04:46:34.673000", "content": "And people say NNs don't work well on tabular data!
", "votes": 0, "replies": [] }, { "id": 794625, "author_name": "kailai", "author_url": "", "post_date": "2020-04-02T00:23:25.717000", "content": "Absolutely smart!
", "votes": 0, "replies": [] }, { "id": 801425, "author_name": "", "author_url": "", "post_date": "2020-04-08T13:25:33.470000", "content": "", "votes": 0, "replies": [] } ], "comment_count": 13, "messages": [ { "id": 794566, "postDate": "2020-04-01T22:39:36.117Z", "content": "Congrats to all the participants in this great and challenging tabular competition! Thank you Kaggle for organizing this competition.
\nMy solution is very simple, NN plays a key role. The categorical features, especially with high cardinality are very suitable for neural network to exert its power.
\n4x NN, 1x Catboost, and blend of some of public kernels. All NN models on same features, and one of them resulted in 0.78672 on the public LB. Thanks to @sergey and @siavash for sharing the public kernels, I used them in blending.
\nThat’s all. I have also tried using various other methods to process the features but they all lead to overfitting.
\nNN uses several state-of-the-art models for CTR prediction, including CIN in xDeepFM, PNN, Cross in DCN, AutoInt, etc.
\n1.Linear+DNN+CIN (0.78672 public LB)
\n2.FM+Cross+PNN (0.78655 public LB)
\n3.FM+DCN+DNN (0.78652 public LB)
\n4.Linear+DNN+AutoInt (0.78665 public LB)
There are many components available for feature extraction on tabular data and they can be combined with various ways. It is a huge workload to trail by coding from scratch every time. Deeptables greatly simplifies this job with only a few lines of code.
\nhttps://github.com/DataCanvasIO/deeptables.
\nI have shared the NN code here.
", "rawMarkdown": "Congrats to all the participants in this great and challenging tabular competition! Thank you Kaggle for organizing this competition.\n\nMy solution is very simple, NN plays a key role. The categorical features, especially with high cardinality are very suitable for neural network to exert its power.\n\n### About Models\n4x NN, 1x Catboost, and blend of some of public kernels. All NN models on same features, and one of them resulted in 0.78672 on the public LB. Thanks to @sergey and @siavash for sharing the public kernels, I used them in blending.\n\n### Feature engineering for NN models\n\n1. All features are converted into category type then Label Encoding\n2. Ordinal Encoding: ord_1~ord_5 -> ord_1_en~ord_5_en\n\nThat’s all. I have also tried using various other methods to process the features but they all lead to overfitting.\n\n### NN Models\nNN uses several state-of-the-art models for CTR prediction, including CIN in xDeepFM, PNN, Cross in DCN, AutoInt, etc.\n\n1.Linear+DNN+CIN (0.78672 public LB)\n2.FM+Cross+PNN (0.78655 public LB)\n3.FM+DCN+DNN (0.78652 public LB)\n4.Linear+DNN+AutoInt (0.78665 public LB)\n\nThere are many components available for feature extraction on tabular data and they can be combined with various ways. It is a huge workload to trail by coding from scratch every time. Deeptables greatly simplifies this job with only a few lines of code.\n\n [https://github.com/DataCanvasIO/deeptables](https://github.com/DataCanvasIO/deeptables).\n\n[I have shared the NN code here.](https://github.com/DataCanvasIO/DeepTables/blob/master/deeptables/examples/Kaggle%20-%20Categorical%20Feature%20Encoding%20Challenge%20II.ipynb)", "votes": 58 }, { "id": 794955, "postDate": "2020-04-02T08:44:11.627Z", "content": "Congratulations on your 1st place! It strikes me that your solution is only \"simple\" because it employs existing libraries and the code is very high-level. Otherwise it is very complicated and so much algorithmic power leads to a negligible improvement. The difference between my best private score based on logistic regression and the winning solution is 0.78820 - 0.78721 = 0.00099. ;-)
", "rawMarkdown": "Congratulations on your 1st place! It strikes me that your solution is only \"simple\" because it employs existing libraries and the code is very high-level. Otherwise it is very complicated and so much algorithmic power leads to a negligible improvement. The difference between my best private score based on logistic regression and the winning solution is 0.78820 - 0.78721 = 0.00099. ;-)", "votes": 3 }, { "id": 794578, "postDate": "2020-04-01T23:00:33.643Z", "content": "@jackguagua Wow....... Not a small effort most of us must have put on this competition i must say and to have found out this very awesome and easy approach is very shocking and surprising. Congratulations. All your efforts, cracking of brain, rewriting codes all over again, manipulating and blending of some public kernels paid off for you eventually. \nCongrats👍
", "rawMarkdown": "@jackguagua Wow....... Not a small effort most of us must have put on this competition i must say and to have found out this very awesome and easy approach is very shocking and surprising. Congratulations. All your efforts, cracking of brain, rewriting codes all over again, manipulating and blending of some public kernels paid off for you eventually. \nCongrats👍 ", "votes": 3, "replies": [ { "id": 795492, "postDate": "2020-04-02T18:55:27.630Z", "content": "Thanks
", "rawMarkdown": "Thanks", "votes": 1 } ] }, { "id": 817796, "postDate": "2020-04-23T12:26:42.777Z", "content": "Nice
", "rawMarkdown": "Nice", "votes": 1 }, { "id": 825843, "postDate": "2020-04-29T09:21:48.793Z", "content": "excellent notebook, you made it look like so easy!
", "rawMarkdown": "excellent notebook, you made it look like so easy! " }, { "id": 814546, "postDate": "2020-04-20T18:51:55.800Z", "content": "Nice work,very well explained.Thanks for sharing 👍
", "rawMarkdown": "Nice work,very well explained.Thanks for sharing 👍 " }, { "id": 813542, "postDate": "2020-04-19T18:41:52.307Z", "content": "Nice Work!
", "rawMarkdown": "Nice Work!" }, { "id": 805785, "postDate": "2020-04-13T05:01:39.990Z", "content": "Nice elegant solution! Love your approach and your model plots!
", "rawMarkdown": "Nice elegant solution! Love your approach and your model plots!" }, { "id": 797236, "postDate": "2020-04-04T10:54:58.293Z", "content": "Very good solution!
", "rawMarkdown": "Very good solution!" }, { "id": 796679, "postDate": "2020-04-03T20:00:10.330Z", "content": "Congratulations first! It is a very simple and elegant approach. I spent a lot of time on imputing missing value and exploring feature interaction. They seems not working well.
\n\nJust curious what did you use for your highest score on public leaderboard.
\n\nBest regards
", "rawMarkdown": "Congratulations first! It is a very simple and elegant approach. I spent a lot of time on imputing missing value and exploring feature interaction. They seems not working well.\n\nJust curious what did you use for your highest score on public leaderboard.\n\nBest regards" }, { "id": 794775, "postDate": "2020-04-02T04:46:34.673Z", "content": "And people say NNs don't work well on tabular data!
", "rawMarkdown": "And people say NNs don't work well on tabular data! " }, { "id": 794625, "postDate": "2020-04-02T00:23:25.717Z", "content": "Absolutely smart!
", "rawMarkdown": "Absolutely smart!" }, { "id": 801425, "postDate": "2020-04-08T13:25:33.470Z", "rawMarkdown": "", "isDeleted": true } ], "complete": true, "source": "live", "fetched_at": "2026-10-19T11:13:51Z", "competition_id": 17000, "competition_slug": "cat-in-the-dat-ii", "team_id": 4277049, "solution_url": "/competitions/cat-in-the-dat-ii/writeups/zetyun-com-dt-1st-place-solution" }