![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F1846168%2Ff8052d17308d8f21c32d010382ae1150%2F2020-01-24%200.07.44.png?generation=1579796947915005&alt=media\")
First of all, thanks to Booz Allen Hamilton and Kaggle team for such an interesting competition. And congratulations to all the winning teams and all the Kagglers who have worked hard and learned a lot throughout this competition.
\n\nWe ranked 38th in Public and 2nd in Private. These final results excited us and one of our teammates, @tiginkgo, has become a new Kaggle master :)
\n\nThe best model we chose achieved 0.563 for Public and also 0.563 for Private.
\n\nWord2Vec features of title series\n- Considering the series of course titles up to the target assessment as a document, processed them with word2vec and calculated the stats (mean/std/max/min) of the obtained vector.
\n\nHistorical feature\n- Count of (session, world, types, title, event_id, event_code) as historical data, grouped by (all, treetop, magma, crystal).\n- Count, mean, max of (event_round, game_time, event_count).
\n\nDecayed historical feature\n- Historical data decayed for (title, type, world, event_id, event_code).\n- Decrease accumulation by half for each session.
\n\nDensity of historical feature\n- The density of historical data for (title, type, world, event_id, event_code).\n- Density = (count) / (elapsed days from a first activated day).
\n\nLagged Assessment\n- Lots of stats (mean/std/...) of num_correct, num_incorrect, accuracy, accuracy_group.\n- The difference of hours from the past assessment.\n- Per full assessments, and per title assessments.
\n\nMeta Features\n- In order to denote “How having a game_session in advance can lead to an assessment result”, we created “meta target features” for each assessment title. We used oof for train data and KFold averages for the other data such as records without test or meta target.
\n\n
The 7th place solution of Elo Merchant Category Recommendation Competition gave us great inspiration, especially for our word2vec and meta features, which were very important parts of our solution.
\n\nWe are deeply grateful to @senkin13 and his excellent explanations are here:\nhttps://www.kaggle.com/c/elo-merchant-category-recommendation/discussion/82055\nhttps://www.slideshare.net/JinZhan/kaggle-days-tokyo-jin-zhan-204409794
", "comments": [ { "id": 740772, "author_name": "Darkate", "author_url": "", "post_date": "2020-02-09T19:00:45.783000", "content": "Congratz! Can you explain Decayed historical feature with more details? I saw you and 15th solution use it, but I don't quite understand how they were calculated... Thanks a lot!
", "votes": 2, "replies": [ { "id": 740899, "author_name": "yabea", "author_url": "", "post_date": "2020-02-09T23:45:31.683000", "content": "accum(t) = coef * accum(t-1) + feature(t)
\n\nfor every sessions of user's history, we decayed last accumulation of a feature by coef (we used 0.5), and add a current feature value.
", "votes": 1, "replies": [] }, { "id": 741050, "author_name": "Darkate", "author_url": "", "post_date": "2020-02-10T05:41:47.693000", "content": "Thanks for the reply and the explanation!
", "votes": 1, "replies": [] } ] }, { "id": 732433, "author_name": "None", "author_url": "", "post_date": "2020-01-29T20:21:52.627000", "content": "I have a question regarding the calculation of the meta features. In your example table there is a session G3 with bird assessment and accuracy_group=2. Do you create the same meta features as for G6 but with META_Bird=2?\nAlso for fitting the LGB Model will you use both meta feature tables and concatenate them?
", "votes": 2, "replies": [ { "id": 733246, "author_name": "yabea", "author_url": "", "post_date": "2020-01-30T22:13:00.403000", "content": "thank you for your questions. I’m sorry to be late for replying. here is the detail of build meta features,\n1. preprocess train.csv to get train_df.\n - a row has one game_session.\n2. copy train_df to get train_meta.\n3. for all users, propagate target of train_labels.csv to train_meta.\n - the way of deciding destination is referred to example and other comment reply.\n - you can get train_meta as an altered version of train_df with 5 meta target columns.\n4. train LGB on train_meta and replace metas by predicted values.\n - per each meta target.\n5. merge train_meta to train_labels.csv
", "votes": 0, "replies": [] }, { "id": 733469, "author_name": "None", "author_url": "", "post_date": "2020-01-31T07:05:10.033000", "content": "Thank you so much!\nThe thing I don't understand is how do you handle it if an installation I'd had two attempts to the same game with different accuracy, say 3 and 1?\nDo you go by the last assessment and assign 1 as meta feature or do you assign 1 and 3?
", "votes": 0, "replies": [] }, { "id": 734125, "author_name": "ZZ", "author_url": "", "post_date": "2020-02-01T00:06:00.677000", "content": "Thanks for sharing!\nFor the train_df of meta-features, what kind of features/columns you used to fit the LGB model for meta targets?
", "votes": 0, "replies": [] }, { "id": 734207, "author_name": "yabea", "author_url": "", "post_date": "2020-02-01T05:00:45.973000", "content": "@thomasx \nWe used the last assessment per each title. In the above example, G1 will be disregarded. You can get more info from an another thread https://www.kaggle.com/c/data-science-bowl-2019/discussion/127388#727967 .
", "votes": 0, "replies": [] }, { "id": 734212, "author_name": "yabea", "author_url": "", "post_date": "2020-02-01T05:05:32.117000", "content": "@zonemercy \nwe used the following features/columns.\n- title\n- type\n- world\n- max value of game_time in a session\n- max value of event_count in a session\n- max value of event_round (from event_data) in a session\n- sum of each event_id in a session\n- sum of each event_code in a session
Very thoughtful and impressive. Congrats! Is there anything special with your NN? The weight for ensemble is 0.3, which is relatively high for NN in this competition, I guess. Did it perform some multi-tasks?
", "votes": 2, "replies": [ { "id": 728615, "author_name": "yabea", "author_url": "", "post_date": "2020-01-25T00:57:39.920000", "content": "we just trained 3-layers MLP on regression task. nothing special like multi task theme you mentioned.
\n\nCongrats and thank you for sharing!!
\n\nCan I ask your Word2Vec features in detail？(I also tried to create w2v feature on title but rmse score was not improved...)\nWhat is the window size, dimensions of the embedding vector？
", "votes": 2, "replies": [ { "id": 727980, "author_name": "yabea", "author_url": "", "post_date": "2020-01-24T09:34:36.493000", "content": "We didn't focus on w2v parameters. we used quite simple setting as follows.\n\nWord2Vec(min_count=2, window=5, size=5, alpha=0.05, negative=5, seed=SEED, workers=1)\n\nTraining w2v seems not to be reproducible... docs says Note that for a fully deterministically-reproducible run, you must also limit the model to a single worker thread (workers=1) in seed section. but we cannot make it.
thx for your clear explanation!
", "votes": -1, "replies": [] } ] }, { "id": 727737, "author_name": "senkin13", "author_url": "", "post_date": "2020-01-24T02:03:37.287000", "content": "Thanks for sharing and congrats,I didn't try w2v and meta features,that's regrettable 😂
", "votes": 2, "replies": [] }, { "id": 727706, "author_name": "u++", "author_url": "", "post_date": "2020-01-24T01:30:02.153000", "content": "Congrats for your team! Can I ask your NN architecture?
", "votes": 2, "replies": [ { "id": 727977, "author_name": "yabea", "author_url": "", "post_date": "2020-01-24T09:28:47.420000", "content": "we adopt 3-layers MLP. here it is. \n```
\n\ndense_2 (Dense) (None, 64) 12480
\n\nbatch_normalization (BatchNo (None, 64) 256
\n\np_re_lu (PReLU) (None, 64) 64
\n\ndropout_1 (Dropout) (None, 64) 0
\n\ndense_3 (Dense) (None, 64) 4160
\n\nbatch_normalization_1 (Batch (None, 64) 256
\n\np_re_lu_1 (PReLU) (None, 64) 64
\n\ndropout_2 (Dropout) (None, 64) 0
\n\ndense_4 (Dense) (None, 64) 4160
\n\nbatch_normalization_2 (Batch (None, 64) 256
\n\np_re_lu_2 (PReLU) (None, 64) 64
\n\ndropout_3 (Dropout) (None, 64) 0
\n\nTotal params: 21,825\nTrainable params: 21,441\nNon-trainable params: 384
\n\n```
", "votes": 4, "replies": [] } ] }, { "id": 727365, "author_name": "MhdSharuk", "author_url": "", "post_date": "2020-01-23T16:50:48.943000", "content": "I really liked what you did there in Meta features.\nCongrats on winning the competition
", "votes": 2, "replies": [] }, { "id": 737148, "author_name": "Yuan", "author_url": "", "post_date": "2020-02-05T00:53:21.600000", "content": "Congratz and thx for the sharing! It is really helpful for learning!👍 🤘
", "votes": 1, "replies": [] }, { "id": 731073, "author_name": "Subbu Vidyasekar", "author_url": "", "post_date": "2020-01-28T10:16:50.920000", "content": "thanks for the post. you explained the feature engineering steps clearly. Congratulations.\nEven I thought to use event_data(json) as features after exploding. since this is my first competition, I had some hesitation to explore the json. your post opened the good direction to me. thanks a lot.
", "votes": 1, "replies": [ { "id": 731253, "author_name": "yabea", "author_url": "", "post_date": "2020-01-28T13:11:55.220000", "content": "we hesitated to explore event_data too. we just quick looked it and fetch only event_round that sounds important. I'm not sure but event_data may have lots of treasures...
", "votes": 0, "replies": [] } ] }, { "id": 730872, "author_name": "Nick Showalter", "author_url": "", "post_date": "2020-01-28T03:22:15.683000", "content": "Thanks for sharing! Very insightful how you implemented the idea of word2vec from a fellow competitor and achieved 2nd place. It just goes to show the importance of transparency in the Kaggle community.
", "votes": 1, "replies": [] }, { "id": 728646, "author_name": "Nathan Barry", "author_url": "", "post_date": "2020-01-25T02:34:32.607000", "content": "Awesome dude, congrats!
", "votes": 1, "replies": [] }, { "id": 728542, "author_name": "Alan Lahoud", "author_url": "", "post_date": "2020-01-24T21:59:04.157000", "content": "Congrats!
", "votes": 1, "replies": [] }, { "id": 728261, "author_name": "Yust", "author_url": "", "post_date": "2020-01-24T14:48:00.347000", "content": "congratulations! \nThanks for the clear explanation!!
", "votes": 1, "replies": [] }, { "id": 728164, "author_name": "Miyabon", "author_url": "", "post_date": "2020-01-24T13:21:17.333000", "content": "Congrats & Thanks for sharing your solution🎉 😄 👍
", "votes": 1, "replies": [] }, { "id": 727780, "author_name": "Massoud Hosseinali", "author_url": "", "post_date": "2020-01-24T03:45:28.257000", "content": "Congratulations. Are you going to opensource your solution?
", "votes": 1, "replies": [ { "id": 727985, "author_name": "yabea", "author_url": "", "post_date": "2020-01-24T09:40:25.323000", "content": "sorry, our code is so messy so far, we cannot share our source code...
", "votes": 1, "replies": [] }, { "id": 728373, "author_name": "Massoud Hosseinali", "author_url": "", "post_date": "2020-01-24T16:47:17.633000", "content": "Thank you. In building the metadata, where you said \"build LGB for each meta target, and replace all values by oof predictions\" what are the features you used in those LGB models to predict meta targets? all other features you have including other meta targets or a specific set of features related to each assessment?
\n\nBy the way, did you use the test set for training or just the train.csv file?
", "votes": 0, "replies": [] }, { "id": 728599, "author_name": "yabea", "author_url": "", "post_date": "2020-01-25T00:29:29.760000", "content": "we appreciate your understanding.
\n\nwe use features (title, type, world, count of event_code, count of event_id, game_time, event_count, event_round) to build LGB for meta target (note that we preprocessed raw train.csv data such that a row has a summary of a game_session). validation strategy is stratified group kfold on installation_id.
\n\nwe didn't use test data for train. we tried it, but it didn't improve our public lb (private too). we didn't select it for final submissions.
", "votes": 1, "replies": [] }, { "id": 728620, "author_name": "Massoud Hosseinali", "author_url": "", "post_date": "2020-01-25T01:03:38.730000", "content": "Thanks.
", "votes": 0, "replies": [] } ] }, { "id": 727709, "author_name": "ML_Bear", "author_url": "", "post_date": "2020-01-24T01:31:40.433000", "content": "Congrats! Thanks for sharing!\nI was very impressed with your feature engineering.
", "votes": 1, "replies": [] }, { "id": 727693, "author_name": "Sahar S", "author_url": "", "post_date": "2020-01-24T01:08:12.973000", "content": "congrats!
", "votes": 1, "replies": [] }, { "id": 727662, "author_name": "pao", "author_url": "", "post_date": "2020-01-24T00:23:50.047000", "content": "Congrats and thank you for sharing!!
\n\nI have a question.\nWhen you use Word2Vec, did you make feature by all data? or oof data?\nWhen I tried w2v (my case is event code sequence. ), I failed to improve because of leakage by using all data. \nI think tltle sequence is not included target info but using future info in train data. I am interested it is no problem or not.
", "votes": 1, "replies": [ { "id": 727973, "author_name": "yabea", "author_url": "", "post_date": "2020-01-24T09:24:07.033000", "content": "we made title w2v features using all data.
\n\nwe made title sequences as following,\n- user A has history of title (t1, t2, t3, t4, t5), t3 and t5 are corresponded to assessment we want predict.\n- from above record, we made 2 title sequences, like (t1-t2-t3) and (t1-t2-t3-t4-t5).
\n\nin our strategy, each sequence cannot have future information. That's why there is no problem.
", "votes": 2, "replies": [] }, { "id": 730808, "author_name": "pao", "author_url": "", "post_date": "2020-01-28T00:25:05.820000", "content": "Thank you for clear answer!!\nI think if the sequences contains target info, it makes leakage(like different cv stackieg). But your solution which use titie is no problem!!
", "votes": 0, "replies": [] } ] }, { "id": 727660, "author_name": "narsil (jobs-in-data.com)", "author_url": "", "post_date": "2020-01-24T00:16:01.987000", "content": "Congrats. Great stuff and write-up!
", "votes": 1, "replies": [] }, { "id": 727627, "author_name": "Paulo S. Santos", "author_url": "", "post_date": "2020-01-23T23:06:43.257000", "content": "@yukia18 i did a world training too, but, not in the same way than you, and i really don't understand why magma had a worst score and didn't train well in any of my models, did you find the same weirdness on your model training with separation by world? and
\n\ngrats for 2nd, i didn't get any medal, because my best score (549) wasn't choosen
", "votes": 1, "replies": [ { "id": 727969, "author_name": "yabea", "author_url": "", "post_date": "2020-01-24T09:18:02.393000", "content": "we didn't separate model per world, just adding features aggregated per world.
", "votes": 0, "replies": [] }, { "id": 728282, "author_name": "Paulo S. Santos", "author_url": "", "post_date": "2020-01-24T15:02:32.607000", "content": "ahh gotcha, thx for the explanation
", "votes": 0, "replies": [] } ] }, { "id": 727576, "author_name": "Y.Nakama", "author_url": "", "post_date": "2020-01-23T20:47:34.793000", "content": "Congrats!
", "votes": 1, "replies": [] }, { "id": 727552, "author_name": "Luke Gray", "author_url": "", "post_date": "2020-01-23T19:59:22.777000", "content": "Congrats!! Great work.
", "votes": 1, "replies": [] }, { "id": 727506, "author_name": "isma", "author_url": "", "post_date": "2020-01-23T18:52:56.580000", "content": "Congratulations on your 2nd place! Really liked the word2vec idea to convert a sequence into a vector :)
\n\nI am struggling to understand the meta features.
\n\nIn the visual example that you kindly shared, the user A undertakes twice the Bird assessment, once getting an accuracy of 2 in session G3 and then 3 in session G6. If I understand correctly how you get to your meta features training set, it would mean that there would be twice the row G1 session with one time a MetaBird target score of 2 and one with a score of 3 while all other columns are the same.\nAm I correct?
", "votes": 1, "replies": [ { "id": 727967, "author_name": "yabea", "author_url": "", "post_date": "2020-01-24T09:16:40.957000", "content": "yes, as you said, Meta Bird of G1 and G2 can be filled with 2 from G3 results (see below). But we considered that all sessions have effect to final assessment results, so we decided overwriting Meta Bird of G1 and G2 by 3 from G6 results.\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F1846168%2F765ba9604773c76f4fb3ccce4d0b92ed%2F2020-01-24%2018.06.49.png?generation=1579857387908782&alt=media\")
Thanks for sharing!
\n\nI wonder if you used adversarial validation in your feature selection as the 1st place team did?
\n\nI tried it myself and easily built a model that separated train from test with accuracy 100%. Using the features from this model is likely to overfit on train.
", "votes": 0, "replies": [ { "id": 817365, "author_name": "yabea", "author_url": "", "post_date": "2020-04-23T05:26:49.100000", "content": "I'm sorry that I didn't notice your question.
\n\nWe actually tried adversarial validation on our 300 features. My model could distinguish train and test easily (I forget a score of AUC).
\n\nBut public/private score wasn't decreased so much.
", "votes": 1, "replies": [] } ] }, { "id": 736518, "author_name": "", "author_url": "", "post_date": "2020-02-04T09:43:27.220000", "content": "Congrats and thanks for sharing!
", "votes": 1, "replies": [] }, { "id": 732562, "author_name": "corochann", "author_url": "", "post_date": "2020-01-30T00:01:13.417000", "content": "Congrats and thanks for sharing!
", "votes": 1, "replies": [] }, { "id": 728584, "author_name": "Thiago Vilas Boas Paiva", "author_url": "", "post_date": "2020-01-24T23:42:12.333000", "content": "Thanks for the solution! Congrats!
", "votes": 1, "replies": [] }, { "id": 727895, "author_name": "Manoj Vinod", "author_url": "", "post_date": "2020-01-24T06:59:43.707000", "content": "Thank you for sharing
", "votes": 1, "replies": [] }, { "id": 727767, "author_name": "MeeCreeps", "author_url": "", "post_date": "2020-01-24T03:05:45.610000", "content": "Thank you for sharing!
", "votes": 1, "replies": [] } ], "comment_count": 51, "messages": [ { "id": 727334, "postDate": "2020-01-23T16:30:47.860Z", "content": "First of all, thanks to Booz Allen Hamilton and Kaggle team for such an interesting competition. And congratulations to all the winning teams and all the Kagglers who have worked hard and learned a lot throughout this competition.
\n\nWe ranked 38th in Public and 2nd in Private. These final results excited us and one of our teammates, @tiginkgo, has become a new Kaggle master :)
\n\nThe best model we chose achieved 0.563 for Public and also 0.563 for Private.
\n\nWord2Vec features of title series\n- Considering the series of course titles up to the target assessment as a document, processed them with word2vec and calculated the stats (mean/std/max/min) of the obtained vector.
\n\nHistorical feature\n- Count of (session, world, types, title, event_id, event_code) as historical data, grouped by (all, treetop, magma, crystal).\n- Count, mean, max of (event_round, game_time, event_count).
\n\nDecayed historical feature\n- Historical data decayed for (title, type, world, event_id, event_code).\n- Decrease accumulation by half for each session.
\n\nDensity of historical feature\n- The density of historical data for (title, type, world, event_id, event_code).\n- Density = (count) / (elapsed days from a first activated day).
\n\nLagged Assessment\n- Lots of stats (mean/std/...) of num_correct, num_incorrect, accuracy, accuracy_group.\n- The difference of hours from the past assessment.\n- Per full assessments, and per title assessments.
\n\nMeta Features\n- In order to denote “How having a game_session in advance can lead to an assessment result”, we created “meta target features” for each assessment title. We used oof for train data and KFold averages for the other data such as records without test or meta target.
\n\n
The 7th place solution of Elo Merchant Category Recommendation Competition gave us great inspiration, especially for our word2vec and meta features, which were very important parts of our solution.
\n\nWe are deeply grateful to @senkin13 and his excellent explanations are here:\nhttps://www.kaggle.com/c/elo-merchant-category-recommendation/discussion/82055\nhttps://www.slideshare.net/JinZhan/kaggle-days-tokyo-jin-zhan-204409794
", "rawMarkdown": "First of all, thanks to Booz Allen Hamilton and Kaggle team for such an interesting competition. And congratulations to all the winning teams and all the Kagglers who have worked hard and learned a lot throughout this competition. \n\nWe ranked 38th in Public and 2nd in Private. These final results excited us and one of our teammates, @tiginkgo, has become a new Kaggle master :)\n\n## Results\nThe best model we chose achieved 0.563 for Public and also 0.563 for Private. \n\n## Feature Engineering \n**Word2Vec features of title series**\n- Considering the series of course titles up to the target assessment as a document, processed them with word2vec and calculated the stats (mean/std/max/min) of the obtained vector.\n\n**Historical feature**\n- Count of (session, world, types, title, event\\_id, event\\_code) as historical data, grouped by (all, treetop, magma, crystal).\n- Count, mean, max of (event\\_round, game\\_time, event\\_count).\n\n**Decayed historical feature**\n- Historical data decayed for (title, type, world, event\\_id, event\\_code).\n- Decrease accumulation by half for each session.\n\n**Density of historical feature**\n- The density of historical data for (title, type, world, event\\_id, event\\_code).\n- Density = (count) / (elapsed days from a first activated day).\n\n**Lagged Assessment**\n- Lots of stats (mean/std/...) of num\\_correct, num\\_incorrect, accuracy, accuracy\\_group.\n- The difference of hours from the past assessment.\n- Per full assessments, and per title assessments.\n\n**Meta Features**\n- In order to denote “How having a game_session in advance can lead to an assessment result”, we created “meta target features” for each assessment title. We used oof for train data and KFold averages for the other data such as records without test or meta target.\n\n\n\n## Feature Selection \n- Delete duplicate columns.\n- Delete high-correlated columns (over 0.99).\n- Finally, fetch top 300 features scored by null importance.\n\n## Modeling\n- For the validation set, we resampled to ensure one sample per one user.\n- StratifiedGroupKFold, 5-fold.\n- RSA (5 random seed) of LGB, CB, and NN.\n\n## Post Processing\n- Ensemble = 0.5 * LGB + 0.2 * CB + 0.3 * NN.\n- Set the threshold to optimize cv qwk.\n\n## Special thanks\nThe 7th place solution of Elo Merchant Category Recommendation Competition gave us great inspiration, especially for our word2vec and meta features, which were very important parts of our solution.\n\nWe are deeply grateful to @senkin13 and his excellent explanations are here:\nhttps://www.kaggle.com/c/elo-merchant-category-recommendation/discussion/82055\nhttps://www.slideshare.net/JinZhan/kaggle-days-tokyo-jin-zhan-204409794 ", "votes": 114 }, { "id": 740772, "postDate": "2020-02-09T19:00:45.783Z", "content": "Congratz! Can you explain Decayed historical feature with more details? I saw you and 15th solution use it, but I don't quite understand how they were calculated... Thanks a lot!
", "rawMarkdown": "Congratz! Can you explain **Decayed historical feature** with more details? I saw you and 15th solution use it, but I don't quite understand how they were calculated... Thanks a lot!", "votes": 2, "replies": [ { "id": 740899, "postDate": "2020-02-09T23:45:31.683Z", "content": "accum(t) = coef * accum(t-1) + feature(t)
\n\nfor every sessions of user's history, we decayed last accumulation of a feature by coef (we used 0.5), and add a current feature value.
", "rawMarkdown": "accum(t) = coef * accum(t-1) + feature(t)\n\nfor every sessions of user's history, we decayed last accumulation of a feature by coef (we used 0.5), and add a current feature value.", "votes": 1 }, { "id": 741050, "postDate": "2020-02-10T05:41:47.693Z", "content": "Thanks for the reply and the explanation!
", "rawMarkdown": "Thanks for the reply and the explanation!", "votes": 1 } ] }, { "id": 732433, "postDate": "2020-01-29T20:21:52.627Z", "content": "I have a question regarding the calculation of the meta features. In your example table there is a session G3 with bird assessment and accuracy_group=2. Do you create the same meta features as for G6 but with META_Bird=2?\nAlso for fitting the LGB Model will you use both meta feature tables and concatenate them?
", "rawMarkdown": "I have a question regarding the calculation of the meta features. In your example table there is a session G3 with bird assessment and accuracy_group=2. Do you create the same meta features as for G6 but with META_Bird=2?\nAlso for fitting the LGB Model will you use both meta feature tables and concatenate them?", "votes": 2, "replies": [ { "id": 733246, "postDate": "2020-01-30T22:13:00.403Z", "content": "thank you for your questions. I’m sorry to be late for replying. here is the detail of build meta features,\n1. preprocess train.csv to get train_df.\n - a row has one game_session.\n2. copy train_df to get train_meta.\n3. for all users, propagate target of train_labels.csv to train_meta.\n - the way of deciding destination is referred to example and other comment reply.\n - you can get train_meta as an altered version of train_df with 5 meta target columns.\n4. train LGB on train_meta and replace metas by predicted values.\n - per each meta target.\n5. merge train_meta to train_labels.csv
", "rawMarkdown": "thank you for your questions. I’m sorry to be late for replying. here is the detail of build meta features,\n1. preprocess train.csv to get train\\_df.\n - a row has one game\\_session.\n2. copy train\\_df to get train\\_meta.\n3. for all users, propagate target of train\\_labels.csv to train\\_meta.\n - the way of deciding destination is referred to example and other comment reply.\n - you can get train\\_meta as an altered version of train\\_df with 5 meta target columns.\n4. train LGB on train\\_meta and replace metas by predicted values.\n - per each meta target.\n5. merge train\\_meta to train\\_labels.csv" }, { "id": 733469, "postDate": "2020-01-31T07:05:10.033Z", "content": "Thank you so much!\nThe thing I don't understand is how do you handle it if an installation I'd had two attempts to the same game with different accuracy, say 3 and 1?\nDo you go by the last assessment and assign 1 as meta feature or do you assign 1 and 3?
", "rawMarkdown": "Thank you so much!\nThe thing I don't understand is how do you handle it if an installation I'd had two attempts to the same game with different accuracy, say 3 and 1?\nDo you go by the last assessment and assign 1 as meta feature or do you assign 1 and 3?" }, { "id": 734125, "postDate": "2020-02-01T00:06:00.677Z", "content": "Thanks for sharing!\nFor the train_df of meta-features, what kind of features/columns you used to fit the LGB model for meta targets?
", "rawMarkdown": "Thanks for sharing!\nFor the train_df of meta-features, what kind of features/columns you used to fit the LGB model for meta targets?" }, { "id": 734207, "postDate": "2020-02-01T05:00:45.973Z", "content": "@thomasx \nWe used the last assessment per each title. In the above example, G1 will be disregarded. You can get more info from an another thread https://www.kaggle.com/c/data-science-bowl-2019/discussion/127388#727967 .
", "rawMarkdown": "@thomasx \nWe used the last assessment per each title. In the above example, G1 will be disregarded. You can get more info from an another thread https://www.kaggle.com/c/data-science-bowl-2019/discussion/127388#727967 ." }, { "id": 734212, "postDate": "2020-02-01T05:05:32.117Z", "content": "@zonemercy \nwe used the following features/columns.\n- title\n- type\n- world\n- max value of game_time in a session\n- max value of event_count in a session\n- max value of event_round (from event_data) in a session\n- sum of each event_id in a session\n- sum of each event_code in a session
Very thoughtful and impressive. Congrats! Is there anything special with your NN? The weight for ensemble is 0.3, which is relatively high for NN in this competition, I guess. Did it perform some multi-tasks?
", "rawMarkdown": "Very thoughtful and impressive. Congrats! Is there anything special with your NN? The weight for ensemble is 0.3, which is relatively high for NN in this competition, I guess. Did it perform some multi-tasks?", "votes": 2, "replies": [ { "id": 728615, "postDate": "2020-01-25T00:57:39.920Z", "content": "we just trained 3-layers MLP on regression task. nothing special like multi task theme you mentioned.
\n\nCongrats and thank you for sharing!!
\n\nCan I ask your Word2Vec features in detail？(I also tried to create w2v feature on title but rmse score was not improved...)\nWhat is the window size, dimensions of the embedding vector？
", "rawMarkdown": "Congrats and thank you for sharing!!\n\nCan I ask your Word2Vec features in detail？(I also tried to create w2v feature on title but rmse score was not improved...)\nWhat is the window size, dimensions of the embedding vector？", "votes": 2, "replies": [ { "id": 727980, "postDate": "2020-01-24T09:34:36.493Z", "content": "We didn't focus on w2v parameters. we used quite simple setting as follows.\n\nWord2Vec(min_count=2, window=5, size=5, alpha=0.05, negative=5, seed=SEED, workers=1)\n\nTraining w2v seems not to be reproducible... docs says Note that for a fully deterministically-reproducible run, you must also limit the model to a single worker thread (workers=1) in seed section. but we cannot make it.
thx for your clear explanation!
", "rawMarkdown": "thx for your clear explanation!", "votes": -1 } ] }, { "id": 727737, "postDate": "2020-01-24T02:03:37.287Z", "content": "Thanks for sharing and congrats,I didn't try w2v and meta features,that's regrettable 😂
", "rawMarkdown": "Thanks for sharing and congrats,I didn't try w2v and meta features,that's regrettable 😂 ", "votes": 2 }, { "id": 727706, "postDate": "2020-01-24T01:30:02.153Z", "content": "Congrats for your team! Can I ask your NN architecture?
", "rawMarkdown": "Congrats for your team! Can I ask your NN architecture? ", "votes": 2, "replies": [ { "id": 727977, "postDate": "2020-01-24T09:28:47.420Z", "content": "we adopt 3-layers MLP. here it is. \n```
\n\ndense_2 (Dense) (None, 64) 12480
\n\nbatch_normalization (BatchNo (None, 64) 256
\n\np_re_lu (PReLU) (None, 64) 64
\n\ndropout_1 (Dropout) (None, 64) 0
\n\ndense_3 (Dense) (None, 64) 4160
\n\nbatch_normalization_1 (Batch (None, 64) 256
\n\np_re_lu_1 (PReLU) (None, 64) 64
\n\ndropout_2 (Dropout) (None, 64) 0
\n\ndense_4 (Dense) (None, 64) 4160
\n\nbatch_normalization_2 (Batch (None, 64) 256
\n\np_re_lu_2 (PReLU) (None, 64) 64
\n\ndropout_3 (Dropout) (None, 64) 0
\n\nTotal params: 21,825\nTrainable params: 21,441\nNon-trainable params: 384
\n\n```
", "rawMarkdown": "we adopt 3-layers MLP. here it is. \n```\n_________________________________________________________________\nLayer (type) Output Shape Param # \n=================================================================\ndense_2 (Dense) (None, 64) 12480 \n_________________________________________________________________\nbatch_normalization (BatchNo (None, 64) 256 \n_________________________________________________________________\np_re_lu (PReLU) (None, 64) 64 \n_________________________________________________________________\ndropout_1 (Dropout) (None, 64) 0 \n_________________________________________________________________\ndense_3 (Dense) (None, 64) 4160 \n_________________________________________________________________\nbatch_normalization_1 (Batch (None, 64) 256 \n_________________________________________________________________\np_re_lu_1 (PReLU) (None, 64) 64 \n_________________________________________________________________\ndropout_2 (Dropout) (None, 64) 0 \n_________________________________________________________________\ndense_4 (Dense) (None, 64) 4160 \n_________________________________________________________________\nbatch_normalization_2 (Batch (None, 64) 256 \n_________________________________________________________________\np_re_lu_2 (PReLU) (None, 64) 64 \n_________________________________________________________________\ndropout_3 (Dropout) (None, 64) 0 \n_________________________________________________________________\ndense_5 (Dense) (None, 1) 65 \n=================================================================\nTotal params: 21,825\nTrainable params: 21,441\nNon-trainable params: 384\n_________________________________________________________________\n```", "votes": 4 } ] }, { "id": 727365, "postDate": "2020-01-23T16:50:48.943Z", "content": "I really liked what you did there in Meta features.\nCongrats on winning the competition
", "rawMarkdown": "I really liked what you did there in Meta features.\nCongrats on winning the competition", "votes": 2 }, { "id": 737148, "postDate": "2020-02-05T00:53:21.600Z", "content": "Congratz and thx for the sharing! It is really helpful for learning!👍 🤘
", "rawMarkdown": "Congratz and thx for the sharing! It is really helpful for learning!👍 🤘 ", "votes": 1 }, { "id": 731073, "postDate": "2020-01-28T10:16:50.920Z", "content": "thanks for the post. you explained the feature engineering steps clearly. Congratulations.\nEven I thought to use event_data(json) as features after exploding. since this is my first competition, I had some hesitation to explore the json. your post opened the good direction to me. thanks a lot.
", "rawMarkdown": "thanks for the post. you explained the feature engineering steps clearly. Congratulations.\nEven I thought to use event_data(json) as features after exploding. since this is my first competition, I had some hesitation to explore the json. your post opened the good direction to me. thanks a lot.", "votes": 1, "replies": [ { "id": 731253, "postDate": "2020-01-28T13:11:55.220Z", "content": "we hesitated to explore event_data too. we just quick looked it and fetch only event_round that sounds important. I'm not sure but event_data may have lots of treasures...
", "rawMarkdown": "we hesitated to explore event\\_data too. we just quick looked it and fetch only event\\_round that sounds important. I'm not sure but event_data may have lots of treasures..." } ] }, { "id": 730872, "postDate": "2020-01-28T03:22:15.683Z", "content": "Thanks for sharing! Very insightful how you implemented the idea of word2vec from a fellow competitor and achieved 2nd place. It just goes to show the importance of transparency in the Kaggle community.
", "rawMarkdown": "Thanks for sharing! Very insightful how you implemented the idea of word2vec from a fellow competitor and achieved 2nd place. It just goes to show the importance of transparency in the Kaggle community. ", "votes": 1 }, { "id": 728646, "postDate": "2020-01-25T02:34:32.607Z", "content": "Awesome dude, congrats!
", "rawMarkdown": "Awesome dude, congrats!", "votes": 1 }, { "id": 728542, "postDate": "2020-01-24T21:59:04.157Z", "content": "Congrats!
", "rawMarkdown": "Congrats!", "votes": 1 }, { "id": 728261, "postDate": "2020-01-24T14:48:00.347Z", "content": "congratulations! \nThanks for the clear explanation!!
", "rawMarkdown": "congratulations! \nThanks for the clear explanation!!", "votes": 1 }, { "id": 728164, "postDate": "2020-01-24T13:21:17.333Z", "content": "Congrats & Thanks for sharing your solution🎉 😄 👍
", "rawMarkdown": "Congrats & Thanks for sharing your solution🎉 😄 👍 ", "votes": 1 }, { "id": 727780, "postDate": "2020-01-24T03:45:28.257Z", "content": "Congratulations. Are you going to opensource your solution?
", "rawMarkdown": "Congratulations. Are you going to opensource your solution?", "votes": 1, "replies": [ { "id": 727985, "postDate": "2020-01-24T09:40:25.323Z", "content": "sorry, our code is so messy so far, we cannot share our source code...
", "rawMarkdown": "sorry, our code is so messy so far, we cannot share our source code...", "votes": 1 }, { "id": 728373, "postDate": "2020-01-24T16:47:17.633Z", "content": "Thank you. In building the metadata, where you said \"build LGB for each meta target, and replace all values by oof predictions\" what are the features you used in those LGB models to predict meta targets? all other features you have including other meta targets or a specific set of features related to each assessment?
\n\nBy the way, did you use the test set for training or just the train.csv file?
", "rawMarkdown": "Thank you. In building the metadata, where you said \"*build LGB for each meta target, and replace all values by oof predictions*\" what are the features you used in those LGB models to predict meta targets? all other features you have including other meta targets or a specific set of features related to each assessment?\n\nBy the way, did you use the test set for training or just the train.csv file?" }, { "id": 728599, "postDate": "2020-01-25T00:29:29.760Z", "content": "we appreciate your understanding.
\n\nwe use features (title, type, world, count of event_code, count of event_id, game_time, event_count, event_round) to build LGB for meta target (note that we preprocessed raw train.csv data such that a row has a summary of a game_session). validation strategy is stratified group kfold on installation_id.
\n\nwe didn't use test data for train. we tried it, but it didn't improve our public lb (private too). we didn't select it for final submissions.
", "rawMarkdown": "we appreciate your understanding.\n\nwe use features (title, type, world, count of event\\_code, count of event\\_id, game\\_time, event\\_count, event\\_round) to build LGB for meta target (note that we preprocessed raw train.csv data such that a row has a summary of a game\\_session). validation strategy is stratified group kfold on installation\\_id.\n\nwe didn't use test data for train. we tried it, but it didn't improve our public lb (private too). we didn't select it for final submissions.", "votes": 1 }, { "id": 728620, "postDate": "2020-01-25T01:03:38.730Z", "content": "Thanks.
", "rawMarkdown": "Thanks." } ] }, { "id": 727709, "postDate": "2020-01-24T01:31:40.433Z", "content": "Congrats! Thanks for sharing!\nI was very impressed with your feature engineering.
", "rawMarkdown": "Congrats! Thanks for sharing!\nI was very impressed with your feature engineering.", "votes": 1 }, { "id": 727693, "postDate": "2020-01-24T01:08:12.973Z", "content": "congrats!
", "rawMarkdown": "congrats!", "votes": 1 }, { "id": 727662, "postDate": "2020-01-24T00:23:50.047Z", "content": "Congrats and thank you for sharing!!
\n\nI have a question.\nWhen you use Word2Vec, did you make feature by all data? or oof data?\nWhen I tried w2v (my case is event code sequence. ), I failed to improve because of leakage by using all data. \nI think tltle sequence is not included target info but using future info in train data. I am interested it is no problem or not.
", "rawMarkdown": "Congrats and thank you for sharing!!\n\nI have a question.\nWhen you use Word2Vec, did you make feature by all data? or oof data?\nWhen I tried w2v (my case is event code sequence. ), I failed to improve because of leakage by using all data. \nI think tltle sequence is not included target info but using future info in train data. I am interested it is no problem or not. ", "votes": 1, "replies": [ { "id": 727973, "postDate": "2020-01-24T09:24:07.033Z", "content": "we made title w2v features using all data.
\n\nwe made title sequences as following,\n- user A has history of title (t1, t2, t3, t4, t5), t3 and t5 are corresponded to assessment we want predict.\n- from above record, we made 2 title sequences, like (t1-t2-t3) and (t1-t2-t3-t4-t5).
\n\nin our strategy, each sequence cannot have future information. That's why there is no problem.
", "rawMarkdown": "we made title w2v features using all data.\n\nwe made title sequences as following,\n- user A has history of title (t1, t2, t3, t4, t5), t3 and t5 are corresponded to assessment we want predict.\n- from above record, we made 2 title sequences, like (t1-t2-t3) and (t1-t2-t3-t4-t5).\n\nin our strategy, each sequence cannot have future information. That's why there is no problem.", "votes": 2 }, { "id": 730808, "postDate": "2020-01-28T00:25:05.820Z", "content": "Thank you for clear answer!!\nI think if the sequences contains target info, it makes leakage(like different cv stackieg). But your solution which use titie is no problem!!
", "rawMarkdown": "Thank you for clear answer!!\nI think if the sequences contains target info, it makes leakage(like different cv stackieg). But your solution which use titie is no problem!!" } ] }, { "id": 727660, "postDate": "2020-01-24T00:16:01.987Z", "content": "Congrats. Great stuff and write-up!
", "rawMarkdown": "Congrats. Great stuff and write-up!", "votes": 1 }, { "id": 727627, "postDate": "2020-01-23T23:06:43.257Z", "content": "@yukia18 i did a world training too, but, not in the same way than you, and i really don't understand why magma had a worst score and didn't train well in any of my models, did you find the same weirdness on your model training with separation by world? and
\n\ngrats for 2nd, i didn't get any medal, because my best score (549) wasn't choosen
", "rawMarkdown": "@yukia18 i did a world training too, but, not in the same way than you, and i really don't understand why magma had a worst score and didn't train well in any of my models, did you find the same weirdness on your model training with separation by world? and \n\ngrats for 2nd, i didn't get any medal, because my best score (549) wasn't choosen", "votes": 1, "replies": [ { "id": 727969, "postDate": "2020-01-24T09:18:02.393Z", "content": "we didn't separate model per world, just adding features aggregated per world.
", "rawMarkdown": "we didn't separate model per world, just adding features aggregated per world." }, { "id": 728282, "postDate": "2020-01-24T15:02:32.607Z", "content": "ahh gotcha, thx for the explanation
", "rawMarkdown": "ahh gotcha, thx for the explanation" } ] }, { "id": 727576, "postDate": "2020-01-23T20:47:34.793Z", "content": "Congrats!
", "rawMarkdown": "Congrats!", "votes": 1 }, { "id": 727552, "postDate": "2020-01-23T19:59:22.777Z", "content": "Congrats!! Great work.
", "rawMarkdown": "Congrats!! Great work.", "votes": 1 }, { "id": 727506, "postDate": "2020-01-23T18:52:56.580Z", "content": "Congratulations on your 2nd place! Really liked the word2vec idea to convert a sequence into a vector :)
\n\nI am struggling to understand the meta features.
\n\nIn the visual example that you kindly shared, the user A undertakes twice the Bird assessment, once getting an accuracy of 2 in session G3 and then 3 in session G6. If I understand correctly how you get to your meta features training set, it would mean that there would be twice the row G1 session with one time a MetaBird target score of 2 and one with a score of 3 while all other columns are the same.\nAm I correct?
", "rawMarkdown": "Congratulations on your 2nd place! Really liked the word2vec idea to convert a sequence into a vector :)\n\nI am struggling to understand the meta features. \n\nIn the visual example that you kindly shared, the user A undertakes twice the Bird assessment, once getting an accuracy of 2 in session G3 and then 3 in session G6. If I understand correctly how you get to your meta features training set, it would mean that there would be twice the row G1 session with one time a MetaBird target score of 2 and one with a score of 3 while all other columns are the same.\nAm I correct?", "votes": 1, "replies": [ { "id": 727967, "postDate": "2020-01-24T09:16:40.957Z", "content": "yes, as you said, Meta Bird of G1 and G2 can be filled with 2 from G3 results (see below). But we considered that all sessions have effect to final assessment results, so we decided overwriting Meta Bird of G1 and G2 by 3 from G6 results.\n
Thanks for sharing!
\n\nI wonder if you used adversarial validation in your feature selection as the 1st place team did?
\n\nI tried it myself and easily built a model that separated train from test with accuracy 100%. Using the features from this model is likely to overfit on train.
", "rawMarkdown": "Thanks for sharing!\n\nI wonder if you used adversarial validation in your feature selection as the 1st place team did?\n\nI tried it myself and easily built a model that separated train from test with accuracy 100%. Using the features from this model is likely to overfit on train.", "replies": [ { "id": 817365, "postDate": "2020-04-23T05:26:49.100Z", "content": "I'm sorry that I didn't notice your question.
\n\nWe actually tried adversarial validation on our 300 features. My model could distinguish train and test easily (I forget a score of AUC).
\n\nBut public/private score wasn't decreased so much.
", "rawMarkdown": "I'm sorry that I didn't notice your question.\n\nWe actually tried adversarial validation on our 300 features. My model could distinguish train and test easily (I forget a score of AUC).\n\nBut public/private score wasn't decreased so much.", "votes": 1 } ] }, { "id": 736518, "postDate": "2020-02-04T09:43:27.220Z", "content": "Congrats and thanks for sharing!
", "rawMarkdown": "Congrats and thanks for sharing!\n\n", "votes": 1 }, { "id": 732562, "postDate": "2020-01-30T00:01:13.417Z", "content": "Congrats and thanks for sharing!
", "rawMarkdown": "Congrats and thanks for sharing!", "votes": 1 }, { "id": 728584, "postDate": "2020-01-24T23:42:12.333Z", "content": "Thanks for the solution! Congrats!
", "rawMarkdown": "Thanks for the solution! Congrats!", "votes": 1 }, { "id": 727895, "postDate": "2020-01-24T06:59:43.707Z", "content": "Thank you for sharing
", "rawMarkdown": "Thank you for sharing\n", "votes": 1 }, { "id": 727767, "postDate": "2020-01-24T03:05:45.610Z", "content": "Thank you for sharing!
", "rawMarkdown": "Thank you for sharing!", "votes": 1 } ], "complete": true, "source": "live", "fetched_at": "2026-10-23T06:53:24Z", "competition_id": 16531, "competition_slug": "data-science-bowl-2019", "team_id": 3940948, "solution_url": "/competitions/data-science-bowl-2019/writeups/fuson-2nd-place-solution" }