Hi All,
\n\nCongratulations to winners, of course!
\n\nIn my opinion, it was great competition especially because it required a lot of feature engineering. I've been concentrating mostly on Prudential, but eventually found this competition much more interesting. It's a pity I spent only the last four days on this one.
\n\nHere's my solution:
\n\nI used only user and session data (I dropped user ids which were not represented in session data from training dataset). Eventually, I generated 3 datasets based on session info: one containing only binary features, one containing seconds counts, and one containing seconds shares by actions, action types, action details, and devices.
\n\na) Averaged predictions of 6 NN models (keras). One layer and two layer feed-forward NN trained on each dataset.
\n\nb) Averaged predictions of XGBoost and Random Forest trained on each dataset.
\n\nc) XGBoost on stacking data (KNN+XGB+RF+ET+NN+LogisticReg+SGD). Some stacking models were trained on all three datasets, some only on one.
\n\nd) Logistic Regression on the same stacking data
\n\nI then averaged predictions of these four models to get the final prediction matrix.
\n\nTo summarize, I spent two days (nights) on data wrangling and preparation and two days on modeling. Damn, I wish I entered earlier!
\n\nP.S. I will upload my solution on GitHub soon.
", "comments": [ { "id": 107819, "author_name": "Anton Laptiev", "author_url": "", "post_date": "2016-02-12T20:40:42.227000", "content": "Hey,
\n\nFor stacking I split data into 3 folds (stratified). XGBoost validation ratio 0.05.\nI actually planned to add RF on stacking data too because it has CV score similar to that of XGBoost, but sklearn did not finish it on time for me:) So, imagine my frustration this morning when, just out of curiosity, I submitted the model with RF added and got 8th position on the LB (0.88652) :( I have to buy a more powerful computer (or start competitions earlier), I guess.
", "votes": 0, "replies": [] }, { "id": 107771, "author_name": "RG", "author_url": "", "post_date": "2016-02-12T13:46:41.533000", "content": "[quote=Anton Laptiev;107694]
\n\nHi All,
\n\nCongratulations to winners, of course!
\n\nIn my opinion, it was great competition especially because it required a lot of feature engineering. I've been concentrating mostly on Prudential, but eventually found this competition much more interesting. It's a pity I spent only the last four days on this one.
\n\nHere's my solution:
\n\nI used only user and session data (I dropped user ids which were not represented in session data from training dataset). Eventually, I generated 3 datasets based on session info: one containing only binary features, one containing seconds counts, and one containing seconds shares by actions, action types, action details, and devices.
\n\na) Averaged predictions of 6 NN models (keras). One layer and two layer feed-forward NN trained on each dataset.
\n\nb) Averaged predictions of XGBoost and Random Forest trained on each dataset.
\n\nc) XGBoost on stacking data (KNN+XGB+RF+ET+NN+LogisticReg+SGD). Some stacking models were trained on all three datasets, some only on one.
\n\nd) Logistic Regression on the same stacking data
\n\nI then averaged predictions of these four models to get the final prediction matrix.
\n\nTo summarize, I spent two days (nights) on data wrangling and preparation and two days on modeling. Damn, I wish I entered earlier!
\n\nP.S. I will upload my solution on GitHub soon.
\n\n[/quote]\nAnton - Could you share more details on data split on stacking models? Specifically
\n\nThank you for sharing your answer. I tried to make some new features like 5-gram action type and 2-gram action, but I didn't get an more improvement.
\n\nI am looking forward to see other's solutions.
", "votes": 0, "replies": [] } ], "comment_count": 3, "messages": [ { "id": 107694, "postDate": "2016-02-12T01:02:19.483Z", "content": "Hi All,
\n\nCongratulations to winners, of course!
\n\nIn my opinion, it was great competition especially because it required a lot of feature engineering. I've been concentrating mostly on Prudential, but eventually found this competition much more interesting. It's a pity I spent only the last four days on this one.
\n\nHere's my solution:
\n\nI used only user and session data (I dropped user ids which were not represented in session data from training dataset). Eventually, I generated 3 datasets based on session info: one containing only binary features, one containing seconds counts, and one containing seconds shares by actions, action types, action details, and devices.
\n\na) Averaged predictions of 6 NN models (keras). One layer and two layer feed-forward NN trained on each dataset.
\n\nb) Averaged predictions of XGBoost and Random Forest trained on each dataset.
\n\nc) XGBoost on stacking data (KNN+XGB+RF+ET+NN+LogisticReg+SGD). Some stacking models were trained on all three datasets, some only on one.
\n\nd) Logistic Regression on the same stacking data
\n\nI then averaged predictions of these four models to get the final prediction matrix.
\n\nTo summarize, I spent two days (nights) on data wrangling and preparation and two days on modeling. Damn, I wish I entered earlier!
\n\nP.S. I will upload my solution on GitHub soon.
", "rawMarkdown": "Hi All,\r\n\r\nCongratulations to winners, of course! \r\n\r\nIn my opinion, it was great competition especially because it required a lot of feature engineering. I've been concentrating mostly on Prudential, but eventually found this competition much more interesting. It's a pity I spent only the last four days on this one.\r\n\r\nHere's my solution:\r\n\r\n1. Data\r\n\r\nI used only user and session data (I dropped user ids which were not represented in session data from training dataset). Eventually, I generated 3 datasets based on session info: one containing only binary features, one containing seconds counts, and one containing seconds shares by actions, action types, action details, and devices. \r\n\r\n2. Models\r\n\r\na) Averaged predictions of 6 NN models (keras). One layer and two layer feed-forward NN trained on each dataset.\r\n\r\nb) Averaged predictions of XGBoost and Random Forest trained on each dataset.\r\n\r\nc) XGBoost on stacking data (KNN+XGB+RF+ET+NN+LogisticReg+SGD). Some stacking models were trained on all three datasets, some only on one.\r\n\r\nd) Logistic Regression on the same stacking data\r\n\r\nI then averaged predictions of these four models to get the final prediction matrix.\r\n\r\nTo summarize, I spent two days (nights) on data wrangling and preparation and two days on modeling. Damn, I wish I entered earlier!\r\n\r\nP.S. I will upload my solution on GitHub soon.", "votes": 7 }, { "id": 107819, "postDate": "2016-02-12T20:40:42.227Z", "content": "Hey,
\n\nFor stacking I split data into 3 folds (stratified). XGBoost validation ratio 0.05.\nI actually planned to add RF on stacking data too because it has CV score similar to that of XGBoost, but sklearn did not finish it on time for me:) So, imagine my frustration this morning when, just out of curiosity, I submitted the model with RF added and got 8th position on the LB (0.88652) :( I have to buy a more powerful computer (or start competitions earlier), I guess.
", "rawMarkdown": "Hey,\r\n\r\nFor stacking I split data into 3 folds (stratified). XGBoost validation ratio 0.05.\r\nI actually planned to add RF on stacking data too because it has CV score similar to that of XGBoost, but sklearn did not finish it on time for me:) So, imagine my frustration this morning when, just out of curiosity, I submitted the model with RF added and got 8th position on the LB (0.88652) :( I have to buy a more powerful computer (or start competitions earlier), I guess.\r\n\r\n" }, { "id": 107771, "postDate": "2016-02-12T13:46:41.533Z", "content": "[quote=Anton Laptiev;107694]
\n\nHi All,
\n\nCongratulations to winners, of course!
\n\nIn my opinion, it was great competition especially because it required a lot of feature engineering. I've been concentrating mostly on Prudential, but eventually found this competition much more interesting. It's a pity I spent only the last four days on this one.
\n\nHere's my solution:
\n\nI used only user and session data (I dropped user ids which were not represented in session data from training dataset). Eventually, I generated 3 datasets based on session info: one containing only binary features, one containing seconds counts, and one containing seconds shares by actions, action types, action details, and devices.
\n\na) Averaged predictions of 6 NN models (keras). One layer and two layer feed-forward NN trained on each dataset.
\n\nb) Averaged predictions of XGBoost and Random Forest trained on each dataset.
\n\nc) XGBoost on stacking data (KNN+XGB+RF+ET+NN+LogisticReg+SGD). Some stacking models were trained on all three datasets, some only on one.
\n\nd) Logistic Regression on the same stacking data
\n\nI then averaged predictions of these four models to get the final prediction matrix.
\n\nTo summarize, I spent two days (nights) on data wrangling and preparation and two days on modeling. Damn, I wish I entered earlier!
\n\nP.S. I will upload my solution on GitHub soon.
\n\n[/quote]\nAnton - Could you share more details on data split on stacking models? Specifically
\n\nThank you for sharing your answer. I tried to make some new features like 5-gram action type and 2-gram action, but I didn't get an more improvement.
\n\nI am looking forward to see other's solutions.
", "rawMarkdown": "Thank you for sharing your answer. I tried to make some new features like 5-gram action type and 2-gram action, but I didn't get an more improvement.\r\n\r\nI am looking forward to see other's solutions." } ], "complete": true, "source": "live", "fetched_at": "2026-10-19T13:52:54Z", "competition_id": 4651, "competition_slug": "airbnb-recruiting-new-user-bookings", "team_id": 263036, "solution_url": "/competitions/airbnb-recruiting-new-user-bookings/writeups/anton-laptiev-my-solution" }