{ "id": 18976, "title": "2nd Place Solution", "content": "

Thanks to organizers and kagglers.\nHere is a short summary of my approach and source code.\nI hope this will help.

\n\n

[short summary]\nkaggle-airbnb-2nd-place-solution.png

\n\n

[github]\nKeiku/kaggle-airbnb-recruiting-new-user-bookings: 2nd Place Solution in Kaggle Airbnb New User Bookings competition https://github.com/Keiku/kaggle-airbnb-recruiting-new-user-bookings

", "comments": [ { "id": 108301, "author_name": "Keiku", "author_url": "", "post_date": "2016-02-16T16:32:01.040000", "content": "

Sorry for not explaining enough. (and I'm sorry for my poor English.)

\n\n\n\n

I created age_cln and age_cln2 as numerical features of 1st and 2nd layer model, and I created age_bucket in order to join age_gender_bkts dataset to train_users and test_users.

\n\n\n\n

Feature Hashing by R code: \nX_sp <- sparseMatrix(i = X$id_num, j = X$feature, x = X$value)

\n\n\n\n

I created models in the following steps.

\n\n

There is a little mistake. \nwrong : 19 models / correct: 18 models

\n\n
    \n
  1. I calculated the out of bag prediction of 18 features(stacked generalization).
    2. \n
  2. I selected randomly 1240 features (=(1312+66)*90/100) from base features(1312) and out of bag prediction(66 = number of categories of 18 features).
    3. \n
  3. I created repeatedly single XGBoost by 1240 features(2-3 is run 90 times).
    4. \n
  4. I selected Best XGBoost model(5 fold-CV 0.833714) from 90 models.
    5. \n
  5. I finally got Public:0.88209/Private:0.88682 using Best XGBoost(5 fold-CV 0.833714).
    6. \n
\n\n

I think that it is difficult to get more than 0.88100(Public) in single XGBoost. You need to use stacked generalization.

\n\n

The following can help you.

\n\n", "votes": 9, "replies": [] }, { "id": 243391, "author_name": "ClaireChiang", "author_url": "", "post_date": "2017-11-13T23:25:20.913000", "content": "

Dear Mr. Keiichi, \nThank you for sharing the insight with us. I'm very new to data science. One simple question here. For the below figure, why two GLM are in the categories of Age features and lag features? Does it mean you use GLM at these two categories for modeling? Yous answer is much appreciated. \nhttps://raw.githubusercontent.com/Keiku/kaggle-airbnb-recruiting-new-user-bookings/master/Learning_Architecture.png

", "votes": 1, "replies": [ { "id": 243962, "author_name": "Keiku", "author_url": "", "post_date": "2017-11-15T08:23:15.537000", "content": "

Dear ClaireChiang,

\n\n

Sorry for my late reply. I created five models (three XGBoosts, two GLMNet) that estimate the missing part of age, with the fulfilling part of age as training data. More specifically, XGBoost predicts the the raw age feature and the age features of the two cleaning patterns. On the other hand, GLMNet predicts only age features of two types of cleaning patterns. As a result, I have created the models that imputes the missing values of five kinds of age in total. For details, you may find it in the source code.

\n\n

The following source code X3, X8, X9, X16, X17 parts are predicting age.

\n\n\n\n

https://github.com/Keiku/kaggle-airbnb-recruiting-new-user-bookings/blob/master/script/stacking.R

", "votes": 0, "replies": [] }, { "id": 246309, "author_name": "ClaireChiang", "author_url": "", "post_date": "2017-11-20T23:34:59.430000", "content": "

Mr. Keiichi-san, Thank you!

", "votes": 0, "replies": [] } ] }, { "id": 112106, "author_name": "Keiku", "author_url": "", "post_date": "2016-03-18T05:55:46.497000", "content": "

My approach blog post was published in no free hunch of Kaggle. The details are as below.

\n\n

Airbnb New User Bookings, Winner’s Interview: 2nd place, Keiichi Kuroyanagi (@Keiku) | no free hunch http://blog.kaggle.com/2016/03/17/airbnb-new-user-bookings-winners-interview-2nd-place-keiichi-kuroyanagi-keiku/

", "votes": 3, "replies": [] }, { "id": 2326903, "author_name": "Kenny", "author_url": "", "post_date": "2023-07-02T13:48:09.080000", "content": "

Great thoughts on those features you created. I'm just curious about how important do the various combinations of year, month and day values perform as features in your model? To me it seems refreshing idea to apply time related features in this way.

", "votes": 0, "replies": [] }, { "id": 263786, "author_name": "Allen", "author_url": "", "post_date": "2017-12-31T22:03:46.397000", "content": "

Hi Keiku, hope you are well. Thank you for posting your solution. I was wondering if you have the submission file in csv format handy? I just completed this challenge on my own and wanted to compare my answers with what you had. Much appreciated if you could provide the csv file as well. Thank you.

", "votes": 0, "replies": [ { "id": 263803, "author_name": "Keiku", "author_url": "", "post_date": "2018-01-01T03:03:02.253000", "content": "

@Allen

\n\n

Kaggle's system has download function of submission file once submitted, so I have submission csv files. I can share the submission csv file, but you can submit to the leaderboard even in the competition that ended, so please try using leaderboard to verify. Even when challenging other competitions, you can not see the answers of other challengers. I think that it is better to keep a habit of verifying with only Public's LB.

", "votes": 1, "replies": [] }, { "id": 263974, "author_name": "Allen", "author_url": "", "post_date": "2018-01-01T21:16:18.200000", "content": "

Thanks for your response Keiku. I did not realize I could still submit after the competition ended. I will do that. Happy New Year from the US!

", "votes": 0, "replies": [] } ] }, { "id": 165788, "author_name": "Keiku", "author_url": "", "post_date": "2017-03-07T03:45:50.293000", "content": "

@lotusky Thank you for having interest in my solution. The github link of my solution is as follows.

\n\n

Keiku/kaggle-airbnb-recruiting-new-user-bookings: 2nd Place Solution in Kaggle Airbnb New User Bookings competition\nhttps://github.com/Keiku/kaggle-airbnb-recruiting-new-user-bookings

", "votes": 0, "replies": [] }, { "id": 165787, "author_name": "lotusky", "author_url": "", "post_date": "2017-03-07T03:36:57.920000", "content": "

Hey Keiku, Thanks for the sharing. I read your interview in the kaggle, want to learn how you did it. I can't find the code you mentioned in the blog, can you share the link of the code?

", "votes": 0, "replies": [] }, { "id": 120630, "author_name": "Keiku", "author_url": "", "post_date": "2016-05-19T16:38:04.110000", "content": "

@AminaKeldibek

\n\n

Sorry for my late reply. These lag features are predicted by stacked generalization(Test dataset don't have the lag features). I divided into "rough" category in order to make it easier to predict. Although I was able to predict the lag value by regression without dividing, I didn't try it.

", "votes": 0, "replies": [] }, { "id": 120565, "author_name": "Sunshine", "author_url": "", "post_date": "2016-05-19T03:55:24.577000", "content": "

Dear @Keiku!

\n\n

Thanks for your explanation! I have one question regarding feature extraction. In you interview you stated that you calculated the lag of "date_first_booking" and "date_account_created" and divided this lag feature into four categories (0, [1, 365], [-349,0), NA). Could you please explain why you decided you choose these particular categories and the same about lag of "date_first_booking" and "timestamp_first_active".

\n\n

Thank you a lot!

", "votes": 0, "replies": [] }, { "id": 108304, "author_name": "David Gasquez", "author_url": "", "post_date": "2016-02-16T16:42:46.887000", "content": "

Thanks @Keiku, I'll take a look over the links!

", "votes": 0, "replies": [] }, { "id": 108267, "author_name": "David Gasquez", "author_url": "", "post_date": "2016-02-16T14:37:57.553000", "content": "

First, congratulations! I'm trying to understand your workflow, and if some questions popped out during this process:

\n\n\n\n

Later,

\n\n\n\n

It was my first competition and I did not try with ensembling, my single XGBoost gave me 0.88090 in the public leaderboard and 0.88590 in the private. I think this can vastly improve the predictions, again, thanks for sharing!

", "votes": 0, "replies": [] } ], "comment_count": 15, "messages": [ { "id": 108084, "postDate": "2016-02-15T13:05:06.870Z", "content": "

Thanks to organizers and kagglers.\nHere is a short summary of my approach and source code.\nI hope this will help.

\n\n

[short summary]\nkaggle-airbnb-2nd-place-solution.png

\n\n

[github]\nKeiku/kaggle-airbnb-recruiting-new-user-bookings: 2nd Place Solution in Kaggle Airbnb New User Bookings competition https://github.com/Keiku/kaggle-airbnb-recruiting-new-user-bookings

", "rawMarkdown": "Thanks to organizers and kagglers.\r\nHere is a short summary of my approach and source code.\r\nI hope this will help.\r\n\r\n[short summary]\r\nkaggle-airbnb-2nd-place-solution.png\r\n\r\n[github]\r\nKeiku/kaggle-airbnb-recruiting-new-user-bookings: 2nd Place Solution in Kaggle Airbnb New User Bookings competition https://github.com/Keiku/kaggle-airbnb-recruiting-new-user-bookings", "votes": 29 }, { "id": 108301, "postDate": "2016-02-16T16:32:01.040Z", "content": "

Sorry for not explaining enough. (and I'm sorry for my poor English.)

\n\n\n\n

I created age_cln and age_cln2 as numerical features of 1st and 2nd layer model, and I created age_bucket in order to join age_gender_bkts dataset to train_users and test_users.

\n\n\n\n

Feature Hashing by R code: \nX_sp <- sparseMatrix(i = X$id_num, j = X$feature, x = X$value)

\n\n\n\n

I created models in the following steps.

\n\n

There is a little mistake. \nwrong : 19 models / correct: 18 models

\n\n
    \n
  1. I calculated the out of bag prediction of 18 features(stacked generalization).
    2. \n
  2. I selected randomly 1240 features (=(1312+66)*90/100) from base features(1312) and out of bag prediction(66 = number of categories of 18 features).
    3. \n
  3. I created repeatedly single XGBoost by 1240 features(2-3 is run 90 times).
    4. \n
  4. I selected Best XGBoost model(5 fold-CV 0.833714) from 90 models.
    5. \n
  5. I finally got Public:0.88209/Private:0.88682 using Best XGBoost(5 fold-CV 0.833714).
    6. \n
\n\n

I think that it is difficult to get more than 0.88100(Public) in single XGBoost. You need to use stacked generalization.

\n\n

The following can help you.

\n\n", "rawMarkdown": "Sorry for not explaining enough. (and I'm sorry for my poor English.)\r\n\r\n - About age features.\r\n\r\nI created age_cln and age_cln2 as numerical features of 1st and 2nd layer model, and I created age_bucket in order to join age_gender_bkts dataset to train_users and test_users.\r\n\r\n - About extracting from sessions.csv.\r\n 1. I summarized secs_elapsed and flg(1) by user_id, action.(similarly, action_type, action_detail...)\r\n 2. Datasets of the above are binded into one dataset(df_all_feats) and it is hashing in a lump using the below code.\r\n\r\nFeature Hashing by R code: \r\nX_sp <- sparseMatrix(i = X$id_num, j = X$feature, x = X$value)\r\n\r\n - About my prediction.\r\n\r\nI created models in the following steps.\r\n\r\nThere is a little mistake. \r\nwrong : 19 models / correct: 18 models\r\n\r\n 1. I calculated the out of bag prediction of 18 features(stacked generalization).\r\n 2. I selected randomly 1240 features (=(1312+66)*90/100) from base features(1312) and out of bag prediction(66 = number of categories of 18 features).\r\n 3. I created repeatedly single XGBoost by 1240 features(2-3 is run 90 times).\r\n 4. I selected Best XGBoost model(5 fold-CV 0.833714) from 90 models.\r\n 5. I finally got Public:0.88209/Private:0.88682 using Best XGBoost(5 fold-CV 0.833714).\r\n\r\nI think that it is difficult to get more than 0.88100(Public) in single XGBoost. You need to use stacked generalization.\r\n\r\nThe following can help you.\r\n\r\n - Kaggle Ensembling Guide | MLWave http://mlwave.com/kaggle-ensembling-guide/\r\n - 1st PLACE - WINNER SOLUTION - Gilberto Titericz & Stanislav Semenov - Otto Group Product Classification Challenge | Kaggle https://www.kaggle.com/c/otto-group-product-classification-challenge/forums/t/14335/1st-place-winner-solution-gilberto-titericz-stanislav-semenov?limit=all", "votes": 9 }, { "id": 243391, "postDate": "2017-11-13T23:25:20.913Z", "content": "

Dear Mr. Keiichi, \nThank you for sharing the insight with us. I'm very new to data science. One simple question here. For the below figure, why two GLM are in the categories of Age features and lag features? Does it mean you use GLM at these two categories for modeling? Yous answer is much appreciated. \nhttps://raw.githubusercontent.com/Keiku/kaggle-airbnb-recruiting-new-user-bookings/master/Learning_Architecture.png

", "rawMarkdown": "Dear Mr. Keiichi, \nThank you for sharing the insight with us. I'm very new to data science. One simple question here. For the below figure, why two GLM are in the categories of Age features and lag features? Does it mean you use GLM at these two categories for modeling? Yous answer is much appreciated. \nhttps://raw.githubusercontent.com/Keiku/kaggle-airbnb-recruiting-new-user-bookings/master/Learning_Architecture.png ", "votes": 1, "replies": [ { "id": 243962, "postDate": "2017-11-15T08:23:15.537Z", "content": "

Dear ClaireChiang,

\n\n

Sorry for my late reply. I created five models (three XGBoosts, two GLMNet) that estimate the missing part of age, with the fulfilling part of age as training data. More specifically, XGBoost predicts the the raw age feature and the age features of the two cleaning patterns. On the other hand, GLMNet predicts only age features of two types of cleaning patterns. As a result, I have created the models that imputes the missing values of five kinds of age in total. For details, you may find it in the source code.

\n\n

The following source code X3, X8, X9, X16, X17 parts are predicting age.

\n\n\n\n

https://github.com/Keiku/kaggle-airbnb-recruiting-new-user-bookings/blob/master/script/stacking.R

", "rawMarkdown": "Dear ClaireChiang,\n\nSorry for my late reply. I created five models (three XGBoosts, two GLMNet) that estimate the missing part of age, with the fulfilling part of age as training data. More specifically, XGBoost predicts the the raw age feature and the age features of the two cleaning patterns. On the other hand, GLMNet predicts only age features of two types of cleaning patterns. As a result, I have created the models that imputes the missing values of five kinds of age in total. For details, you may find it in the source code.\n\nThe following source code X3, X8, X9, X16, X17 parts are predicting age.\n\n* Age cleaning pattern 1: age_cln = ifelse(age >= 1920, 2015 - age, age)\n* Age cleaning pattern 2: age_cln2 = ifelse(age_cln < 14 | age_cln > 100, -1, age_cln)\n\nhttps://github.com/Keiku/kaggle-airbnb-recruiting-new-user-bookings/blob/master/script/stacking.R" }, { "id": 246309, "postDate": "2017-11-20T23:34:59.430Z", "content": "

Mr. Keiichi-san, Thank you!

", "rawMarkdown": "Mr. Keiichi-san, Thank you! " } ] }, { "id": 112106, "postDate": "2016-03-18T05:55:46.497Z", "content": "

My approach blog post was published in no free hunch of Kaggle. The details are as below.

\n\n

Airbnb New User Bookings, Winner’s Interview: 2nd place, Keiichi Kuroyanagi (@Keiku) | no free hunch http://blog.kaggle.com/2016/03/17/airbnb-new-user-bookings-winners-interview-2nd-place-keiichi-kuroyanagi-keiku/

", "rawMarkdown": "My approach blog post was published in no free hunch of Kaggle. The details are as below.\r\n\r\nAirbnb New User Bookings, Winner’s Interview: 2nd place, Keiichi Kuroyanagi (@Keiku) | no free hunch http://blog.kaggle.com/2016/03/17/airbnb-new-user-bookings-winners-interview-2nd-place-keiichi-kuroyanagi-keiku/", "votes": 3 }, { "id": 2326903, "postDate": "2023-07-02T13:48:09.080Z", "content": "

Great thoughts on those features you created. I'm just curious about how important do the various combinations of year, month and day values perform as features in your model? To me it seems refreshing idea to apply time related features in this way.

", "rawMarkdown": "Great thoughts on those features you created. I'm just curious about how important do the various combinations of year, month and day values perform as features in your model? To me it seems refreshing idea to apply time related features in this way." }, { "id": 263786, "postDate": "2017-12-31T22:03:46.397Z", "content": "

Hi Keiku, hope you are well. Thank you for posting your solution. I was wondering if you have the submission file in csv format handy? I just completed this challenge on my own and wanted to compare my answers with what you had. Much appreciated if you could provide the csv file as well. Thank you.

", "rawMarkdown": "Hi Keiku, hope you are well. Thank you for posting your solution. I was wondering if you have the submission file in csv format handy? I just completed this challenge on my own and wanted to compare my answers with what you had. Much appreciated if you could provide the csv file as well. Thank you. ", "replies": [ { "id": 263803, "postDate": "2018-01-01T03:03:02.253Z", "content": "

@Allen

\n\n

Kaggle's system has download function of submission file once submitted, so I have submission csv files. I can share the submission csv file, but you can submit to the leaderboard even in the competition that ended, so please try using leaderboard to verify. Even when challenging other competitions, you can not see the answers of other challengers. I think that it is better to keep a habit of verifying with only Public's LB.

", "rawMarkdown": "@Allen\n\nKaggle's system has download function of submission file once submitted, so I have submission csv files. I can share the submission csv file, but you can submit to the leaderboard even in the competition that ended, so please try using leaderboard to verify. Even when challenging other competitions, you can not see the answers of other challengers. I think that it is better to keep a habit of verifying with only Public's LB.", "votes": 1 }, { "id": 263974, "postDate": "2018-01-01T21:16:18.200Z", "content": "

Thanks for your response Keiku. I did not realize I could still submit after the competition ended. I will do that. Happy New Year from the US!

", "rawMarkdown": "Thanks for your response Keiku. I did not realize I could still submit after the competition ended. I will do that. Happy New Year from the US!" } ] }, { "id": 165788, "postDate": "2017-03-07T03:45:50.293Z", "content": "

@lotusky Thank you for having interest in my solution. The github link of my solution is as follows.

\n\n

Keiku/kaggle-airbnb-recruiting-new-user-bookings: 2nd Place Solution in Kaggle Airbnb New User Bookings competition\nhttps://github.com/Keiku/kaggle-airbnb-recruiting-new-user-bookings

", "rawMarkdown": "@lotusky Thank you for having interest in my solution. The github link of my solution is as follows. \n\nKeiku/kaggle-airbnb-recruiting-new-user-bookings: 2nd Place Solution in Kaggle Airbnb New User Bookings competition\nhttps://github.com/Keiku/kaggle-airbnb-recruiting-new-user-bookings" }, { "id": 165787, "postDate": "2017-03-07T03:36:57.920Z", "content": "

Hey Keiku, Thanks for the sharing. I read your interview in the kaggle, want to learn how you did it. I can't find the code you mentioned in the blog, can you share the link of the code?

", "rawMarkdown": "Hey Keiku, Thanks for the sharing. I read your interview in the kaggle, want to learn how you did it. I can't find the code you mentioned in the blog, can you share the link of the code?" }, { "id": 120630, "postDate": "2016-05-19T16:38:04.110Z", "content": "

@AminaKeldibek

\n\n

Sorry for my late reply. These lag features are predicted by stacked generalization(Test dataset don't have the lag features). I divided into "rough" category in order to make it easier to predict. Although I was able to predict the lag value by regression without dividing, I didn't try it.

", "rawMarkdown": "@AminaKeldibek\r\n\r\nSorry for my late reply. These lag features are predicted by stacked generalization(Test dataset don't have the lag features). I divided into \"rough\" category in order to make it easier to predict. Although I was able to predict the lag value by regression without dividing, I didn't try it." }, { "id": 120565, "postDate": "2016-05-19T03:55:24.577Z", "content": "

Dear @Keiku!

\n\n

Thanks for your explanation! I have one question regarding feature extraction. In you interview you stated that you calculated the lag of "date_first_booking" and "date_account_created" and divided this lag feature into four categories (0, [1, 365], [-349,0), NA). Could you please explain why you decided you choose these particular categories and the same about lag of "date_first_booking" and "timestamp_first_active".

\n\n

Thank you a lot!

", "rawMarkdown": "Dear @Keiku!\r\n\r\nThanks for your explanation! I have one question regarding feature extraction. In you interview you stated that you calculated the lag of \"date_first_booking\" and \"date_account_created\" and divided this lag feature into four categories (0, [1, 365], [-349,0), NA). Could you please explain why you decided you choose these particular categories and the same about lag of \"date_first_booking\" and \"timestamp_first_active\".\r\n\r\nThank you a lot!" }, { "id": 108304, "postDate": "2016-02-16T16:42:46.887Z", "content": "

Thanks @Keiku, I'll take a look over the links!

", "rawMarkdown": "Thanks @Keiku, I'll take a look over the links!\r\n\r\n" }, { "id": 108267, "postDate": "2016-02-16T14:37:57.553Z", "content": "

First, congratulations! I'm trying to understand your workflow, and if some questions popped out during this process:

\n\n\n\n

Later,

\n\n\n\n

It was my first competition and I did not try with ensembling, my single XGBoost gave me 0.88090 in the public leaderboard and 0.88590 in the private. I think this can vastly improve the predictions, again, thanks for sharing!

", "rawMarkdown": "First, congratulations! I'm trying to understand your workflow, and if some questions popped out during this process:\r\n\r\n - Generate 1312 features with `preprocessing.R`\r\n\r\n- You create up to 3 columns with date related with age. Do you use the 3 or are only temporary?\r\n- What features are you extracting from `sessions.csv`? I was reading the code but I'm don't know R as much as I want.\r\n\r\nLater,\r\n\r\n- Predict 19 variables as it can be age, country...\r\n- Fit a new XGBoost model using 90% of features from the 19 variables model before. What does it mean stacked generalization?\r\n- Predict country destination with this latest model(trained with the 90% of the output of the 19 models)\r\n\r\n\r\nIt was my first competition and I did not try with ensembling, my single XGBoost gave me 0.88090 in the public leaderboard and 0.88590 in the private. I think this can vastly improve the predictions, again, thanks for sharing!\r\n\r\n\r\n\r\n" } ], "complete": true, "source": "live", "fetched_at": "2026-10-19T13:53:27Z", "competition_id": 4651, "competition_slug": "airbnb-recruiting-new-user-bookings", "team_id": 267288, "solution_url": "/competitions/airbnb-recruiting-new-user-bookings/writeups/keiku-2nd-place-solution" }