{
  "id": 18973,
  "title": "Solution Presentation (#54)",
  "content": "<p>Thanks to organizers and Kagglers for this interesting challenge and its sharings in the forums, and congratulations to the top 10 !</p>\n\n<p>FWIW, here is a presentation of my approach.</p>\n\n<p>Cheers !</p>",
  "comments": [
    {
      "id": 110028,
      "author_name": "Paul",
      "author_url": "",
      "post_date": "2016-03-02T07:38:50.770000",
      "content": "<p>Hi Randombishop, thanks for sharing. Your slide is beautiful and well-organized.</p>\n\n<p>I have some questions, in page 8 you described:</p>\n\n<blockquote>\n  <p>After reaching single models ceiling, I tried submitting 5 different\n  variations of XGBoost models per day and built a majority vote\n  ensemble based on the three most different amongst the 50 best ones.</p>\n</blockquote>\n\n<ol>\n<li>How did you define 'most different'? All of them are XGBoost models, right? </li>\n</ol>\n\n<p>in page 9,</p>\n\n<blockquote>\n  <p>Majority vote ensemble of three most different models produces\n  interesting results, but was not enough to make significant\n  improvements.</p>\n</blockquote>\n\n<ol start=\"2\">\n<li>Is this another ensemble (mixed both best XGBoost and Random Forest models), which is different with page 8? </li>\n</ol>\n\n<p>Thank you.</p>",
      "votes": 1,
      "replies": []
    },
    {
      "id": 108150,
      "author_name": "Randombishop",
      "author_url": "",
      "post_date": "2016-02-15T20:59:15.653000",
      "content": "<p>Here are my R scripts for modeling.</p>\n\n<p>For feature engineering, I used Java and a MySql database, and the code is quite messy, so not very shareable :)</p>",
      "votes": 1,
      "replies": []
    },
    {
      "id": 110031,
      "author_name": "Randombishop",
      "author_url": "",
      "post_date": "2016-03-02T08:59:22.587000",
      "content": "<p>Thanks for your interest Paul, nice question !</p>\n\n<ol>\n<li>Models were all XGBoost indeed, and I defined &quot;most different&quot; as follows :</li>\n</ol>\n\n<p>a. Difference between two predictions for the same example is 0.4 , 0.2 , 0.1 , 0.05 , 0.025 or 0 depending on the level on which they disagree. (If all 5 predictions are the same, diff=0, if they disagree on the last prediction, small difference, then gets bigger if they disagree on the first ranks.)</p>\n\n<p>b. Difference between two submission files is the average difference between their predictions on each example.</p>\n\n<p>c. Difference between three models is the average difference between the three possible pairs.</p>\n\n<p>d. The three most different models are the ones that maximimze difference accross all possible combinations of three files in a list selected submissions.</p>\n\n<ol start=\"2\">\n<li>I am mentioning here the same ensembles of three most different XGBoosts, as I generated  many of them with different list of submissions. These ensembles score generally better than their components both on public and private leaderboard, this is why I wrote &quot;interesting results&quot;, but still, the best private leaderboard model was just a single model, this is why I wrote that the approach was not enough to make significant improvements.</li>\n</ol>\n\n<p>In case this is not clear and you prefer code reading, here is the code for ensembling submission files. </p>\n\n<p>Cheers !</p>",
      "votes": 2,
      "replies": []
    },
    {
      "id": 108137,
      "author_name": "",
      "author_url": "",
      "post_date": "2016-02-15T19:34:27.023000",
      "content": "",
      "votes": 0,
      "replies": []
    },
    {
      "id": 108106,
      "author_name": "",
      "author_url": "",
      "post_date": "2016-02-15T15:52:57.277000",
      "content": "",
      "votes": 0,
      "replies": []
    },
    {
      "id": 110308,
      "author_name": "Paul",
      "author_url": "",
      "post_date": "2016-03-04T14:43:08.153000",
      "content": "<p>I got it !Thanks for your explanation.</p>",
      "votes": 0,
      "replies": []
    }
  ],
  "comment_count": 6,
  "messages": [
    {
      "id": 108073,
      "postDate": "2016-02-15T10:22:28.887Z",
      "content": "<p>Thanks to organizers and Kagglers for this interesting challenge and its sharings in the forums, and congratulations to the top 10 !</p>\n\n<p>FWIW, here is a presentation of my approach.</p>\n\n<p>Cheers !</p>",
      "rawMarkdown": "Thanks to organizers and Kagglers for this interesting challenge and its sharings in the forums, and congratulations to the top 10 !\r\n\r\nFWIW, here is a presentation of my approach.\r\n\r\nCheers !",
      "votes": 20
    },
    {
      "id": 110028,
      "postDate": "2016-03-02T07:38:50.770Z",
      "content": "<p>Hi Randombishop, thanks for sharing. Your slide is beautiful and well-organized.</p>\n\n<p>I have some questions, in page 8 you described:</p>\n\n<blockquote>\n  <p>After reaching single models ceiling, I tried submitting 5 different\n  variations of XGBoost models per day and built a majority vote\n  ensemble based on the three most different amongst the 50 best ones.</p>\n</blockquote>\n\n<ol>\n<li>How did you define 'most different'? All of them are XGBoost models, right? </li>\n</ol>\n\n<p>in page 9,</p>\n\n<blockquote>\n  <p>Majority vote ensemble of three most different models produces\n  interesting results, but was not enough to make significant\n  improvements.</p>\n</blockquote>\n\n<ol start=\"2\">\n<li>Is this another ensemble (mixed both best XGBoost and Random Forest models), which is different with page 8? </li>\n</ol>\n\n<p>Thank you.</p>",
      "rawMarkdown": "Hi Randombishop, thanks for sharing. Your slide is beautiful and well-organized.\r\n\r\nI have some questions, in page 8 you described:\r\n\r\n> After reaching single models ceiling, I tried submitting 5 different\r\n> variations of XGBoost models per day and built a majority vote\r\n> ensemble based on the three most different amongst the 50 best ones.\r\n\r\n 1. How did you define 'most different'? All of them are XGBoost models, right? \r\n\r\nin page 9,\r\n\r\n> Majority vote ensemble of three most different models produces\r\n> interesting results, but was not enough to make significant\r\n> improvements.\r\n\r\n 2. Is this another ensemble (mixed both best XGBoost and Random Forest models), which is different with page 8? \r\n\r\nThank you.",
      "votes": 1
    },
    {
      "id": 108150,
      "postDate": "2016-02-15T20:59:15.653Z",
      "content": "<p>Here are my R scripts for modeling.</p>\n\n<p>For feature engineering, I used Java and a MySql database, and the code is quite messy, so not very shareable :)</p>",
      "rawMarkdown": "Here are my R scripts for modeling.\r\n\r\nFor feature engineering, I used Java and a MySql database, and the code is quite messy, so not very shareable :)",
      "votes": 1
    },
    {
      "id": 110031,
      "postDate": "2016-03-02T08:59:22.587Z",
      "content": "<p>Thanks for your interest Paul, nice question !</p>\n\n<ol>\n<li>Models were all XGBoost indeed, and I defined &quot;most different&quot; as follows :</li>\n</ol>\n\n<p>a. Difference between two predictions for the same example is 0.4 , 0.2 , 0.1 , 0.05 , 0.025 or 0 depending on the level on which they disagree. (If all 5 predictions are the same, diff=0, if they disagree on the last prediction, small difference, then gets bigger if they disagree on the first ranks.)</p>\n\n<p>b. Difference between two submission files is the average difference between their predictions on each example.</p>\n\n<p>c. Difference between three models is the average difference between the three possible pairs.</p>\n\n<p>d. The three most different models are the ones that maximimze difference accross all possible combinations of three files in a list selected submissions.</p>\n\n<ol start=\"2\">\n<li>I am mentioning here the same ensembles of three most different XGBoosts, as I generated  many of them with different list of submissions. These ensembles score generally better than their components both on public and private leaderboard, this is why I wrote &quot;interesting results&quot;, but still, the best private leaderboard model was just a single model, this is why I wrote that the approach was not enough to make significant improvements.</li>\n</ol>\n\n<p>In case this is not clear and you prefer code reading, here is the code for ensembling submission files. </p>\n\n<p>Cheers !</p>",
      "rawMarkdown": "Thanks for your interest Paul, nice question !\r\n\r\n1. Models were all XGBoost indeed, and I defined \"most different\" as follows :\r\n\r\na. Difference between two predictions for the same example is 0.4 , 0.2 , 0.1 , 0.05 , 0.025 or 0 depending on the level on which they disagree. (If all 5 predictions are the same, diff=0, if they disagree on the last prediction, small difference, then gets bigger if they disagree on the first ranks.)\r\n\r\nb. Difference between two submission files is the average difference between their predictions on each example.\r\n\r\nc. Difference between three models is the average difference between the three possible pairs.\r\n\r\nd. The three most different models are the ones that maximimze difference accross all possible combinations of three files in a list selected submissions.\r\n\r\n\r\n2. I am mentioning here the same ensembles of three most different XGBoosts, as I generated  many of them with different list of submissions. These ensembles score generally better than their components both on public and private leaderboard, this is why I wrote \"interesting results\", but still, the best private leaderboard model was just a single model, this is why I wrote that the approach was not enough to make significant improvements.\r\n\r\nIn case this is not clear and you prefer code reading, here is the code for ensembling submission files. \r\n\r\nCheers !\r\n\r\n\r\n",
      "votes": 2
    },
    {
      "id": 108137,
      "postDate": "2016-02-15T19:34:27.023Z",
      "rawMarkdown": "",
      "isDeleted": true
    },
    {
      "id": 108106,
      "postDate": "2016-02-15T15:52:57.277Z",
      "rawMarkdown": "",
      "isDeleted": true
    },
    {
      "id": 110308,
      "postDate": "2016-03-04T14:43:08.153Z",
      "content": "<p>I got it !Thanks for your explanation.</p>",
      "rawMarkdown": "I got it !Thanks for your explanation."
    }
  ],
  "complete": true,
  "source": "live",
  "fetched_at": "2026-10-19T13:53:17Z",
  "competition_id": 4651,
  "competition_slug": "airbnb-recruiting-new-user-bookings",
  "team_id": 244468,
  "solution_url": "/competitions/airbnb-recruiting-new-user-bookings/writeups/randombishop-solution-presentation-54"
}