{
  "id": 22162,
  "title": "26th place solution",
  "url": "/competitions/facebook-v-predicting-check-ins/discussion/22162",
  "author_name": "Mircea Neagovici",
  "post_date": "2016-07-11T17:37:23.467000",
  "votes": 2,
  "comment_count": 0,
  "views": 373,
  "content": "<p>Congratulations everyone, and especially to the guys at the top. Thanks to the Kaggle community for the discussions on the forum and for sharing several great ideas.</p>\n\n<p>My solution was an assemble of 8 models</p>\n\n<ul>\n<li>2 XGB models. I used a 75x75 grid and extended the train data by 1.75. Best score with this approach was 0.58417.</li>\n<li>2 XGB models, also on a grid (100x100) but with overlap. I ran XGB for 99*99 cells with area 4 times the cell size, so each cell was included in 4 larger cells. For a given cell the scores comes from the 4 2x2 cells that contain it. I kept the top 5 places in each larger cell and calculated a simple sum of the scores for the smaller cell. Best score for these models was 0.59279</li>\n<li>1 model based on RF</li>\n<li>3 models using KNN, very similar to the public scripts. </li>\n</ul>\n\n<p>For the first 5 models I ran a grid search for determining the best cell size, extended train percentage and number of places threshold. Because for larger cells the threshold is higher I used a variable threshold first to eliminate 10% of the places, then tuned the threshold.</p>\n\n<p>I tried to improve the score by using a finer grid (like 200x200) and for each cell train XGB using a circle with radius between 1 and 2 the size of the cell. I was expecting this method to be slow but potentially improve the score, which was not the case. If anyone has tried the same approach I'd be interested to hear their results.</p>\n\n<p>Because I wasn't able to make XGB use 100% CPU on my machine, I created a queue of tasks (one for each cell) and used 10 threads to execute these tasks. To speed up the execution I ran on my desktop and my laptop and merged results. In the last few days of the competition I also used an Azure VM but the cost of a VM with 16 CPU is quite high.</p>\n\n<p>The code (needs cleanup) is at <a href=\"https://github.com/mircean/ML/tree/master/Facebook\">https://github.com/mircean/ML/tree/master/Facebook</a>. </p>",
  "messages": [
    {
      "id": 126664,
      "postDate": "2016-07-11T17:37:23.467Z",
      "content": "<p>Congratulations everyone, and especially to the guys at the top. Thanks to the Kaggle community for the discussions on the forum and for sharing several great ideas.</p>\n\n<p>My solution was an assemble of 8 models</p>\n\n<ul>\n<li>2 XGB models. I used a 75x75 grid and extended the train data by 1.75. Best score with this approach was 0.58417.</li>\n<li>2 XGB models, also on a grid (100x100) but with overlap. I ran XGB for 99*99 cells with area 4 times the cell size, so each cell was included in 4 larger cells. For a given cell the scores comes from the 4 2x2 cells that contain it. I kept the top 5 places in each larger cell and calculated a simple sum of the scores for the smaller cell. Best score for these models was 0.59279</li>\n<li>1 model based on RF</li>\n<li>3 models using KNN, very similar to the public scripts. </li>\n</ul>\n\n<p>For the first 5 models I ran a grid search for determining the best cell size, extended train percentage and number of places threshold. Because for larger cells the threshold is higher I used a variable threshold first to eliminate 10% of the places, then tuned the threshold.</p>\n\n<p>I tried to improve the score by using a finer grid (like 200x200) and for each cell train XGB using a circle with radius between 1 and 2 the size of the cell. I was expecting this method to be slow but potentially improve the score, which was not the case. If anyone has tried the same approach I'd be interested to hear their results.</p>\n\n<p>Because I wasn't able to make XGB use 100% CPU on my machine, I created a queue of tasks (one for each cell) and used 10 threads to execute these tasks. To speed up the execution I ran on my desktop and my laptop and merged results. In the last few days of the competition I also used an Azure VM but the cost of a VM with 16 CPU is quite high.</p>\n\n<p>The code (needs cleanup) is at <a href=\"https://github.com/mircean/ML/tree/master/Facebook\">https://github.com/mircean/ML/tree/master/Facebook</a>. </p>",
      "rawMarkdown": "Congratulations everyone, and especially to the guys at the top. Thanks to the Kaggle community for the discussions on the forum and for sharing several great ideas.\r\n\r\nMy solution was an assemble of 8 models\r\n\r\n* 2 XGB models. I used a 75x75 grid and extended the train data by 1.75. Best score with this approach was 0.58417.\r\n* 2 XGB models, also on a grid (100x100) but with overlap. I ran XGB for 99*99 cells with area 4 times the cell size, so each cell was included in 4 larger cells. For a given cell the scores comes from the 4 2x2 cells that contain it. I kept the top 5 places in each larger cell and calculated a simple sum of the scores for the smaller cell. Best score for these models was 0.59279\r\n* 1 model based on RF\r\n* 3 models using KNN, very similar to the public scripts. \r\n\r\nFor the first 5 models I ran a grid search for determining the best cell size, extended train percentage and number of places threshold. Because for larger cells the threshold is higher I used a variable threshold first to eliminate 10% of the places, then tuned the threshold.\r\n\r\nI tried to improve the score by using a finer grid (like 200x200) and for each cell train XGB using a circle with radius between 1 and 2 the size of the cell. I was expecting this method to be slow but potentially improve the score, which was not the case. If anyone has tried the same approach I'd be interested to hear their results.\r\n\r\nBecause I wasn't able to make XGB use 100% CPU on my machine, I created a queue of tasks (one for each cell) and used 10 threads to execute these tasks. To speed up the execution I ran on my desktop and my laptop and merged results. In the last few days of the competition I also used an Azure VM but the cost of a VM with 16 CPU is quite high.\r\n\r\nThe code (needs cleanup) is at https://github.com/mircean/ML/tree/master/Facebook. ",
      "votes": 2
    }
  ],
  "comments": [],
  "raw_markdown_by_id": {
    "126664": "Congratulations everyone, and especially to the guys at the top. Thanks to the Kaggle community for the discussions on the forum and for sharing several great ideas.\r\n\r\nMy solution was an assemble of 8 models\r\n\r\n* 2 XGB models. I used a 75x75 grid and extended the train data by 1.75. Best score with this approach was 0.58417.\r\n* 2 XGB models, also on a grid (100x100) but with overlap. I ran XGB for 99*99 cells with area 4 times the cell size, so each cell was included in 4 larger cells. For a given cell the scores comes from the 4 2x2 cells that contain it. I kept the top 5 places in each larger cell and calculated a simple sum of the scores for the smaller cell. Best score for these models was 0.59279\r\n* 1 model based on RF\r\n* 3 models using KNN, very similar to the public scripts. \r\n\r\nFor the first 5 models I ran a grid search for determining the best cell size, extended train percentage and number of places threshold. Because for larger cells the threshold is higher I used a variable threshold first to eliminate 10% of the places, then tuned the threshold.\r\n\r\nI tried to improve the score by using a finer grid (like 200x200) and for each cell train XGB using a circle with radius between 1 and 2 the size of the cell. I was expecting this method to be slow but potentially improve the score, which was not the case. If anyone has tried the same approach I'd be interested to hear their results.\r\n\r\nBecause I wasn't able to make XGB use 100% CPU on my machine, I created a queue of tasks (one for each cell) and used 10 threads to execute these tasks. To speed up the execution I ran on my desktop and my laptop and merged results. In the last few days of the competition I also used an Azure VM but the cost of a VM with 16 CPU is quite high.\r\n\r\nThe code (needs cleanup) is at https://github.com/mircean/ML/tree/master/Facebook. "
  }
}