{
  "id": 21795,
  "title": "why performance don't get better when the grid is denser?",
  "url": "/competitions/facebook-v-predicting-check-ins/discussion/21795",
  "author_name": "",
  "post_date": "2016-06-20T04:21:38.967Z",
  "votes": null,
  "comment_count": 5,
  "views": 803,
  "content": "<p>Hi</p>\n\n<p>I am doing some grid size experiment in my basic gird knn model. My intuitive is that smaller grid size will get better result. I tried this idea in my train/val, I found that my cv score increase 5% when I use 100 * 100 gird.  However, my lb score doesn't increase that much  (~0.65 on my cv while ~0.58 in lb). </p>\n\n<p>I feel like my intuitive is wrong. And I am thinking is it because the smaller grid size will decrease the the number of training point which degrade the model performance?</p>\n\n<p>Anyone tries similar experiment or any idea? thank you</p>",
  "messages": [
    {
      "id": "124580",
      "postDate": "06/20/2016 04:21:38",
      "content": "<p>Hi</p>\n\n<p>I am doing some grid size experiment in my basic gird knn model. My intuitive is that smaller grid size will get better result. I tried this idea in my train/val, I found that my cv score increase 5% when I use 100 * 100 gird.  However, my lb score doesn't increase that much  (~0.65 on my cv while ~0.58 in lb). </p>\n\n<p>I feel like my intuitive is wrong. And I am thinking is it because the smaller grid size will decrease the the number of training point which degrade the model performance?</p>\n\n<p>Anyone tries similar experiment or any idea? thank you</p>",
      "rawMarkdown": "Hi\r\n\r\nI am doing some grid size experiment in my basic gird knn model. My intuitive is that smaller grid size will get better result. I tried this idea in my train/val, I found that my cv score increase 5% when I use 100 * 100 gird.  However, my lb score doesn't increase that much  (~0.65 on my cv while ~0.58 in lb). \r\n\r\nI feel like my intuitive is wrong. And I am thinking is it because the smaller grid size will decrease the the number of training point which degrade the model performance?\r\n\r\nAnyone tries similar experiment or any idea? thank you",
      "votes": null
    },
    {
      "id": "124582",
      "postDate": "06/20/2016 04:39:52",
      "content": "<p>When we use too small grids, some of useful information may be lost outside the grid even if we use overwrapped grids. When we use too large grids, noisy data may contaminate inside the grid. Therefore, I think that optimization is required. </p>",
      "rawMarkdown": "When we use too small grids, some of useful information may be lost outside the grid even if we use overwrapped grids. When we use too large grids, noisy data may contaminate inside the grid. Therefore, I think that optimization is required.",
      "votes": null
    },
    {
      "id": "124607",
      "postDate": "06/20/2016 11:04:21",
      "content": "<p>@ YuyaYamamoto</p>\n\n<p>Thank for your reply! gonna try to optimize the grid size:)</p>",
      "rawMarkdown": "YuyaYamamoto\r\n\r\nThank for your reply! gonna try to optimize the grid size:)",
      "votes": null
    },
    {
      "id": "124664",
      "postDate": "06/21/2016 01:03:05",
      "content": "<p>One of the interesting challenges here is that the optimal grid size varies over time and location. </p>",
      "rawMarkdown": "One of the interesting challenges here is that the optimal grid size varies over time and location.",
      "votes": null
    },
    {
      "id": "124745",
      "postDate": "06/21/2016 22:28:42",
      "content": "<p>[quote=JeffH;124664]</p>\n\n<p>One of the interesting challenges here is that the optimal grid size varies over time and location. </p>\n\n<p>[/quote]</p>\n\n<p>Sir, care to elaborate? </p>",
      "rawMarkdown": "[quote=JeffH;124664]\r\n\r\nOne of the interesting challenges here is that the optimal grid size varies over time and location. \r\n\r\n[/quote]\r\n\r\nSir, care to elaborate?",
      "votes": null
    },
    {
      "id": "124806",
      "postDate": "06/22/2016 14:00:45",
      "content": "<p>When you use too dense grid, the number of samples in each grid will become smaller, which increasing the risk of overfitting validation set. This can explain why you get better local result but no gain on LB.</p>",
      "rawMarkdown": "When you use too dense grid, the number of samples in each grid will become smaller, which increasing the risk of overfitting validation set. This can explain why you get better local result but no gain on LB.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 124582,
      "author_name": "nejumi",
      "author_url": "",
      "post_date": "06/20/2016 04:39:52",
      "content": "<p>When we use too small grids, some of useful information may be lost outside the grid even if we use overwrapped grids. When we use too large grids, noisy data may contaminate inside the grid. Therefore, I think that optimization is required. </p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 124607,
      "author_name": "kuanchen",
      "author_url": "",
      "post_date": "06/20/2016 11:04:21",
      "content": "<p>@ YuyaYamamoto</p>\n\n<p>Thank for your reply! gonna try to optimize the grid size:)</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 124664,
      "author_name": "jeffhebert",
      "author_url": "",
      "post_date": "06/21/2016 01:03:05",
      "content": "<p>One of the interesting challenges here is that the optimal grid size varies over time and location. </p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 124745,
      "author_name": "ascending",
      "author_url": "",
      "post_date": "06/21/2016 22:28:42",
      "content": "<p>[quote=JeffH;124664]</p>\n\n<p>One of the interesting challenges here is that the optimal grid size varies over time and location. </p>\n\n<p>[/quote]</p>\n\n<p>Sir, care to elaborate? </p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 124806,
      "author_name": "aikinogard",
      "author_url": "",
      "post_date": "06/22/2016 14:00:45",
      "content": "<p>When you use too dense grid, the number of samples in each grid will become smaller, which increasing the risk of overfitting validation set. This can explain why you get better local result but no gain on LB.</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "124580": "Hi\r\n\r\nI am doing some grid size experiment in my basic gird knn model. My intuitive is that smaller grid size will get better result. I tried this idea in my train/val, I found that my cv score increase 5% when I use 100 * 100 gird.  However, my lb score doesn't increase that much  (~0.65 on my cv while ~0.58 in lb). \r\n\r\nI feel like my intuitive is wrong. And I am thinking is it because the smaller grid size will decrease the the number of training point which degrade the model performance?\r\n\r\nAnyone tries similar experiment or any idea? thank you",
    "124582": "When we use too small grids, some of useful information may be lost outside the grid even if we use overwrapped grids. When we use too large grids, noisy data may contaminate inside the grid. Therefore, I think that optimization is required.",
    "124607": "YuyaYamamoto\r\n\r\nThank for your reply! gonna try to optimize the grid size:)",
    "124664": "One of the interesting challenges here is that the optimal grid size varies over time and location.",
    "124745": "[quote=JeffH;124664]\r\n\r\nOne of the interesting challenges here is that the optimal grid size varies over time and location. \r\n\r\n[/quote]\r\n\r\nSir, care to elaborate?",
    "124806": "When you use too dense grid, the number of samples in each grid will become smaller, which increasing the risk of overfitting validation set. This can explain why you get better local result but no gain on LB."
  },
  "source": "meta"
}