{
  "id": 21893,
  "title": "Testing your model against a small benchmark dataset",
  "url": "/competitions/grupo-bimbo-inventory-demand/discussion/21893",
  "author_name": "",
  "post_date": "2016-06-26T13:30:53.140Z",
  "votes": null,
  "comment_count": 2,
  "views": 600,
  "content": "<p>Hi all,\nI am new to this platform and to such large amount of data. After struggling with it for quite some time I am able to build up a simple model. Now I want to check it. Kaggle requires you to submit the solution to all the 7M queries at once to get a score. It is taking quite some time to produce the result for all the queries (my laptop isn't that powerful also). So is there any method that I can use to check how my model is performing. Checking against median or mean solution simply wouldn't work as far as I can guess by looking at the evaluation metric. If possible, this would help me to check against a smaller test size maybe ~40K.</p>",
  "messages": [
    {
      "id": "125118",
      "postDate": "06/26/2016 13:30:53",
      "content": "<p>Hi all,\nI am new to this platform and to such large amount of data. After struggling with it for quite some time I am able to build up a simple model. Now I want to check it. Kaggle requires you to submit the solution to all the 7M queries at once to get a score. It is taking quite some time to produce the result for all the queries (my laptop isn't that powerful also). So is there any method that I can use to check how my model is performing. Checking against median or mean solution simply wouldn't work as far as I can guess by looking at the evaluation metric. If possible, this would help me to check against a smaller test size maybe ~40K.</p>",
      "rawMarkdown": "Hi all,\r\nI am new to this platform and to such large amount of data. After struggling with it for quite some time I am able to build up a simple model. Now I want to check it. Kaggle requires you to submit the solution to all the 7M queries at once to get a score. It is taking quite some time to produce the result for all the queries (my laptop isn't that powerful also). So is there any method that I can use to check how my model is performing. Checking against median or mean solution simply wouldn't work as far as I can guess by looking at the evaluation metric. If possible, this would help me to check against a smaller test size maybe ~40K.",
      "votes": null
    },
    {
      "id": "125136",
      "postDate": "06/26/2016 17:31:33",
      "content": "<p>It's often better to build a local validation framework. Say, use the 9th week as the validation set. If you want to have a score in LB. You can prepare a baseline (median or mean), and replace the subset which your algorithm predicts. If your algorithm performs better than the baseline, you will observe a better score at LB. </p>",
      "rawMarkdown": "It's often better to build a local validation framework. Say, use the 9th week as the validation set. If you want to have a score in LB. You can prepare a baseline (median or mean), and replace the subset which your algorithm predicts. If your algorithm performs better than the baseline, you will observe a better score at LB.",
      "votes": null
    },
    {
      "id": "125140",
      "postDate": "06/26/2016 17:58:41",
      "content": "<p>Oh... Awesome. I will try that.</p>",
      "rawMarkdown": "Oh... Awesome. I will try that.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 125136,
      "author_name": "amazingbob",
      "author_url": "",
      "post_date": "06/26/2016 17:31:33",
      "content": "<p>It's often better to build a local validation framework. Say, use the 9th week as the validation set. If you want to have a score in LB. You can prepare a baseline (median or mean), and replace the subset which your algorithm predicts. If your algorithm performs better than the baseline, you will observe a better score at LB. </p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 125140,
      "author_name": "bytestorm",
      "author_url": "",
      "post_date": "06/26/2016 17:58:41",
      "content": "<p>Oh... Awesome. I will try that.</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "125118": "Hi all,\r\nI am new to this platform and to such large amount of data. After struggling with it for quite some time I am able to build up a simple model. Now I want to check it. Kaggle requires you to submit the solution to all the 7M queries at once to get a score. It is taking quite some time to produce the result for all the queries (my laptop isn't that powerful also). So is there any method that I can use to check how my model is performing. Checking against median or mean solution simply wouldn't work as far as I can guess by looking at the evaluation metric. If possible, this would help me to check against a smaller test size maybe ~40K.",
    "125136": "It's often better to build a local validation framework. Say, use the 9th week as the validation set. If you want to have a score in LB. You can prepare a baseline (median or mean), and replace the subset which your algorithm predicts. If your algorithm performs better than the baseline, you will observe a better score at LB.",
    "125140": "Oh... Awesome. I will try that."
  },
  "source": "meta"
}