{
  "id": 21958,
  "title": "A rookie question",
  "url": "/competitions/grupo-bimbo-inventory-demand/discussion/21958",
  "author_name": "",
  "post_date": "2016-06-29T14:44:37.503Z",
  "votes": null,
  "comment_count": 2,
  "views": 785,
  "content": "<p>Dear All, </p>\n\n<p>I have strange situation. I use xgboost and validate the result of model two ways:\n1. I take one week as test, 8 week as train, or\n2. I do the same on loop so every week became test one time.</p>\n\n<p>Both ways I have RMLSE about 0,16. On leaderboard it is about 0,5. I can not understand why.</p>\n\n<p>Could you please give me a hint where should I look for mistake?</p>\n\n<p>Best regards!</p>\n\n<p>PS Just in case, here is code for RMLSE, should be fine I suppose</p>\n\n<pre><code>e &lt;- sqrt(mean((log(pred + 1) - log(target + 1))^2))\n</code></pre>",
  "messages": [
    {
      "id": "125446",
      "postDate": "06/29/2016 14:44:37",
      "content": "<p>Dear All, </p>\n\n<p>I have strange situation. I use xgboost and validate the result of model two ways:\n1. I take one week as test, 8 week as train, or\n2. I do the same on loop so every week became test one time.</p>\n\n<p>Both ways I have RMLSE about 0,16. On leaderboard it is about 0,5. I can not understand why.</p>\n\n<p>Could you please give me a hint where should I look for mistake?</p>\n\n<p>Best regards!</p>\n\n<p>PS Just in case, here is code for RMLSE, should be fine I suppose</p>\n\n<pre><code>e &lt;- sqrt(mean((log(pred + 1) - log(target + 1))^2))\n</code></pre>",
      "rawMarkdown": "Dear All, \r\n\r\nI have strange situation. I use xgboost and validate the result of model two ways:\r\n1. I take one week as test, 8 week as train, or\r\n2. I do the same on loop so every week became test one time.\r\n\r\nBoth ways I have RMLSE about 0,16. On leaderboard it is about 0,5. I can not understand why.\r\n\r\nCould you please give me a hint where should I look for mistake?\r\n\r\nBest regards!\r\n\r\nPS Just in case, here is code for RMLSE, should be fine I suppose\r\n\r\n    e <- sqrt(mean((log(pred + 1) - log(target + 1))^2))",
      "votes": null
    },
    {
      "id": "125458",
      "postDate": "06/29/2016 16:02:45",
      "content": "<p>If your features include information of test set (such as counting features, just don't count on the test set), you are easily overfitting rmse. Otherwise, I didn't see a problem in your setup. </p>",
      "rawMarkdown": "If your features include information of test set (such as counting features, just don't count on the test set), you are easily overfitting rmse. Otherwise, I didn't see a problem in your setup.",
      "votes": null
    },
    {
      "id": "125568",
      "postDate": "06/30/2016 13:14:38",
      "content": "<p>bobye, thank you for reply!</p>",
      "rawMarkdown": "bobye, thank you for reply!",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 125458,
      "author_name": "amazingbob",
      "author_url": "",
      "post_date": "06/29/2016 16:02:45",
      "content": "<p>If your features include information of test set (such as counting features, just don't count on the test set), you are easily overfitting rmse. Otherwise, I didn't see a problem in your setup. </p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 125568,
      "author_name": "sergeycherkasov",
      "author_url": "",
      "post_date": "06/30/2016 13:14:38",
      "content": "<p>bobye, thank you for reply!</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "125446": "Dear All, \r\n\r\nI have strange situation. I use xgboost and validate the result of model two ways:\r\n1. I take one week as test, 8 week as train, or\r\n2. I do the same on loop so every week became test one time.\r\n\r\nBoth ways I have RMLSE about 0,16. On leaderboard it is about 0,5. I can not understand why.\r\n\r\nCould you please give me a hint where should I look for mistake?\r\n\r\nBest regards!\r\n\r\nPS Just in case, here is code for RMLSE, should be fine I suppose\r\n\r\n    e <- sqrt(mean((log(pred + 1) - log(target + 1))^2))",
    "125458": "If your features include information of test set (such as counting features, just don't count on the test set), you are easily overfitting rmse. Otherwise, I didn't see a problem in your setup.",
    "125568": "bobye, thank you for reply!"
  },
  "source": "meta"
}