{
  "id": 22268,
  "title": "Pre Processing: use the log function!",
  "url": "/competitions/grupo-bimbo-inventory-demand/discussion/22268",
  "author_name": "",
  "post_date": "2016-07-14T13:14:37.250Z",
  "votes": 4,
  "comment_count": 6,
  "views": 923,
  "content": "<p>For those of you who don't already know, minimizing RMSLE for x is equivalent to minimizing RMSE for log(x).</p>\n\n<p>In <strong>any</strong> Kaggle competition using RMSLE as a metric, the first step BEFORE you do anything is to take the log of the target (log1p in R, numpy.log1p in python). You can then do all your pre-processing in terms of means, and your modeling in terms of RMSE.</p>\n\n<p>Don't forget to call expm1 or numpy.expm1 after making your predictions.  This undoes the log transform so your scores on Kaggle will be correct.</p>\n\n<p>(one note: we use log1p instead of log because the target contains some zeros and this effectively handles them)</p>",
  "messages": [
    {
      "id": "127521",
      "postDate": "07/14/2016 13:14:37",
      "content": "<p>For those of you who don't already know, minimizing RMSLE for x is equivalent to minimizing RMSE for log(x).</p>\n\n<p>In <strong>any</strong> Kaggle competition using RMSLE as a metric, the first step BEFORE you do anything is to take the log of the target (log1p in R, numpy.log1p in python). You can then do all your pre-processing in terms of means, and your modeling in terms of RMSE.</p>\n\n<p>Don't forget to call expm1 or numpy.expm1 after making your predictions.  This undoes the log transform so your scores on Kaggle will be correct.</p>\n\n<p>(one note: we use log1p instead of log because the target contains some zeros and this effectively handles them)</p>",
      "rawMarkdown": "For those of you who don't already know, minimizing RMSLE for x is equivalent to minimizing RMSE for log(x).\r\n\r\nIn **any** Kaggle competition using RMSLE as a metric, the first step BEFORE you do anything is to take the log of the target (log1p in R, numpy.log1p in python). You can then do all your pre-processing in terms of means, and your modeling in terms of RMSE.\r\n\r\nDon't forget to call expm1 or numpy.expm1 after making your predictions.  This undoes the log transform so your scores on Kaggle will be correct.\r\n\r\n(one note: we use log1p instead of log because the target contains some zeros and this effectively handles them)",
      "votes": null
    },
    {
      "id": "127527",
      "postDate": "07/14/2016 13:28:07",
      "content": "<p>Thanks for the info, Zach.</p>\n\n<p>How would we go about ensembling two submission files? I've tried plain mean and <code>expm1((log1p(a) + log1p(b)) / 2)</code> but they both performed very badly on LB.</p>",
      "rawMarkdown": "Thanks for the info, Zach.\r\n\r\nHow would we go about ensembling two submission files? I've tried plain mean and `expm1((log1p(a) + log1p(b)) / 2)` but they both performed very badly on LB.",
      "votes": null
    },
    {
      "id": "127529",
      "postDate": "07/14/2016 13:35:54",
      "content": "<p>Interesting.  Did the individual models both do well on the leaderboard?  How did (a + b) / 2 do?</p>",
      "rawMarkdown": "Interesting.  Did the individual models both do well on the leaderboard?  How did (a + b) / 2 do?",
      "votes": null
    },
    {
      "id": "127530",
      "postDate": "07/14/2016 13:37:46",
      "content": "<p>[quote=Zach;127529]</p>\n\n<p>Interesting.  Did the individual models both do well on the leaderboard?  How did (a + b) / 2 do?</p>\n\n<p>[/quote]</p>\n\n<p>Both models ~0.47, simple mean got 0.67 :/</p>\n\n<p>Pearson correlation between models, 0.84</p>",
      "rawMarkdown": "[quote=Zach;127529]\r\n\r\nInteresting.  Did the individual models both do well on the leaderboard?  How did (a + b) / 2 do?\r\n\r\n[/quote]\r\n\r\nBoth models ~0.47, simple mean got 0.67 :/\r\n\r\nPearson correlation between models, 0.84",
      "votes": null
    },
    {
      "id": "127533",
      "postDate": "07/14/2016 13:41:55",
      "content": "<p>What was the score of expm1((log1p(a) + log1p(b)) / 2)  ?</p>",
      "rawMarkdown": "What was the score of expm1((log1p(a) + log1p(b)) / 2)  ?",
      "votes": null
    },
    {
      "id": "127668",
      "postDate": "07/14/2016 20:56:25",
      "content": "<p>[quote=anokas;127530]</p>\n\n<p>[quote=Zach;127529]</p>\n\n<p>Interesting.  Did the individual models both do well on the leaderboard?  How did (a + b) / 2 do?</p>\n\n<p>[/quote]</p>\n\n<p>Both models ~0.47, simple mean got 0.67 :/</p>\n\n<p>Pearson correlation between models, 0.84</p>\n\n<p>[/quote]</p>\n\n<p>Make sure you sort the ids before ensembling. I know its a simple mistake but I got caught out with one of my submissions.</p>",
      "rawMarkdown": "[quote=anokas;127530]\r\n\r\n[quote=Zach;127529]\r\n\r\nInteresting.  Did the individual models both do well on the leaderboard?  How did (a + b) / 2 do?\r\n\r\n[/quote]\r\n\r\nBoth models ~0.47, simple mean got 0.67 :/\r\n\r\nPearson correlation between models, 0.84\r\n\r\n[/quote]\r\n\r\nMake sure you sort the ids before ensembling. I know its a simple mistake but I got caught out with one of my submissions.",
      "votes": null
    },
    {
      "id": "127730",
      "postDate": "07/15/2016 05:23:32",
      "content": "<p>I made that remark several weeks ago, see <a href=\"https://www.kaggle.com/kobakhit/grupo-bimbo-inventory-demand/xgboost/comments#124599\">https://www.kaggle.com/kobakhit/grupo-bimbo-inventory-demand/xgboost/comments#124599</a></p>\n\n<p>It is good practice to read all the forum entries of a competition.  I learn a lot doing it.</p>",
      "rawMarkdown": "I made that remark several weeks ago, see https://www.kaggle.com/kobakhit/grupo-bimbo-inventory-demand/xgboost/comments#124599\r\n\r\nIt is good practice to read all the forum entries of a competition.  I learn a lot doing it.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 127527,
      "author_name": "anokas",
      "author_url": "",
      "post_date": "07/14/2016 13:28:07",
      "content": "<p>Thanks for the info, Zach.</p>\n\n<p>How would we go about ensembling two submission files? I've tried plain mean and <code>expm1((log1p(a) + log1p(b)) / 2)</code> but they both performed very badly on LB.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 127529,
      "author_name": "zachmayer",
      "author_url": "",
      "post_date": "07/14/2016 13:35:54",
      "content": "<p>Interesting.  Did the individual models both do well on the leaderboard?  How did (a + b) / 2 do?</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 127530,
      "author_name": "anokas",
      "author_url": "",
      "post_date": "07/14/2016 13:37:46",
      "content": "<p>[quote=Zach;127529]</p>\n\n<p>Interesting.  Did the individual models both do well on the leaderboard?  How did (a + b) / 2 do?</p>\n\n<p>[/quote]</p>\n\n<p>Both models ~0.47, simple mean got 0.67 :/</p>\n\n<p>Pearson correlation between models, 0.84</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 127533,
      "author_name": "zachmayer",
      "author_url": "",
      "post_date": "07/14/2016 13:41:55",
      "content": "<p>What was the score of expm1((log1p(a) + log1p(b)) / 2)  ?</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 127668,
      "author_name": "centsoft",
      "author_url": "",
      "post_date": "07/14/2016 20:56:25",
      "content": "<p>[quote=anokas;127530]</p>\n\n<p>[quote=Zach;127529]</p>\n\n<p>Interesting.  Did the individual models both do well on the leaderboard?  How did (a + b) / 2 do?</p>\n\n<p>[/quote]</p>\n\n<p>Both models ~0.47, simple mean got 0.67 :/</p>\n\n<p>Pearson correlation between models, 0.84</p>\n\n<p>[/quote]</p>\n\n<p>Make sure you sort the ids before ensembling. I know its a simple mistake but I got caught out with one of my submissions.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 127730,
      "author_name": "cpmpml",
      "author_url": "",
      "post_date": "07/15/2016 05:23:32",
      "content": "<p>I made that remark several weeks ago, see <a href=\"https://www.kaggle.com/kobakhit/grupo-bimbo-inventory-demand/xgboost/comments#124599\">https://www.kaggle.com/kobakhit/grupo-bimbo-inventory-demand/xgboost/comments#124599</a></p>\n\n<p>It is good practice to read all the forum entries of a competition.  I learn a lot doing it.</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "127521": "For those of you who don't already know, minimizing RMSLE for x is equivalent to minimizing RMSE for log(x).\r\n\r\nIn **any** Kaggle competition using RMSLE as a metric, the first step BEFORE you do anything is to take the log of the target (log1p in R, numpy.log1p in python). You can then do all your pre-processing in terms of means, and your modeling in terms of RMSE.\r\n\r\nDon't forget to call expm1 or numpy.expm1 after making your predictions.  This undoes the log transform so your scores on Kaggle will be correct.\r\n\r\n(one note: we use log1p instead of log because the target contains some zeros and this effectively handles them)",
    "127527": "Thanks for the info, Zach.\r\n\r\nHow would we go about ensembling two submission files? I've tried plain mean and `expm1((log1p(a) + log1p(b)) / 2)` but they both performed very badly on LB.",
    "127529": "Interesting.  Did the individual models both do well on the leaderboard?  How did (a + b) / 2 do?",
    "127530": "[quote=Zach;127529]\r\n\r\nInteresting.  Did the individual models both do well on the leaderboard?  How did (a + b) / 2 do?\r\n\r\n[/quote]\r\n\r\nBoth models ~0.47, simple mean got 0.67 :/\r\n\r\nPearson correlation between models, 0.84",
    "127533": "What was the score of expm1((log1p(a) + log1p(b)) / 2)  ?",
    "127668": "[quote=anokas;127530]\r\n\r\n[quote=Zach;127529]\r\n\r\nInteresting.  Did the individual models both do well on the leaderboard?  How did (a + b) / 2 do?\r\n\r\n[/quote]\r\n\r\nBoth models ~0.47, simple mean got 0.67 :/\r\n\r\nPearson correlation between models, 0.84\r\n\r\n[/quote]\r\n\r\nMake sure you sort the ids before ensembling. I know its a simple mistake but I got caught out with one of my submissions.",
    "127730": "I made that remark several weeks ago, see https://www.kaggle.com/kobakhit/grupo-bimbo-inventory-demand/xgboost/comments#124599\r\n\r\nIt is good practice to read all the forum entries of a competition.  I learn a lot doing it."
  },
  "source": "meta"
}