{
  "id": 41640,
  "title": "Reproducing the eval metric",
  "url": "/competitions/favorita-grocery-sales-forecasting/discussion/41640",
  "author_name": "",
  "post_date": "2017-10-21T19:20:46.204293200Z",
  "votes": 1,
  "comment_count": 1,
  "views": 0,
  "content": "<p>When I write up the test metric locally, it estimates that using 0 as the prediction for all samples gives an RMSE of ~1.8, but when I upload a submission of all 0s, I get ~1.1 RMSE.</p>\n\n<p>('2017-08-03', 1.7687277497181995)\n('2017-08-04', 1.8300465351365809)\n('2017-08-05', 1.9126148389467124)\n('2017-08-06', 1.963348122867391)\n('2017-08-07', 1.8179390818260135)\n('2017-08-08', 1.7663755754824728)\n('2017-08-09', 1.7713765347330042)\n('2017-08-10', 1.7137904800659447)\n('2017-08-11', 1.8320157485151416)\n('2017-08-12', 1.8011667853294202)\n('2017-08-13', 1.855424643594738)\n('2017-08-14', 1.7853790290046632)\n('estimated accuracy is ', 1.8451043111262129)</p>\n\n<p>Anyone else experiencing this?</p>",
  "messages": [
    {
      "id": "233916",
      "postDate": "10/21/2017 19:20:46",
      "content": "<p>When I write up the test metric locally, it estimates that using 0 as the prediction for all samples gives an RMSE of ~1.8, but when I upload a submission of all 0s, I get ~1.1 RMSE.</p>\n\n<p>('2017-08-03', 1.7687277497181995)\n('2017-08-04', 1.8300465351365809)\n('2017-08-05', 1.9126148389467124)\n('2017-08-06', 1.963348122867391)\n('2017-08-07', 1.8179390818260135)\n('2017-08-08', 1.7663755754824728)\n('2017-08-09', 1.7713765347330042)\n('2017-08-10', 1.7137904800659447)\n('2017-08-11', 1.8320157485151416)\n('2017-08-12', 1.8011667853294202)\n('2017-08-13', 1.855424643594738)\n('2017-08-14', 1.7853790290046632)\n('estimated accuracy is ', 1.8451043111262129)</p>\n\n<p>Anyone else experiencing this?</p>",
      "rawMarkdown": "When I write up the test metric locally, it estimates that using 0 as the prediction for all samples gives an RMSE of ~1.8, but when I upload a submission of all 0s, I get ~1.1 RMSE.\n\n('2017-08-03', 1.7687277497181995)\n('2017-08-04', 1.8300465351365809)\n('2017-08-05', 1.9126148389467124)\n('2017-08-06', 1.963348122867391)\n('2017-08-07', 1.8179390818260135)\n('2017-08-08', 1.7663755754824728)\n('2017-08-09', 1.7713765347330042)\n('2017-08-10', 1.7137904800659447)\n('2017-08-11', 1.8320157485151416)\n('2017-08-12', 1.8011667853294202)\n('2017-08-13', 1.855424643594738)\n('2017-08-14', 1.7853790290046632)\n('estimated accuracy is ', 1.8451043111262129)\n\nAnyone else experiencing this?",
      "votes": null
    },
    {
      "id": "236907",
      "postDate": "10/28/2017 16:26:18",
      "content": "<p>Probably, it is due to the fact, that at train dataset we have only samples with non-zero unit sales. Of course, we don't know whether there are samples with zero unit sales in test dataset. But, it would definitely make sense to have such samples.</p>",
      "rawMarkdown": "Probably, it is due to the fact, that at train dataset we have only samples with non-zero unit sales. Of course, we don't know whether there are samples with zero unit sales in test dataset. But, it would definitely make sense to have such samples.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 236907,
      "author_name": "deyury",
      "author_url": "",
      "post_date": "10/28/2017 16:26:18",
      "content": "<p>Probably, it is due to the fact, that at train dataset we have only samples with non-zero unit sales. Of course, we don't know whether there are samples with zero unit sales in test dataset. But, it would definitely make sense to have such samples.</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "233916": "When I write up the test metric locally, it estimates that using 0 as the prediction for all samples gives an RMSE of ~1.8, but when I upload a submission of all 0s, I get ~1.1 RMSE.\n\n('2017-08-03', 1.7687277497181995)\n('2017-08-04', 1.8300465351365809)\n('2017-08-05', 1.9126148389467124)\n('2017-08-06', 1.963348122867391)\n('2017-08-07', 1.8179390818260135)\n('2017-08-08', 1.7663755754824728)\n('2017-08-09', 1.7713765347330042)\n('2017-08-10', 1.7137904800659447)\n('2017-08-11', 1.8320157485151416)\n('2017-08-12', 1.8011667853294202)\n('2017-08-13', 1.855424643594738)\n('2017-08-14', 1.7853790290046632)\n('estimated accuracy is ', 1.8451043111262129)\n\nAnyone else experiencing this?",
    "236907": "Probably, it is due to the fact, that at train dataset we have only samples with non-zero unit sales. Of course, we don't know whether there are samples with zero unit sales in test dataset. But, it would definitely make sense to have such samples."
  },
  "source": "meta"
}