{
  "id": 81529,
  "title": "How did LB score differ from CV score?",
  "url": "/competitions/ga-customer-revenue-prediction/discussion/81529",
  "author_name": "",
  "post_date": "2019-02-22T07:16:53.627358200Z",
  "votes": 3,
  "comment_count": 3,
  "views": 0,
  "content": "<p>Hi Kagglers,</p>\n\n<p>Thanks everyone for your great efforts to this <strong>meaningless</strong> competition. I just wanted to ask how different LB score is rather than CV score.</p>\n\n<p>My CV score was around 0.3094 and LB score was 0.8868. I saw people claiming that their CV score was also around 0.30 - 0.33 before submission deadline. If my submission was filled with 0, my CV score was 0.3302. I used a validation strategy like below:\n           <em>5.5 month train - 1.5month gap - 2 month test</em>\nThe target of train period was obtained from test period.</p>\n\n<p>What i predicted(scored data or predicted data) is from 1 May 2018 to 15 October 2018.</p>\n\n<p>I am asking: Why is LB score so different than CV score?</p>",
  "messages": [
    {
      "id": "476460",
      "postDate": "02/22/2019 07:16:53",
      "content": "<p>Hi Kagglers,</p>\n\n<p>Thanks everyone for your great efforts to this <strong>meaningless</strong> competition. I just wanted to ask how different LB score is rather than CV score.</p>\n\n<p>My CV score was around 0.3094 and LB score was 0.8868. I saw people claiming that their CV score was also around 0.30 - 0.33 before submission deadline. If my submission was filled with 0, my CV score was 0.3302. I used a validation strategy like below:\n           <em>5.5 month train - 1.5month gap - 2 month test</em>\nThe target of train period was obtained from test period.</p>\n\n<p>What i predicted(scored data or predicted data) is from 1 May 2018 to 15 October 2018.</p>\n\n<p>I am asking: Why is LB score so different than CV score?</p>",
      "rawMarkdown": "Hi Kagglers,\n\nThanks everyone for your great efforts to this **meaningless** competition. I just wanted to ask how different LB score is rather than CV score.\n\nMy CV score was around 0.3094 and LB score was 0.8868. I saw people claiming that their CV score was also around 0.30 - 0.33 before submission deadline. If my submission was filled with 0, my CV score was 0.3302. I used a validation strategy like below:\n           *5.5 month train - 1.5month gap - 2 month test*\nThe target of train period was obtained from test period.\n\nWhat i predicted(scored data or predicted data) is from 1 May 2018 to 15 October 2018.\n\nI am asking: Why is LB score so different than CV score?",
      "votes": null
    },
    {
      "id": "476488",
      "postDate": "02/22/2019 08:29:52",
      "content": "<p>I guess there is something wrong. Also among my submissions, it seems I had labeled one submission as buggy (as far as I remember it was overestimating target a bit). I didn't select that submission and it scored better than my actual submission. </p>\n\n<p>But it may also be that test period includes Christmas time which I didn't put specific effort.</p>",
      "rawMarkdown": "I guess there is something wrong. Also among my submissions, it seems I had labeled one submission as buggy (as far as I remember it was overestimating target a bit). I didn't select that submission and it scored better than my actual submission. \n\nBut it may also be that test period includes Christmas time which I didn't put specific effort.",
      "votes": null
    },
    {
      "id": "476563",
      "postDate": "02/22/2019 10:37:23",
      "content": "<p>My best theory is that there must have been some sampling of the training dataset. That would also explain the really strange lack of strong seasonality of the data. </p>\n\n<p>I differ about this competition being meaningless. Or, at least, I think it was much more meaningless in its initial format in which data from the future could be used. The competition has been a pain, that is true, but not meaningless in my opinion. </p>",
      "rawMarkdown": "My best theory is that there must have been some sampling of the training dataset. That would also explain the really strange lack of strong seasonality of the data. \n\nI differ about this competition being meaningless. Or, at least, I think it was much more meaningless in its initial format in which data from the future could be used. The competition has been a pain, that is true, but not meaningless in my opinion.",
      "votes": null
    },
    {
      "id": "476587",
      "postDate": "02/22/2019 11:23:08",
      "content": "<p>My best scoring submission, also not selected, I had labelled \"sanity check\"! Again similar to you, I felt some of my submissions were overestimating the target but perhaps focusing on ones with lower estimates was not a good move on my part.</p>",
      "rawMarkdown": "My best scoring submission, also not selected, I had labelled \"sanity check\"! Again similar to you, I felt some of my submissions were overestimating the target but perhaps focusing on ones with lower estimates was not a good move on my part.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 476488,
      "author_name": "aerdem4",
      "author_url": "",
      "post_date": "02/22/2019 08:29:52",
      "content": "<p>I guess there is something wrong. Also among my submissions, it seems I had labeled one submission as buggy (as far as I remember it was overestimating target a bit). I didn't select that submission and it scored better than my actual submission. </p>\n\n<p>But it may also be that test period includes Christmas time which I didn't put specific effort.</p>",
      "votes": null,
      "replies": [
        {
          "id": 476587,
          "author_name": "datahobbit",
          "author_url": "",
          "post_date": "02/22/2019 11:23:08",
          "content": "<p>My best scoring submission, also not selected, I had labelled \"sanity check\"! Again similar to you, I felt some of my submissions were overestimating the target but perhaps focusing on ones with lower estimates was not a good move on my part.</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 476563,
      "author_name": "miguelpm",
      "author_url": "",
      "post_date": "02/22/2019 10:37:23",
      "content": "<p>My best theory is that there must have been some sampling of the training dataset. That would also explain the really strange lack of strong seasonality of the data. </p>\n\n<p>I differ about this competition being meaningless. Or, at least, I think it was much more meaningless in its initial format in which data from the future could be used. The competition has been a pain, that is true, but not meaningless in my opinion. </p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "476460": "Hi Kagglers,\n\nThanks everyone for your great efforts to this **meaningless** competition. I just wanted to ask how different LB score is rather than CV score.\n\nMy CV score was around 0.3094 and LB score was 0.8868. I saw people claiming that their CV score was also around 0.30 - 0.33 before submission deadline. If my submission was filled with 0, my CV score was 0.3302. I used a validation strategy like below:\n           *5.5 month train - 1.5month gap - 2 month test*\nThe target of train period was obtained from test period.\n\nWhat i predicted(scored data or predicted data) is from 1 May 2018 to 15 October 2018.\n\nI am asking: Why is LB score so different than CV score?",
    "476488": "I guess there is something wrong. Also among my submissions, it seems I had labeled one submission as buggy (as far as I remember it was overestimating target a bit). I didn't select that submission and it scored better than my actual submission. \n\nBut it may also be that test period includes Christmas time which I didn't put specific effort.",
    "476563": "My best theory is that there must have been some sampling of the training dataset. That would also explain the really strange lack of strong seasonality of the data. \n\nI differ about this competition being meaningless. Or, at least, I think it was much more meaningless in its initial format in which data from the future could be used. The competition has been a pain, that is true, but not meaningless in my opinion.",
    "476587": "My best scoring submission, also not selected, I had labelled \"sanity check\"! Again similar to you, I felt some of my submissions were overestimating the target but perhaps focusing on ones with lower estimates was not a good move on my part."
  },
  "source": "meta"
}