{
  "id": 301257,
  "title": "Different score during submission",
  "url": "/competitions/g-research-crypto-forecasting/discussion/301257",
  "author_name": "",
  "post_date": "2022-01-16T17:42:11.530702800Z",
  "votes": null,
  "comment_count": 2,
  "views": 0,
  "content": "<p>I have seen people claim \"2021-06-13 00:00:00\" is the start of the testing set for the leaderboard.  However, when I make a submission and when I predict on data after 2021-06-13 I get different results.  I don't know if I made a mistake when creating the notebook submission or if this was just a general guideline.  Does this occur for other people.  In my situation, my score dropped by 0.01</p>",
  "messages": [
    {
      "id": "1652513",
      "postDate": "01/16/2022 17:42:11",
      "content": "<p>I have seen people claim \"2021-06-13 00:00:00\" is the start of the testing set for the leaderboard.  However, when I make a submission and when I predict on data after 2021-06-13 I get different results.  I don't know if I made a mistake when creating the notebook submission or if this was just a general guideline.  Does this occur for other people.  In my situation, my score dropped by 0.01</p>",
      "rawMarkdown": "I have seen people claim \"2021-06-13 00:00:00\" is the start of the testing set for the leaderboard.  However, when I make a submission and when I predict on data after 2021-06-13 I get different results.  I don't know if I made a mistake when creating the notebook submission or if this was just a general guideline.  Does this occur for other people.  In my situation, my score dropped by 0.01",
      "votes": null
    },
    {
      "id": "1652638",
      "postDate": "01/16/2022 20:15:10",
      "content": "<p>Also find that scores differ, suspect that they might have some additional data that they are using at the that is not in the training set?</p>",
      "rawMarkdown": "Also find that scores differ, suspect that they might have some additional data that they are using at the that is not in the training set?",
      "votes": null
    },
    {
      "id": "1654867",
      "postDate": "01/18/2022 22:20:06",
      "content": "<p>My score still differs by a bit, but I realized I never ran .eval on my pytorch model which caused the model to decrease in score.</p>",
      "rawMarkdown": "My score still differs by a bit, but I realized I never ran .eval on my pytorch model which caused the model to decrease in score.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 1652638,
      "author_name": "jonnydossantos",
      "author_url": "",
      "post_date": "01/16/2022 20:15:10",
      "content": "<p>Also find that scores differ, suspect that they might have some additional data that they are using at the that is not in the training set?</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 1654867,
      "author_name": "petersk20",
      "author_url": "",
      "post_date": "01/18/2022 22:20:06",
      "content": "<p>My score still differs by a bit, but I realized I never ran .eval on my pytorch model which caused the model to decrease in score.</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "1652513": "I have seen people claim \"2021-06-13 00:00:00\" is the start of the testing set for the leaderboard.  However, when I make a submission and when I predict on data after 2021-06-13 I get different results.  I don't know if I made a mistake when creating the notebook submission or if this was just a general guideline.  Does this occur for other people.  In my situation, my score dropped by 0.01",
    "1652638": "Also find that scores differ, suspect that they might have some additional data that they are using at the that is not in the training set?",
    "1654867": "My score still differs by a bit, but I realized I never ran .eval on my pytorch model which caused the model to decrease in score."
  },
  "source": "meta"
}