{
  "id": 292537,
  "title": "First chunk ignored during evaluation",
  "url": "/competitions/g-research-crypto-forecasting/discussion/292537",
  "author_name": "",
  "post_date": "2021-12-02T13:46:36.701843500Z",
  "votes": 10,
  "comment_count": 4,
  "views": 0,
  "content": "<p>I just found out something in this notebook <a href=\"https://www.kaggle.com/swaralipibose/detailed-api-introduction-300351?scriptVersionId=81353200\" target=\"_blank\">https://www.kaggle.com/swaralipibose/detailed-api-introduction-300351?scriptVersionId=81353200</a> with a naive submission, in version 1 the first-minute predictions are replaced with 0 submission files while others are kept with actual predictions but in version 2 we do not replace with 0 and the score is the same!</p>",
  "messages": [
    {
      "id": "1603394",
      "postDate": "12/02/2021 13:46:36",
      "content": "<p>I just found out something in this notebook <a href=\"https://www.kaggle.com/swaralipibose/detailed-api-introduction-300351?scriptVersionId=81353200\" target=\"_blank\">https://www.kaggle.com/swaralipibose/detailed-api-introduction-300351?scriptVersionId=81353200</a> with a naive submission, in version 1 the first-minute predictions are replaced with 0 submission files while others are kept with actual predictions but in version 2 we do not replace with 0 and the score is the same!</p>",
      "rawMarkdown": "I just found out something in this notebook https://www.kaggle.com/swaralipibose/detailed-api-introduction-300351?scriptVersionId=81353200 with a naive submission, in version 1 the first-minute predictions are replaced with 0 submission files while others are kept with actual predictions but in version 2 we do not replace with 0 and the score is the same!",
      "votes": null
    },
    {
      "id": "1603552",
      "postDate": "12/02/2021 15:11:15",
      "content": "<p>Have you estimated the effect your change should make? Given the metric (weighted correlation) and that the test set spans about 130,000 (if I'm not mistaken) minutes I would naïvely have guessed that making a reasonably small change to the observations for one minute only wouldn't result in a change within the precision of the leaderboard.</p>",
      "rawMarkdown": "Have you estimated the effect your change should make? Given the metric (weighted correlation) and that the test set spans about 130,000 (if I'm not mistaken) minutes I would naïvely have guessed that making a reasonably small change to the observations for one minute only wouldn't result in a change within the precision of the leaderboard.",
      "votes": null
    },
    {
      "id": "1604013",
      "postDate": "12/03/2021 03:12:26",
      "content": "<p>There are only 56 rows in public lb 14 such rows should have a change how did you come up with 130,000  , see submission.csv of any kernel </p>",
      "rawMarkdown": "There are only 56 rows in public lb 14 such rows should have a change how did you come up with 130,000  , see submission.csv of any kernel",
      "votes": null
    },
    {
      "id": "1608714",
      "postDate": "12/06/2021 15:24:28",
      "content": "<p>There are more rows for the public LB score, they are just fed into our models only on prediction. this is used to make sure people won't use the test.csv in any way. [On this competition it doesn't matter anyway since we have the public test dataset in our training set anyway]</p>",
      "rawMarkdown": "There are more rows for the public LB score, they are just fed into our models only on prediction. this is used to make sure people won't use the test.csv in any way. [On this competition it doesn't matter anyway since we have the public test dataset in our training set anyway]",
      "votes": null
    },
    {
      "id": "1610545",
      "postDate": "12/07/2021 10:04:26",
      "content": "<p><code>Expect to see roughly three months worth of data in the test set.</code> <a href=\"https://www.kaggle.com/yamqwe\" target=\"_blank\">@yamqwe</a> if you meant this line then my question is how are you sure that they mean public lb data and not private leaderboard data they have just said test set ?</p>",
      "rawMarkdown": "``` Expect to see roughly three months worth of data in the test set. ``` @yamqwe if you meant this line then my question is how are you sure that they mean public lb data and not private leaderboard data they have just said test set ?",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 1603552,
      "author_name": "christoffer",
      "author_url": "",
      "post_date": "12/02/2021 15:11:15",
      "content": "<p>Have you estimated the effect your change should make? Given the metric (weighted correlation) and that the test set spans about 130,000 (if I'm not mistaken) minutes I would naïvely have guessed that making a reasonably small change to the observations for one minute only wouldn't result in a change within the precision of the leaderboard.</p>",
      "votes": null,
      "replies": [
        {
          "id": 1604013,
          "author_name": "swaralipibose",
          "author_url": "",
          "post_date": "12/03/2021 03:12:26",
          "content": "<p>There are only 56 rows in public lb 14 such rows should have a change how did you come up with 130,000  , see submission.csv of any kernel </p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1608714,
          "author_name": "",
          "author_url": "",
          "post_date": "12/06/2021 15:24:28",
          "content": "<p>There are more rows for the public LB score, they are just fed into our models only on prediction. this is used to make sure people won't use the test.csv in any way. [On this competition it doesn't matter anyway since we have the public test dataset in our training set anyway]</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1610545,
          "author_name": "swaralipibose",
          "author_url": "",
          "post_date": "12/07/2021 10:04:26",
          "content": "<p><code>Expect to see roughly three months worth of data in the test set.</code> <a href=\"https://www.kaggle.com/yamqwe\" target=\"_blank\">@yamqwe</a> if you meant this line then my question is how are you sure that they mean public lb data and not private leaderboard data they have just said test set ?</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "1603394": "I just found out something in this notebook https://www.kaggle.com/swaralipibose/detailed-api-introduction-300351?scriptVersionId=81353200 with a naive submission, in version 1 the first-minute predictions are replaced with 0 submission files while others are kept with actual predictions but in version 2 we do not replace with 0 and the score is the same!",
    "1603552": "Have you estimated the effect your change should make? Given the metric (weighted correlation) and that the test set spans about 130,000 (if I'm not mistaken) minutes I would naïvely have guessed that making a reasonably small change to the observations for one minute only wouldn't result in a change within the precision of the leaderboard.",
    "1604013": "There are only 56 rows in public lb 14 such rows should have a change how did you come up with 130,000  , see submission.csv of any kernel",
    "1608714": "There are more rows for the public LB score, they are just fed into our models only on prediction. this is used to make sure people won't use the test.csv in any way. [On this competition it doesn't matter anyway since we have the public test dataset in our training set anyway]",
    "1610545": "``` Expect to see roughly three months worth of data in the test set. ``` @yamqwe if you meant this line then my question is how are you sure that they mean public lb data and not private leaderboard data they have just said test set ?"
  },
  "source": "meta"
}