{
  "id": 247842,
  "title": "Why public leaderboard score on only 12% of data?",
  "url": "/competitions/coleridgeinitiative-show-us-the-data/discussion/247842",
  "author_name": "",
  "post_date": "2021-06-21T11:12:19.891873200Z",
  "votes": 1,
  "comment_count": 1,
  "views": 0,
  "content": "<p>Can anyone help me understand the logic behind having only 12% data for the public leaderboard?</p>\n<p>I understand that excessive tuning of a model based on the test dataset/public leaderboard score can lead to overfitting to the testing set and hence a sub-par model performance but then there has to be a certain limit to the breakup. </p>\n<p>I think the scores are going to be highly varied on the private leaderboard. The idea behind having a public and private leaderboard is to not make an overfitted model but then it's also that the score on the public board is somewhat representative of the score on the private board, in my opinion. I am not sure if the split here serves that purpose.</p>\n<p>I also think that the organizers are not only aiming that the participants not make overfitted models, there is more to that for keeping just 12% data. It's the first time that I have come across this weird split.</p>\n<p>Let me know your thoughts.</p>",
  "messages": [
    {
      "id": "1359561",
      "postDate": "06/21/2021 11:12:19",
      "content": "<p>Can anyone help me understand the logic behind having only 12% data for the public leaderboard?</p>\n<p>I understand that excessive tuning of a model based on the test dataset/public leaderboard score can lead to overfitting to the testing set and hence a sub-par model performance but then there has to be a certain limit to the breakup. </p>\n<p>I think the scores are going to be highly varied on the private leaderboard. The idea behind having a public and private leaderboard is to not make an overfitted model but then it's also that the score on the public board is somewhat representative of the score on the private board, in my opinion. I am not sure if the split here serves that purpose.</p>\n<p>I also think that the organizers are not only aiming that the participants not make overfitted models, there is more to that for keeping just 12% data. It's the first time that I have come across this weird split.</p>\n<p>Let me know your thoughts.</p>",
      "rawMarkdown": "Can anyone help me understand the logic behind having only 12% data for the public leaderboard?\n\nI understand that excessive tuning of a model based on the test dataset/public leaderboard score can lead to overfitting to the testing set and hence a sub-par model performance but then there has to be a certain limit to the breakup. \n\nI think the scores are going to be highly varied on the private leaderboard. The idea behind having a public and private leaderboard is to not make an overfitted model but then it's also that the score on the public board is somewhat representative of the score on the private board, in my opinion. I am not sure if the split here serves that purpose.\n\nI also think that the organizers are not only aiming that the participants not make overfitted models, there is more to that for keeping just 12% data. It's the first time that I have come across this weird split.\n\nLet me know your thoughts.",
      "votes": null
    },
    {
      "id": "1359586",
      "postDate": "06/21/2021 11:28:47",
      "content": "<p>just give me unlimited time and unlimited submission chances, if the public lb uses 100% of the test data set, I can make the model prediction 100% accurate</p>",
      "rawMarkdown": "just give me unlimited time and unlimited submission chances, if the public lb uses 100% of the test data set, I can make the model prediction 100% accurate",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 1359586,
      "author_name": "yus002",
      "author_url": "",
      "post_date": "06/21/2021 11:28:47",
      "content": "<p>just give me unlimited time and unlimited submission chances, if the public lb uses 100% of the test data set, I can make the model prediction 100% accurate</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "1359561": "Can anyone help me understand the logic behind having only 12% data for the public leaderboard?\n\nI understand that excessive tuning of a model based on the test dataset/public leaderboard score can lead to overfitting to the testing set and hence a sub-par model performance but then there has to be a certain limit to the breakup. \n\nI think the scores are going to be highly varied on the private leaderboard. The idea behind having a public and private leaderboard is to not make an overfitted model but then it's also that the score on the public board is somewhat representative of the score on the private board, in my opinion. I am not sure if the split here serves that purpose.\n\nI also think that the organizers are not only aiming that the participants not make overfitted models, there is more to that for keeping just 12% data. It's the first time that I have come across this weird split.\n\nLet me know your thoughts.",
    "1359586": "just give me unlimited time and unlimited submission chances, if the public lb uses 100% of the test data set, I can make the model prediction 100% accurate"
  },
  "source": "meta"
}