{
  "id": 121945,
  "title": "Overfitting on public leaderboard ",
  "url": "/competitions/Kannada-MNIST/discussion/121945",
  "author_name": "",
  "post_date": "2019-12-16T17:34:23.993726800Z",
  "votes": 1,
  "comment_count": 4,
  "views": 0,
  "content": "<p>I see people are using complex architectures to improve the score on the public leaderboard. They might be overfitting the leaderboard. I think at the end, the order in the private leaderboard will change a lot, since everybody seems to be between 97% and 99%. Will the simplest solution win ?</p>",
  "messages": [
    {
      "id": "696508",
      "postDate": "12/16/2019 17:34:23",
      "content": "<p>I see people are using complex architectures to improve the score on the public leaderboard. They might be overfitting the leaderboard. I think at the end, the order in the private leaderboard will change a lot, since everybody seems to be between 97% and 99%. Will the simplest solution win ?</p>",
      "rawMarkdown": "I see people are using complex architectures to improve the score on the public leaderboard. They might be overfitting the leaderboard. I think at the end, the order in the private leaderboard will change a lot, since everybody seems to be between 97% and 99%. Will the simplest solution win ?",
      "votes": null
    },
    {
      "id": "696540",
      "postDate": "12/16/2019 18:28:54",
      "content": "<p>will be interesting to see for sure. I was curious about that.</p>",
      "rawMarkdown": "will be interesting to see for sure. I was curious about that.",
      "votes": null
    },
    {
      "id": "696932",
      "postDate": "12/17/2019 09:04:35",
      "content": "<blockquote>\n  <p>I think at the end, the order in the private leaderboard will change a lot, since everybody seems to be between 97% and 99%. Will the simplest solution win ?</p>\n</blockquote>\n\n<p>I'm also aware of this, it may happen, but I think no matter what architecture /methods you use, cross-validation will be necessary to rank higher.</p>",
      "rawMarkdown": "&gt; I think at the end, the order in the private leaderboard will change a lot, since everybody seems to be between 97% and 99%. Will the simplest solution win ?\n\nI'm also aware of this, it may happen, but I think no matter what architecture /methods you use, cross-validation will be necessary to rank higher.",
      "votes": null
    },
    {
      "id": "697021",
      "postDate": "12/17/2019 11:30:10",
      "content": "<p>About Cross-Validation in Deep Learning, how do i make sure I keep the same weight at the start of training for each fold.</p>",
      "rawMarkdown": "About Cross-Validation in Deep Learning, how do i make sure I keep the same weight at the start of training for each fold.",
      "votes": null
    },
    {
      "id": "697801",
      "postDate": "12/18/2019 12:41:42",
      "content": "<p>You can save the weights of the initial model and reload it at the start of each fold, but I think it's okay to randomly initialize the weights.</p>",
      "rawMarkdown": "You can save the weights of the initial model and reload it at the start of each fold, but I think it's okay to randomly initialize the weights.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 696540,
      "author_name": "georgedittmar",
      "author_url": "",
      "post_date": "12/16/2019 18:28:54",
      "content": "<p>will be interesting to see for sure. I was curious about that.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 696932,
      "author_name": "ccchang801023",
      "author_url": "",
      "post_date": "12/17/2019 09:04:35",
      "content": "<blockquote>\n  <p>I think at the end, the order in the private leaderboard will change a lot, since everybody seems to be between 97% and 99%. Will the simplest solution win ?</p>\n</blockquote>\n\n<p>I'm also aware of this, it may happen, but I think no matter what architecture /methods you use, cross-validation will be necessary to rank higher.</p>",
      "votes": null,
      "replies": [
        {
          "id": 697021,
          "author_name": "kayveen",
          "author_url": "",
          "post_date": "12/17/2019 11:30:10",
          "content": "<p>About Cross-Validation in Deep Learning, how do i make sure I keep the same weight at the start of training for each fold.</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 697801,
          "author_name": "ccchang801023",
          "author_url": "",
          "post_date": "12/18/2019 12:41:42",
          "content": "<p>You can save the weights of the initial model and reload it at the start of each fold, but I think it's okay to randomly initialize the weights.</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "696508": "I see people are using complex architectures to improve the score on the public leaderboard. They might be overfitting the leaderboard. I think at the end, the order in the private leaderboard will change a lot, since everybody seems to be between 97% and 99%. Will the simplest solution win ?",
    "696540": "will be interesting to see for sure. I was curious about that.",
    "696932": "&gt; I think at the end, the order in the private leaderboard will change a lot, since everybody seems to be between 97% and 99%. Will the simplest solution win ?\n\nI'm also aware of this, it may happen, but I think no matter what architecture /methods you use, cross-validation will be necessary to rank higher.",
    "697021": "About Cross-Validation in Deep Learning, how do i make sure I keep the same weight at the start of training for each fold.",
    "697801": "You can save the weights of the initial model and reload it at the start of each fold, but I think it's okay to randomly initialize the weights."
  },
  "source": "meta"
}