{
  "id": 336138,
  "title": "Relationship between train data, public LB test data, and private LB test data",
  "url": "/competitions/AI4Code/discussion/336138",
  "author_name": "Apolo",
  "post_date": "2022-07-09T14:34:54.147000",
  "votes": 7,
  "comment_count": 0,
  "views": 0,
  "content": "<p>Hi, I have recently started this competition and find it very interesting, but I have a concern regarding the relationship between train data, public LB test data, and private LB test data.</p>\n<p>Looking at the data description, there is a clear statement regarding private LB test data, and it is stated that private LB test data are kaggle notebooks created between the final submission deadline (August 11, 2022) and the end of the competition (November 10, 2022).</p>\n<blockquote>\n  <ul>\n  <li>The first-stage test set contains notebooks from an approximately 90-day historical window of time.</li>\n  <li>The second-stage test set will contain a similar number of notebooks, collected from a future 90-day window of time. This is necessary to prevent models from looking up the order of existing public notebooks. The selection criteria for second-stage notebooks will be monitored for competition fairness purposes. For example, it will exclude competition participants' own notebooks.</li>\n  </ul>\n</blockquote>\n<p>On the other hand, the period of data acquisition for train data and public LB test data is not clearly stated, so there is a concern that the different nature of train data, public LB test data, and private LB test data may cause severe shake ups. </p>\n<p>The acquisition period of the kaggle notebooks will change which competition's notebooks they are.<br>\nIn my view, not only the participants but also the code and markdown that is likely to be written will differ from competition to competition.</p>\n<p>So, I would like to use <code>competition</code>  instead of <code>ancestor_id</code> as a grouping factor when constructing validation splits, but that kind of information is unknown.</p>\n<p>If train data and public LB test data are the notebooks created in different time periods at regular intervals, I can trust public LB.</p>\n<p>Is there any information from the host regarding this?<br>\nWhat are your thoughts on this?</p>",
  "messages": [
    {
      "id": 1849461,
      "postDate": "2022-07-09T14:34:54.147Z",
      "content": "<p>Hi, I have recently started this competition and find it very interesting, but I have a concern regarding the relationship between train data, public LB test data, and private LB test data.</p>\n<p>Looking at the data description, there is a clear statement regarding private LB test data, and it is stated that private LB test data are kaggle notebooks created between the final submission deadline (August 11, 2022) and the end of the competition (November 10, 2022).</p>\n<blockquote>\n  <ul>\n  <li>The first-stage test set contains notebooks from an approximately 90-day historical window of time.</li>\n  <li>The second-stage test set will contain a similar number of notebooks, collected from a future 90-day window of time. This is necessary to prevent models from looking up the order of existing public notebooks. The selection criteria for second-stage notebooks will be monitored for competition fairness purposes. For example, it will exclude competition participants' own notebooks.</li>\n  </ul>\n</blockquote>\n<p>On the other hand, the period of data acquisition for train data and public LB test data is not clearly stated, so there is a concern that the different nature of train data, public LB test data, and private LB test data may cause severe shake ups. </p>\n<p>The acquisition period of the kaggle notebooks will change which competition's notebooks they are.<br>\nIn my view, not only the participants but also the code and markdown that is likely to be written will differ from competition to competition.</p>\n<p>So, I would like to use <code>competition</code>  instead of <code>ancestor_id</code> as a grouping factor when constructing validation splits, but that kind of information is unknown.</p>\n<p>If train data and public LB test data are the notebooks created in different time periods at regular intervals, I can trust public LB.</p>\n<p>Is there any information from the host regarding this?<br>\nWhat are your thoughts on this?</p>",
      "rawMarkdown": "Hi, I have recently started this competition and find it very interesting, but I have a concern regarding the relationship between train data, public LB test data, and private LB test data.\n\nLooking at the data description, there is a clear statement regarding private LB test data, and it is stated that private LB test data are kaggle notebooks created between the final submission deadline (August 11, 2022) and the end of the competition (November 10, 2022).\n\n> - The first-stage test set contains notebooks from an approximately 90-day historical window of time.\n- The second-stage test set will contain a similar number of notebooks, collected from a future 90-day window of time. This is necessary to prevent models from looking up the order of existing public notebooks. The selection criteria for second-stage notebooks will be monitored for competition fairness purposes. For example, it will exclude competition participants' own notebooks.\n\nOn the other hand, the period of data acquisition for train data and public LB test data is not clearly stated, so there is a concern that the different nature of train data, public LB test data, and private LB test data may cause severe shake ups. \n\nThe acquisition period of the kaggle notebooks will change which competition's notebooks they are.\nIn my view, not only the participants but also the code and markdown that is likely to be written will differ from competition to competition.\n\nSo, I would like to use `competition`  instead of `ancestor_id` as a grouping factor when constructing validation splits, but that kind of information is unknown.\n\nIf train data and public LB test data are the notebooks created in different time periods at regular intervals, I can trust public LB.\n\nIs there any information from the host regarding this?\nWhat are your thoughts on this?",
      "votes": 7
    }
  ],
  "comments": [],
  "raw_markdown_by_id": {
    "1849461": "Hi, I have recently started this competition and find it very interesting, but I have a concern regarding the relationship between train data, public LB test data, and private LB test data.\n\nLooking at the data description, there is a clear statement regarding private LB test data, and it is stated that private LB test data are kaggle notebooks created between the final submission deadline (August 11, 2022) and the end of the competition (November 10, 2022).\n\n> - The first-stage test set contains notebooks from an approximately 90-day historical window of time.\n- The second-stage test set will contain a similar number of notebooks, collected from a future 90-day window of time. This is necessary to prevent models from looking up the order of existing public notebooks. The selection criteria for second-stage notebooks will be monitored for competition fairness purposes. For example, it will exclude competition participants' own notebooks.\n\nOn the other hand, the period of data acquisition for train data and public LB test data is not clearly stated, so there is a concern that the different nature of train data, public LB test data, and private LB test data may cause severe shake ups. \n\nThe acquisition period of the kaggle notebooks will change which competition's notebooks they are.\nIn my view, not only the participants but also the code and markdown that is likely to be written will differ from competition to competition.\n\nSo, I would like to use `competition`  instead of `ancestor_id` as a grouping factor when constructing validation splits, but that kind of information is unknown.\n\nIf train data and public LB test data are the notebooks created in different time periods at regular intervals, I can trust public LB.\n\nIs there any information from the host regarding this?\nWhat are your thoughts on this?"
  }
}