{
  "id": 613724,
  "title": "Is there data drift between the training data and the test data?",
  "url": "/competitions/csiro-biomass/discussion/613724",
  "author_name": "",
  "post_date": "2025-10-29T09:12:24.802374200Z",
  "votes": 26,
  "comment_count": 4,
  "views": 0,
  "content": "<p>Thank you for organizing this wonderful competition.</p>\n<p>Does the test data contain many entries that differ from the training data (e.g., Sampling_Date or State)?</p>\n<p>I noticed a significant discrepancy between my locally performed 3-fold cross-validation evaluation and the leaderboard evaluation, which made me curious. \n[CV Score: 0.54~0.58, LB Score: 0.44], [fold2 Validation Score: 0.542, fold2 LB Score: 0.36]</p>\n<p>Also, is the data split between public and private sets random?</p>",
  "messages": [
    {
      "id": "3308371",
      "postDate": "10/29/2025 09:12:24",
      "content": "<p>Thank you for organizing this wonderful competition.</p>\n<p>Does the test data contain many entries that differ from the training data (e.g., Sampling_Date or State)?</p>\n<p>I noticed a significant discrepancy between my locally performed 3-fold cross-validation evaluation and the leaderboard evaluation, which made me curious. \n[CV Score: 0.54~0.58, LB Score: 0.44], [fold2 Validation Score: 0.542, fold2 LB Score: 0.36]</p>\n<p>Also, is the data split between public and private sets random?</p>",
      "rawMarkdown": "Thank you for organizing this wonderful competition.\n\nDoes the test data contain many entries that differ from the training data (e.g., Sampling_Date or State)?\n\nI noticed a significant discrepancy between my locally performed 3-fold cross-validation evaluation and the leaderboard evaluation, which made me curious. \n[CV Score: 0.54~0.58, LB Score: 0.44], [fold2 Validation Score: 0.542, fold2 LB Score: 0.36]\n\nAlso, is the data split between public and private sets random?",
      "votes": null
    },
    {
      "id": "3308666",
      "postDate": "10/30/2025 00:00:50",
      "content": "<p>The data split between the public and private sets is not completely random.</p>\n<p>We used data from various seasons and states throughout the year as the training set. For validation and testing, we included some data from overlapping time periods and locations, while also incorporating data from non-overlapping time periods to evaluate the model's generalization ability.</p>",
      "rawMarkdown": "The data split between the public and private sets is not completely random.\n\nWe used data from various seasons and states throughout the year as the training set. For validation and testing, we included some data from overlapping time periods and locations, while also incorporating data from non-overlapping time periods to evaluate the model's generalization ability.",
      "votes": null
    },
    {
      "id": "3308677",
      "postDate": "10/30/2025 01:59:39",
      "content": "<blockquote>\n  <p>Of the initial collection of 3,187 samples, 1,162 passed the comprehensive quality control process. </p>\n</blockquote>\n<p>The key point of this competition lies here: the quantity ratio of the training set you provided, the public leaderboard test set, and the private leaderboard test set is close to 1:1:1, with none exceeding 400 samples. Given such a small amount of data, the public leaderboard's reference value becomes extremely important. This is because, with so little data, the validation performance of each fold in cross-validation is not very representative.</p>",
      "rawMarkdown": ">Of the initial collection of 3,187 samples, 1,162 passed the comprehensive quality control process. \n\nThe key point of this competition lies here: the quantity ratio of the training set you provided, the public leaderboard test set, and the private leaderboard test set is close to 1:1:1, with none exceeding 400 samples. Given such a small amount of data, the public leaderboard's reference value becomes extremely important. This is because, with so little data, the validation performance of each fold in cross-validation is not very representative.",
      "votes": null
    },
    {
      "id": "3308678",
      "postDate": "10/30/2025 02:05:18",
      "content": "<p>True, deliberately set different distributions with only a few training samples, this could just be a lottery competition.</p>",
      "rawMarkdown": "True, deliberately set different distributions with only a few training samples, this could just be a lottery competition.",
      "votes": null
    },
    {
      "id": "3308688",
      "postDate": "10/30/2025 03:18:58",
      "content": "<p>The training data covers the universal biomass distribution across all seasons and locations, making it representative. The test data internally should be unbiased.</p>",
      "rawMarkdown": "The training data covers the universal biomass distribution across all seasons and locations, making it representative. The test data internally should be unbiased.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 3308666,
      "author_name": "qiyuliaocsiro",
      "author_url": "",
      "post_date": "10/30/2025 00:00:50",
      "content": "<p>The data split between the public and private sets is not completely random.</p>\n<p>We used data from various seasons and states throughout the year as the training set. For validation and testing, we included some data from overlapping time periods and locations, while also incorporating data from non-overlapping time periods to evaluate the model's generalization ability.</p>",
      "votes": null,
      "replies": [
        {
          "id": 3308677,
          "author_name": "sweetyheehee",
          "author_url": "",
          "post_date": "10/30/2025 01:59:39",
          "content": "<blockquote>\n  <p>Of the initial collection of 3,187 samples, 1,162 passed the comprehensive quality control process. </p>\n</blockquote>\n<p>The key point of this competition lies here: the quantity ratio of the training set you provided, the public leaderboard test set, and the private leaderboard test set is close to 1:1:1, with none exceeding 400 samples. Given such a small amount of data, the public leaderboard's reference value becomes extremely important. This is because, with so little data, the validation performance of each fold in cross-validation is not very representative.</p>",
          "votes": null,
          "replies": [
            {
              "id": 3308678,
              "author_name": "lizhecheng",
              "author_url": "",
              "post_date": "10/30/2025 02:05:18",
              "content": "<p>True, deliberately set different distributions with only a few training samples, this could just be a lottery competition.</p>",
              "votes": null,
              "replies": []
            },
            {
              "id": 3308688,
              "author_name": "qiyuliaocsiro",
              "author_url": "",
              "post_date": "10/30/2025 03:18:58",
              "content": "<p>The training data covers the universal biomass distribution across all seasons and locations, making it representative. The test data internally should be unbiased.</p>",
              "votes": null,
              "replies": []
            }
          ]
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "3308371": "Thank you for organizing this wonderful competition.\n\nDoes the test data contain many entries that differ from the training data (e.g., Sampling_Date or State)?\n\nI noticed a significant discrepancy between my locally performed 3-fold cross-validation evaluation and the leaderboard evaluation, which made me curious. \n[CV Score: 0.54~0.58, LB Score: 0.44], [fold2 Validation Score: 0.542, fold2 LB Score: 0.36]\n\nAlso, is the data split between public and private sets random?",
    "3308666": "The data split between the public and private sets is not completely random.\n\nWe used data from various seasons and states throughout the year as the training set. For validation and testing, we included some data from overlapping time periods and locations, while also incorporating data from non-overlapping time periods to evaluate the model's generalization ability.",
    "3308677": ">Of the initial collection of 3,187 samples, 1,162 passed the comprehensive quality control process. \n\nThe key point of this competition lies here: the quantity ratio of the training set you provided, the public leaderboard test set, and the private leaderboard test set is close to 1:1:1, with none exceeding 400 samples. Given such a small amount of data, the public leaderboard's reference value becomes extremely important. This is because, with so little data, the validation performance of each fold in cross-validation is not very representative.",
    "3308678": "True, deliberately set different distributions with only a few training samples, this could just be a lottery competition.",
    "3308688": "The training data covers the universal biomass distribution across all seasons and locations, making it representative. The test data internally should be unbiased."
  },
  "source": "meta"
}