{
  "id": 670318,
  "title": "What should I submit?",
  "url": "/competitions/csiro-biomass/discussion/670318",
  "author_name": "Komil Parmar",
  "post_date": "2026-01-27T14:05:35.726000",
  "votes": -1,
  "comment_count": 4,
  "views": 0,
  "content": "<p>Hi everyone,</p>\n<p>I’m a bit unsure which models to trust for the final submission and would really appreciate some advice from others tracking local vs LB behaviour.</p>\n<p>Currently I have an ensemble of models, each getting around 0.75/0.74 on the LB that I’ve shortlisted for final evaluation. Besides these, I also have a few more experimental models with an interesting pattern: my models only start diverging from a near-perfect correlation between local R2 and LB after about 0.72. Up to 0.72, my out-of-fold R2 and LB scores are essentially the same, and I once managed to keep this alignment at 0.73 as well, but not beyond that.</p>\n<p>Past this point, models that show higher local R2 scores start to look overfitted, even though they can sometimes score slightly better on the LB (0.74–0.75). Given this behaviour, I’m not sure whether I should trust these “overfitted but higher LB” models for the final evaluation, or stick to the models where local and LB remain tightly correlated (even if the LB is a bit lower).</p>\n<p>For those who have gone through similar patterns in this competition (or others with noisy targets / small LB), how do you decide what to submit in the final round? Do you prioritize:</p>\n<ul>\n<li>Models/ensembles where OOF and LB track closely, even at the cost of a slightly lower peak LB.</li>\n<li>Models that squeeze out the best LB so far, despite signs of overfitting locally.</li>\n</ul>\n<p>Any insights or heuristics you use for this trade-off (e.g., ensembling “stable” and “spiky” models together, looking at variance across CV folds, etc.) would be super helpful.</p>\n<p>Best of Luck for the competition to everyone.</p>",
  "messages": [
    {
      "id": 3397849,
      "postDate": "2026-01-28T06:18:50.887Z",
      "content": "<p>One for stable conditions, and one for spiky ones, I think. Good luck.</p>",
      "rawMarkdown": "One for stable conditions, and one for spiky ones, I think. Good luck.",
      "votes": 1,
      "replies": [
        {
          "id": 3397875,
          "postDate": "2026-01-28T07:13:01.003Z",
          "content": "<p>Indeed, I agree with your idea as well. I was wondering whether you choose the submission version based on whether post-processing is applied or not?</p>",
          "rawMarkdown": "Indeed, I agree with your idea as well. I was wondering whether you choose the submission version based on whether post-processing is applied or not?",
          "replies": [
            {
              "id": 3397988,
              "postDate": "2026-01-28T11:54:33.327Z",
              "rawMarkdown": "",
              "isDeleted": true
            }
          ]
        }
      ]
    },
    {
      "id": 3397788,
      "postDate": "2026-01-28T04:08:01.700Z",
      "content": "<p>I like to do one risky and one safe.</p>",
      "rawMarkdown": "I like to do one risky and one safe.",
      "votes": 1
    },
    {
      "id": 3397530,
      "postDate": "2026-01-27T14:05:35.727Z",
      "content": "<p>Hi everyone,</p>\n<p>I’m a bit unsure which models to trust for the final submission and would really appreciate some advice from others tracking local vs LB behaviour.</p>\n<p>Currently I have an ensemble of models, each getting around 0.75/0.74 on the LB that I’ve shortlisted for final evaluation. Besides these, I also have a few more experimental models with an interesting pattern: my models only start diverging from a near-perfect correlation between local R2 and LB after about 0.72. Up to 0.72, my out-of-fold R2 and LB scores are essentially the same, and I once managed to keep this alignment at 0.73 as well, but not beyond that.</p>\n<p>Past this point, models that show higher local R2 scores start to look overfitted, even though they can sometimes score slightly better on the LB (0.74–0.75). Given this behaviour, I’m not sure whether I should trust these “overfitted but higher LB” models for the final evaluation, or stick to the models where local and LB remain tightly correlated (even if the LB is a bit lower).</p>\n<p>For those who have gone through similar patterns in this competition (or others with noisy targets / small LB), how do you decide what to submit in the final round? Do you prioritize:</p>\n<ul>\n<li>Models/ensembles where OOF and LB track closely, even at the cost of a slightly lower peak LB.</li>\n<li>Models that squeeze out the best LB so far, despite signs of overfitting locally.</li>\n</ul>\n<p>Any insights or heuristics you use for this trade-off (e.g., ensembling “stable” and “spiky” models together, looking at variance across CV folds, etc.) would be super helpful.</p>\n<p>Best of Luck for the competition to everyone.</p>",
      "rawMarkdown": "Hi everyone,\n\nI’m a bit unsure which models to trust for the final submission and would really appreciate some advice from others tracking local vs LB behaviour.\n\nCurrently I have an ensemble of models, each getting around 0.75/0.74 on the LB that I’ve shortlisted for final evaluation. Besides these, I also have a few more experimental models with an interesting pattern: my models only start diverging from a near-perfect correlation between local R2 and LB after about 0.72. Up to 0.72, my out-of-fold R2 and LB scores are essentially the same, and I once managed to keep this alignment at 0.73 as well, but not beyond that.\n\nPast this point, models that show higher local R2 scores start to look overfitted, even though they can sometimes score slightly better on the LB (0.74–0.75). Given this behaviour, I’m not sure whether I should trust these “overfitted but higher LB” models for the final evaluation, or stick to the models where local and LB remain tightly correlated (even if the LB is a bit lower).\n\nFor those who have gone through similar patterns in this competition (or others with noisy targets / small LB), how do you decide what to submit in the final round? Do you prioritize:\n\n- Models/ensembles where OOF and LB track closely, even at the cost of a slightly lower peak LB.\n- Models that squeeze out the best LB so far, despite signs of overfitting locally.\n\nAny insights or heuristics you use for this trade-off (e.g., ensembling “stable” and “spiky” models together, looking at variance across CV folds, etc.) would be super helpful.\n\nBest of Luck for the competition to everyone.",
      "votes": -1
    }
  ],
  "comments": [
    {
      "id": 3397849,
      "author_name": "Cho Royou",
      "author_url": "",
      "post_date": "2026-01-28T06:18:50.887000",
      "content": "<p>One for stable conditions, and one for spiky ones, I think. Good luck.</p>",
      "votes": 1,
      "replies": [
        {
          "id": 3397875,
          "author_name": "Chunyi Li",
          "author_url": "",
          "post_date": "2026-01-28T07:13:01.003000",
          "content": "<p>Indeed, I agree with your idea as well. I was wondering whether you choose the submission version based on whether post-processing is applied or not?</p>",
          "votes": 0,
          "replies": [
            {
              "id": 3397988,
              "author_name": "",
              "author_url": "",
              "post_date": "2026-01-28T11:54:33.327000",
              "content": "",
              "votes": 0,
              "replies": []
            }
          ]
        }
      ]
    },
    {
      "id": 3397788,
      "author_name": "PC Jimmmy",
      "author_url": "",
      "post_date": "2026-01-28T04:08:01.700000",
      "content": "<p>I like to do one risky and one safe.</p>",
      "votes": 1,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "3397849": "One for stable conditions, and one for spiky ones, I think. Good luck.",
    "3397788": "I like to do one risky and one safe.",
    "3397530": "Hi everyone,\n\nI’m a bit unsure which models to trust for the final submission and would really appreciate some advice from others tracking local vs LB behaviour.\n\nCurrently I have an ensemble of models, each getting around 0.75/0.74 on the LB that I’ve shortlisted for final evaluation. Besides these, I also have a few more experimental models with an interesting pattern: my models only start diverging from a near-perfect correlation between local R2 and LB after about 0.72. Up to 0.72, my out-of-fold R2 and LB scores are essentially the same, and I once managed to keep this alignment at 0.73 as well, but not beyond that.\n\nPast this point, models that show higher local R2 scores start to look overfitted, even though they can sometimes score slightly better on the LB (0.74–0.75). Given this behaviour, I’m not sure whether I should trust these “overfitted but higher LB” models for the final evaluation, or stick to the models where local and LB remain tightly correlated (even if the LB is a bit lower).\n\nFor those who have gone through similar patterns in this competition (or others with noisy targets / small LB), how do you decide what to submit in the final round? Do you prioritize:\n\n- Models/ensembles where OOF and LB track closely, even at the cost of a slightly lower peak LB.\n- Models that squeeze out the best LB so far, despite signs of overfitting locally.\n\nAny insights or heuristics you use for this trade-off (e.g., ensembling “stable” and “spiky” models together, looking at variance across CV folds, etc.) would be super helpful.\n\nBest of Luck for the competition to everyone."
  }
}