{
  "id": 671053,
  "title": "10th Place Solution",
  "url": "/competitions/csiro-biomass/writeups/10th-place-solution",
  "author_name": "",
  "post_date": "2026-01-30T23:34:59.573Z",
  "votes": 14,
  "comment_count": 5,
  "views": 0,
  "content": "<p>We would first like to express our gratitude to the competition host and the Kaggle staff for organizing this outstanding competition.\nBelow, we introduce the my solution</p>\n<h1>Solution</h1>\n<table>\n<thead>\n<tr>\n<th>Item</th>\n<th>Description</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>Model</td>\n<td>DINOv3 ViT-L/16 distilled (Fine-tuned: regression head + last two transformer blocks)</td>\n</tr>\n<tr>\n<td>Input</td>\n<td>Images resized to 768×1536</td>\n</tr>\n<tr>\n<td>Output</td>\n<td>5 target variables + auxiliary-loss features (Height, NDVI, and whether Species is Ryegrass (Ryegrass_Clover))</td>\n</tr>\n<tr>\n<td>Loss function</td>\n<td>Huber loss weighted the same way as the evaluation metric + auxiliary loss</td>\n</tr>\n<tr>\n<td>Preprocessing</td>\n<td>- Removed 8 samples suspected as label noise after inspection<br> - Standardized the target variables</td>\n</tr>\n<tr>\n<td>CV</td>\n<td>Grouped by State - Sampling-Date, then manually reviewed and merged similar groups. Split into 5 folds using a greedy method so that each target variable’s distribution is similar across folds.<br> CV and LB seemed correlated, so I tried to improve both.</td>\n</tr>\n<tr>\n<td>Anti-shake measures</td>\n<td>Data augmentation to reproduce unnatural artifacts present in the train set (red-dot artifacts, and a date stamp in the bottom-right corner)</td>\n</tr>\n<tr>\n<td>Ensemble</td>\n<td>Trained 3 models (with/without auxiliary losses) and averaged their predictions</td>\n</tr>\n<tr>\n<td>Other</td>\n<td>- EMA<br> - TTA (horizontal flip)</td>\n</tr>\n</tbody>\n</table>\n<h2>1. Model, Input/Output, and Ensemble</h2>\n<p>I used DINOv3 ViT-L/16 distilled. Images were resized to 768×1536 and fed into the model. The model outputs consist of the five target variables plus optional auxiliary-loss features, and the final prediction is the average of three models with different output configurations.</p>\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F14349659%2F7f8e1ff15c87b96301e346f81ac3003c%2Fmodel2.png?generation=1769785563062926&amp;alt=media\" alt=\"model\"></p>\n<table>\n<thead>\n<tr>\n<th>Pattern</th>\n<th>Output configuration</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>1</td>\n<td>5 target variables</td>\n</tr>\n<tr>\n<td>2</td>\n<td>5 target variables + Height + NDVI</td>\n</tr>\n<tr>\n<td>3</td>\n<td>5 target variables + Height + NDVI + Species (binary classification: whether it is Ryegrass (Ryegrass_Clover))</td>\n</tr>\n</tbody>\n</table>\n<h2>2. Preprocessing</h2>\n<p>For samples with State=WA and Sampling_Date=2015-08-21, the label indicates Dry_Green_g = 0. However, upon inspecting the images, there were clearly plants other than Clover present. I therefore treated these as label errors and removed the following 8 samples from the training data:</p>\n<pre><code>- ID681680726.jpg\n- ID697718693.jpg\n- ID40849327.jpg\n- ID230058600.jpg\n- ID1403107574.jpg\n- ID1337107565.jpg\n- ID1139918758.jpg\n- ID1761544403.jpg\n</code></pre>\n<p></p>\n<p>This improved the score by +0.01 on CV, +0.01 on Public, and +0.01 on Private.\n(I corrected it because I had made a mistake.)</p>\n<h2>3. Anti-shake Measures</h2>\n<p>The training data contains unnatural artifacts such as the following, so I applied data augmentation to reproduce them:</p>\n<ul>\n<li>Red-dot artifacts, likely caused by flash</li>\n<li>The capture date/time appearing in the bottom-right corner of the image</li>\n</ul>\n<p>This did not help on CV or Public, but improved the Private score by +0.01. When I tested them separately, the date/time augmentation did not improve the Public score, so this suggests the private set may contain some red-dot artifacts.</p>\n<h2>4. Tried but Not Effective</h2>\n<ul>\n<li>Manifold Mixup</li>\n<li>Tuning all layers with LoRA</li>\n<li>Pseudo-labeling using data that was not used for training</li>\n<li>Using patch tokens (maybe I just used it incorrectly?)</li>\n<li>Using PlantCLEF2024 data / a pre-trained model (DINOv2)</li>\n</ul>\n<h1>In Closing</h1>\n<p>If you have any questions, feel free to ask. \nThank you for reading!</p>",
  "messages": [
    {
      "id": "3399501",
      "postDate": "01/30/2026 23:34:48",
      "content": "<p>We would first like to express our gratitude to the competition host and the Kaggle staff for organizing this outstanding competition.\nBelow, we introduce the my solution</p>\n<h1>Solution</h1>\n<table>\n<thead>\n<tr>\n<th>Item</th>\n<th>Description</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>Model</td>\n<td>DINOv3 ViT-L/16 distilled (Fine-tuned: regression head + last two transformer blocks)</td>\n</tr>\n<tr>\n<td>Input</td>\n<td>Images resized to 768×1536</td>\n</tr>\n<tr>\n<td>Output</td>\n<td>5 target variables + auxiliary-loss features (Height, NDVI, and whether Species is Ryegrass (Ryegrass_Clover))</td>\n</tr>\n<tr>\n<td>Loss function</td>\n<td>Huber loss weighted the same way as the evaluation metric + auxiliary loss</td>\n</tr>\n<tr>\n<td>Preprocessing</td>\n<td>- Removed 8 samples suspected as label noise after inspection<br> - Standardized the target variables</td>\n</tr>\n<tr>\n<td>CV</td>\n<td>Grouped by State - Sampling-Date, then manually reviewed and merged similar groups. Split into 5 folds using a greedy method so that each target variable’s distribution is similar across folds.<br> CV and LB seemed correlated, so I tried to improve both.</td>\n</tr>\n<tr>\n<td>Anti-shake measures</td>\n<td>Data augmentation to reproduce unnatural artifacts present in the train set (red-dot artifacts, and a date stamp in the bottom-right corner)</td>\n</tr>\n<tr>\n<td>Ensemble</td>\n<td>Trained 3 models (with/without auxiliary losses) and averaged their predictions</td>\n</tr>\n<tr>\n<td>Other</td>\n<td>- EMA<br> - TTA (horizontal flip)</td>\n</tr>\n</tbody>\n</table>\n<h2>1. Model, Input/Output, and Ensemble</h2>\n<p>I used DINOv3 ViT-L/16 distilled. Images were resized to 768×1536 and fed into the model. The model outputs consist of the five target variables plus optional auxiliary-loss features, and the final prediction is the average of three models with different output configurations.</p>\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F14349659%2F7f8e1ff15c87b96301e346f81ac3003c%2Fmodel2.png?generation=1769785563062926&amp;alt=media\" alt=\"model\"></p>\n<table>\n<thead>\n<tr>\n<th>Pattern</th>\n<th>Output configuration</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>1</td>\n<td>5 target variables</td>\n</tr>\n<tr>\n<td>2</td>\n<td>5 target variables + Height + NDVI</td>\n</tr>\n<tr>\n<td>3</td>\n<td>5 target variables + Height + NDVI + Species (binary classification: whether it is Ryegrass (Ryegrass_Clover))</td>\n</tr>\n</tbody>\n</table>\n<h2>2. Preprocessing</h2>\n<p>For samples with State=WA and Sampling_Date=2015-08-21, the label indicates Dry_Green_g = 0. However, upon inspecting the images, there were clearly plants other than Clover present. I therefore treated these as label errors and removed the following 8 samples from the training data:</p>\n<pre><code>- ID681680726.jpg\n- ID697718693.jpg\n- ID40849327.jpg\n- ID230058600.jpg\n- ID1403107574.jpg\n- ID1337107565.jpg\n- ID1139918758.jpg\n- ID1761544403.jpg\n</code></pre>\n<p></p>\n<p>This improved the score by +0.01 on CV, +0.01 on Public, and +0.01 on Private.\n(I corrected it because I had made a mistake.)</p>\n<h2>3. Anti-shake Measures</h2>\n<p>The training data contains unnatural artifacts such as the following, so I applied data augmentation to reproduce them:</p>\n<ul>\n<li>Red-dot artifacts, likely caused by flash</li>\n<li>The capture date/time appearing in the bottom-right corner of the image</li>\n</ul>\n<p>This did not help on CV or Public, but improved the Private score by +0.01. When I tested them separately, the date/time augmentation did not improve the Public score, so this suggests the private set may contain some red-dot artifacts.</p>\n<h2>4. Tried but Not Effective</h2>\n<ul>\n<li>Manifold Mixup</li>\n<li>Tuning all layers with LoRA</li>\n<li>Pseudo-labeling using data that was not used for training</li>\n<li>Using patch tokens (maybe I just used it incorrectly?)</li>\n<li>Using PlantCLEF2024 data / a pre-trained model (DINOv2)</li>\n</ul>\n<h1>In Closing</h1>\n<p>If you have any questions, feel free to ask. \nThank you for reading!</p>",
      "rawMarkdown": "We would first like to express our gratitude to the competition host and the Kaggle staff for organizing this outstanding competition.\nBelow, we introduce the my solution\n\n# Solution\n\n| Item | Description |\n| --- | --- |\n| Model | DINOv3 ViT-L/16 distilled (Fine-tuned: regression head + last two transformer blocks) |\n| Input | Images resized to 768×1536 |\n| Output | 5 target variables + auxiliary-loss features (Height, NDVI, and whether Species is Ryegrass (Ryegrass_Clover)) |\n| Loss function | Huber loss weighted the same way as the evaluation metric + auxiliary loss |\n| Preprocessing | - Removed 8 samples suspected as label noise after inspection<br> - Standardized the target variables |\n| CV | Grouped by State - Sampling-Date, then manually reviewed and merged similar groups. Split into 5 folds using a greedy method so that each target variable’s distribution is similar across folds.<br> CV and LB seemed correlated, so I tried to improve both. |\n| Anti-shake measures | Data augmentation to reproduce unnatural artifacts present in the train set (red-dot artifacts, and a date stamp in the bottom-right corner) |\n| Ensemble | Trained 3 models (with/without auxiliary losses) and averaged their predictions |\n| Other | - EMA<br> - TTA (horizontal flip) |\n\n## 1. Model, Input/Output, and Ensemble\n\nI used DINOv3 ViT-L/16 distilled. Images were resized to 768×1536 and fed into the model. The model outputs consist of the five target variables plus optional auxiliary-loss features, and the final prediction is the average of three models with different output configurations.\n\n![model](https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F14349659%2F7f8e1ff15c87b96301e346f81ac3003c%2Fmodel2.png?generation=1769785563062926&alt=media)\n\n| Pattern | Output configuration |\n| --- | --- |\n| 1 | 5 target variables |\n| 2 | 5 target variables + Height + NDVI |\n| 3 | 5 target variables + Height + NDVI + Species (binary classification: whether it is Ryegrass (Ryegrass_Clover)) |\n\n## 2. Preprocessing\n\nFor samples with State=WA and Sampling_Date=2015-08-21, the label indicates Dry_Green_g = 0. However, upon inspecting the images, there were clearly plants other than Clover present. I therefore treated these as label errors and removed the following 8 samples from the training data:\n\n```\n- ID681680726.jpg\n- ID697718693.jpg\n- ID40849327.jpg\n- ID230058600.jpg\n- ID1403107574.jpg\n- ID1337107565.jpg\n- ID1139918758.jpg\n- ID1761544403.jpg\n```\n\n~~This improved the score by +0.04 on CV, +0.05 on Public, and +0.01 on Private.~~\n\n\n\n\nThis improved the score by +0.01 on CV, +0.01 on Public, and +0.01 on Private.\n(I corrected it because I had made a mistake.)\n\n\n## 3. Anti-shake Measures\n\nThe training data contains unnatural artifacts such as the following, so I applied data augmentation to reproduce them:\n\n* Red-dot artifacts, likely caused by flash\n* The capture date/time appearing in the bottom-right corner of the image\n\nThis did not help on CV or Public, but improved the Private score by +0.01. When I tested them separately, the date/time augmentation did not improve the Public score, so this suggests the private set may contain some red-dot artifacts.\n\n## 4. Tried but Not Effective\n\n* Manifold Mixup\n* Tuning all layers with LoRA\n* Pseudo-labeling using data that was not used for training\n* Using patch tokens (maybe I just used it incorrectly?)\n* Using PlantCLEF2024 data / a pre-trained model (DINOv2)\n\n\n# In Closing\n\nIf you have any questions, feel free to ask. \nThank you for reading!",
      "votes": null
    },
    {
      "id": "3399572",
      "postDate": "01/31/2026 01:45:06",
      "content": "<p>It was very clear and helpful. Thank you. Especially the model configuration diagram—it's a simple and interesting illustration.</p>",
      "rawMarkdown": "It was very clear and helpful. Thank you. Especially the model configuration diagram—it's a simple and interesting illustration.",
      "votes": null
    },
    {
      "id": "3399666",
      "postDate": "01/31/2026 05:59:52",
      "content": "<pre><code>For samples with State=WA and Sampling_Date=2015-08-21, the label indicates Dry_Green_g = 0. However, upon inspecting the images, there were clearly plants other than Clover present. I therefore treated these as label errors and removed the following 8 samples from the training data:\n</code></pre>\n<p>what was your OOF correlation between the model trained with/without these 8 samples? \nDid the same and for me was 0.99x, extremely similar…</p>",
      "rawMarkdown": "```\nFor samples with State=WA and Sampling_Date=2015-08-21, the label indicates Dry_Green_g = 0. However, upon inspecting the images, there were clearly plants other than Clover present. I therefore treated these as label errors and removed the following 8 samples from the training data:\n```\n\nwhat was your OOF correlation between the model trained with/without these 8 samples? \nDid the same and for me was 0.99x, extremely similar...",
      "votes": null
    },
    {
      "id": "3399798",
      "postDate": "01/31/2026 10:25:43",
      "content": "<p>I’m glad I could help!</p>",
      "rawMarkdown": "I’m glad I could help!",
      "votes": null
    },
    {
      "id": "3399941",
      "postDate": "01/31/2026 15:03:30",
      "content": "<p>The correlation between the predicted Dry_Total_g values from the model trained with the 8 samples and the model trained without them was 0.897. Also, I wrote in my solution that the CV score improved by 0.04, but that was a calculation mistake—the correct improvement was 0.01.</p>",
      "rawMarkdown": "The correlation between the predicted Dry_Total_g values from the model trained with the 8 samples and the model trained without them was 0.897. Also, I wrote in my solution that the CV score improved by 0.04, but that was a calculation mistake—the correct improvement was 0.01.",
      "votes": null
    },
    {
      "id": "3400006",
      "postDate": "01/31/2026 17:14:19",
      "content": "<p>ah makes sense now. thanks and congrats on the gold 🥇</p>",
      "rawMarkdown": "ah makes sense now. thanks and congrats on the gold 🥇",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 3399572,
      "author_name": "minfuka",
      "author_url": "",
      "post_date": "01/31/2026 01:45:06",
      "content": "<p>It was very clear and helpful. Thank you. Especially the model configuration diagram—it's a simple and interesting illustration.</p>",
      "votes": null,
      "replies": [
        {
          "id": 3399798,
          "author_name": "owaowata",
          "author_url": "",
          "post_date": "01/31/2026 10:25:43",
          "content": "<p>I’m glad I could help!</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 3399666,
      "author_name": "yeoyunsianggeremie",
      "author_url": "",
      "post_date": "01/31/2026 05:59:52",
      "content": "<pre><code>For samples with State=WA and Sampling_Date=2015-08-21, the label indicates Dry_Green_g = 0. However, upon inspecting the images, there were clearly plants other than Clover present. I therefore treated these as label errors and removed the following 8 samples from the training data:\n</code></pre>\n<p>what was your OOF correlation between the model trained with/without these 8 samples? \nDid the same and for me was 0.99x, extremely similar…</p>",
      "votes": null,
      "replies": [
        {
          "id": 3399941,
          "author_name": "owaowata",
          "author_url": "",
          "post_date": "01/31/2026 15:03:30",
          "content": "<p>The correlation between the predicted Dry_Total_g values from the model trained with the 8 samples and the model trained without them was 0.897. Also, I wrote in my solution that the CV score improved by 0.04, but that was a calculation mistake—the correct improvement was 0.01.</p>",
          "votes": null,
          "replies": [
            {
              "id": 3400006,
              "author_name": "yeoyunsianggeremie",
              "author_url": "",
              "post_date": "01/31/2026 17:14:19",
              "content": "<p>ah makes sense now. thanks and congrats on the gold 🥇</p>",
              "votes": null,
              "replies": []
            }
          ]
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "3399501": "We would first like to express our gratitude to the competition host and the Kaggle staff for organizing this outstanding competition.\nBelow, we introduce the my solution\n\n# Solution\n\n| Item | Description |\n| --- | --- |\n| Model | DINOv3 ViT-L/16 distilled (Fine-tuned: regression head + last two transformer blocks) |\n| Input | Images resized to 768×1536 |\n| Output | 5 target variables + auxiliary-loss features (Height, NDVI, and whether Species is Ryegrass (Ryegrass_Clover)) |\n| Loss function | Huber loss weighted the same way as the evaluation metric + auxiliary loss |\n| Preprocessing | - Removed 8 samples suspected as label noise after inspection<br> - Standardized the target variables |\n| CV | Grouped by State - Sampling-Date, then manually reviewed and merged similar groups. Split into 5 folds using a greedy method so that each target variable’s distribution is similar across folds.<br> CV and LB seemed correlated, so I tried to improve both. |\n| Anti-shake measures | Data augmentation to reproduce unnatural artifacts present in the train set (red-dot artifacts, and a date stamp in the bottom-right corner) |\n| Ensemble | Trained 3 models (with/without auxiliary losses) and averaged their predictions |\n| Other | - EMA<br> - TTA (horizontal flip) |\n\n## 1. Model, Input/Output, and Ensemble\n\nI used DINOv3 ViT-L/16 distilled. Images were resized to 768×1536 and fed into the model. The model outputs consist of the five target variables plus optional auxiliary-loss features, and the final prediction is the average of three models with different output configurations.\n\n![model](https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F14349659%2F7f8e1ff15c87b96301e346f81ac3003c%2Fmodel2.png?generation=1769785563062926&alt=media)\n\n| Pattern | Output configuration |\n| --- | --- |\n| 1 | 5 target variables |\n| 2 | 5 target variables + Height + NDVI |\n| 3 | 5 target variables + Height + NDVI + Species (binary classification: whether it is Ryegrass (Ryegrass_Clover)) |\n\n## 2. Preprocessing\n\nFor samples with State=WA and Sampling_Date=2015-08-21, the label indicates Dry_Green_g = 0. However, upon inspecting the images, there were clearly plants other than Clover present. I therefore treated these as label errors and removed the following 8 samples from the training data:\n\n```\n- ID681680726.jpg\n- ID697718693.jpg\n- ID40849327.jpg\n- ID230058600.jpg\n- ID1403107574.jpg\n- ID1337107565.jpg\n- ID1139918758.jpg\n- ID1761544403.jpg\n```\n\n~~This improved the score by +0.04 on CV, +0.05 on Public, and +0.01 on Private.~~\n\n\n\n\nThis improved the score by +0.01 on CV, +0.01 on Public, and +0.01 on Private.\n(I corrected it because I had made a mistake.)\n\n\n## 3. Anti-shake Measures\n\nThe training data contains unnatural artifacts such as the following, so I applied data augmentation to reproduce them:\n\n* Red-dot artifacts, likely caused by flash\n* The capture date/time appearing in the bottom-right corner of the image\n\nThis did not help on CV or Public, but improved the Private score by +0.01. When I tested them separately, the date/time augmentation did not improve the Public score, so this suggests the private set may contain some red-dot artifacts.\n\n## 4. Tried but Not Effective\n\n* Manifold Mixup\n* Tuning all layers with LoRA\n* Pseudo-labeling using data that was not used for training\n* Using patch tokens (maybe I just used it incorrectly?)\n* Using PlantCLEF2024 data / a pre-trained model (DINOv2)\n\n\n# In Closing\n\nIf you have any questions, feel free to ask. \nThank you for reading!",
    "3399572": "It was very clear and helpful. Thank you. Especially the model configuration diagram—it's a simple and interesting illustration.",
    "3399666": "```\nFor samples with State=WA and Sampling_Date=2015-08-21, the label indicates Dry_Green_g = 0. However, upon inspecting the images, there were clearly plants other than Clover present. I therefore treated these as label errors and removed the following 8 samples from the training data:\n```\n\nwhat was your OOF correlation between the model trained with/without these 8 samples? \nDid the same and for me was 0.99x, extremely similar...",
    "3399798": "I’m glad I could help!",
    "3399941": "The correlation between the predicted Dry_Total_g values from the model trained with the 8 samples and the model trained without them was 0.897. Also, I wrote in my solution that the CV score improved by 0.04, but that was a calculation mistake—the correct improvement was 0.01.",
    "3400006": "ah makes sense now. thanks and congrats on the gold 🥇"
  },
  "source": "meta"
}