Honestly, I still cannot believe this. My first silver medal. 33rd out of 3000+ teams. I had to refresh the leaderboard three times to make sure it was real :)
\nFirst of all, thank you to CSIRO and everyone involved in organizing this competition. Predicting pasture biomass from images is such a real-world problem and I learned so much working on it.
\nA quick shoutout to the public notebooks and datasets that helped me along the way - the SigLIP model, the data split notebook, and the baseline approaches that many of us built upon. The Kaggle community is genuinely amazing.
\nI joined this competition about a month ago but if I am being honest, most of my serious work happened in the last 15 days. For the first few weeks I was just reading discussions, running public notebooks, trying to understand what works and what does not.
\nThe public leaderboard was stuck around 0.72-0.73 for many people including me. I kept trying small things here and there but nothing was really moving the needle. Then I started thinking differently.
\nThe main idea that changed everything for me was this: instead of using frozen DINO embeddings like everyone else, what if I actually retrain the DINO model from scratch on this specific dataset? And not just any DINO, but the huge version with 1.1 billion parameters.
\nThis was risky because training such a massive model on only 357 images could easily overfit. But with proper regularization and early stopping, it worked. I went from 0.73 to 0.74 and jumped from 106th on public LB to 33rd on private LB.
\nI used vit_huge_plus_patch16_dinov3 as my backbone. This is a Vision Transformer with about 1.1 billion parameters pretrained with DINOv3 self-supervised learning.
\nEach pasture image is 2000x1000 pixels. I split it vertically into left half and right half, each 1000x1000. Both halves go through the same backbone (shared weights), then I concatenate the features together.
\nFor fusing these features, I used something called Local Mamba Blocks. This is basically a lightweight attention mechanism with depthwise convolutions and gating:
\nclass LocalMambaBlock(nn.Module):\n def __init__(self, dim, kernel_size=5, dropout=0.1):\n super().__init__()\n self.norm = nn.LayerNorm(dim)\n self.dwconv = nn.Conv1d(dim, dim, kernel_size, \n padding=kernel_size//2, groups=dim)\n self.gate = nn.Linear(dim, dim)\n self.proj = nn.Linear(dim, dim)\n self.drop = nn.Dropout(dropout)\n\n def forward(self, x):\n shortcut = x\n x = self.norm(x)\n g = torch.sigmoid(self.gate(x))\n x = x * g\n x = self.dwconv(x.transpose(1, 2)).transpose(1, 2)\n x = self.proj(x)\n return shortcut + self.drop(x)\nThe idea is simple: normalize, gate the information flow, apply local convolution, project back, and add residual. Two of these blocks stacked together gave me good feature fusion.
\nAfter fusion, I have three separate prediction heads for Green, Dead, and Clover. Then GDM and Total are computed from these (GDM = Green + Clover, Total = GDM + Dead). This respects the actual physical relationships between the targets.
\nHere is what worked for me:
\nFor augmentation I kept it simple:
\nThe loss function is weighted to match the competition metric. I used Huber loss in log space to handle the wide range of biomass values and the outliers:
\nweights = [0.1, 0.1, 0.1, 0.2, 0.5] # Green, Dead, Clover, GDM, Total\nloss = weighted_mean(HuberLoss(log1p(pred), log1p(target)))\nDINO alone got me to around 0.72-0.73. To push further, I combined it with SigLIP.
\nSigLIP is a vision-language model. I extracted embeddings from images and computed similarity scores against agricultural text concepts like \"bare soil\", \"dense pasture\", \"dead grass\", \"white clover\" etc. These semantic features were then fed into a GBDT ensemble (HistGradientBoosting, GradientBoosting, CatBoost, LightGBM averaged together).
\nThe final prediction is 75% DINO and 25% SigLIP for most targets. For Clover specifically, I use 100% DINO because SigLIP was not good at predicting it.
\nSmall calibrations that helped:
\n# Clover tends to overpredict\nDry_Clover_g = Dry_Clover_g * 0.8\n\n# Dead material adjustments based on predicted range\nif Dry_Dead_g > 20:\n Dry_Dead_g = Dry_Dead_g * 1.1\nelif Dry_Dead_g < 10:\n Dry_Dead_g = Dry_Dead_g * 0.9\n\n# Enforce mass balance\nGDM_g = Dry_Green_g + Dry_Clover_g\nDry_Total_g = GDM_g + Dry_Dead_g\nCross-validation scores:
\n| Fold | \nR2 Score | \n
|---|---|
| 0 | \n0.82 | \n
| 1 | \n0.85 | \n
| 2 | \n0.81 | \n
| 3 | \n0.84 | \n
| Mean | \n0.83 | \n
Leaderboard:
\n| Stage | \nScore | \nRank | \n
|---|---|---|
| Public LB | \n0.73 | \n105th | \n
| Private LB | \n0.74 | \n33rd | \n
The shakeup was real. Trusting CV over public LB paid off.
\nI am sharing my complete solution:
\nTraining Code: A clean single-cell notebook with the full DINO training pipeline. I ran this privately so the notebook does not show my actual training outputs, but the code is exactly what produced my 0.74 submission.
\nhttps://www.kaggle.com/code/ibrahimqasimi/csiro-biomass-33rd-rank-training-0-74-0-63
\nInference Code: The ensemble inference that combines DINO and SigLIP. This is what generates the final submission.csv file.
\nhttps://www.kaggle.com/code/ibrahimqasimi/csiro-biomass-33rd-rank-inference-0-74-0-63
\nModel Checkpoints: The trained model weights (4 folds) that you can use directly for inference.
\nhttps://www.kaggle.com/datasets/ibrahimqasimi/dino-huge-retrain-checkpoints-zul
\nGitHub Repository: Complete code repository with training and inference scripts.
\nhttps://github.com/muhammadibrahim313/CSIRO-Image2Biomass-33rd-Place-Silver-Kaggle-Comp
\nThank you for reading. This was my first silver medal and it means a lot to me. If you have any questions about the solution, feel free to ask in the comments.
\nPS: i am now kaggle Competitions Expert :)
\nMuhammad Ibrahim Qasmi
Honestly, I still cannot believe this. My first silver medal. 33rd out of 3000+ teams. I had to refresh the leaderboard three times to make sure it was real :)
\nFirst of all, thank you to CSIRO and everyone involved in organizing this competition. Predicting pasture biomass from images is such a real-world problem and I learned so much working on it.
\nA quick shoutout to the public notebooks and datasets that helped me along the way - the SigLIP model, the data split notebook, and the baseline approaches that many of us built upon. The Kaggle community is genuinely amazing.
\nI joined this competition about a month ago but if I am being honest, most of my serious work happened in the last 15 days. For the first few weeks I was just reading discussions, running public notebooks, trying to understand what works and what does not.
\nThe public leaderboard was stuck around 0.72-0.73 for many people including me. I kept trying small things here and there but nothing was really moving the needle. Then I started thinking differently.
\nThe main idea that changed everything for me was this: instead of using frozen DINO embeddings like everyone else, what if I actually retrain the DINO model from scratch on this specific dataset? And not just any DINO, but the huge version with 1.1 billion parameters.
\nThis was risky because training such a massive model on only 357 images could easily overfit. But with proper regularization and early stopping, it worked. I went from 0.73 to 0.74 and jumped from 106th on public LB to 33rd on private LB.
\nI used vit_huge_plus_patch16_dinov3 as my backbone. This is a Vision Transformer with about 1.1 billion parameters pretrained with DINOv3 self-supervised learning.
\nEach pasture image is 2000x1000 pixels. I split it vertically into left half and right half, each 1000x1000. Both halves go through the same backbone (shared weights), then I concatenate the features together.
\nFor fusing these features, I used something called Local Mamba Blocks. This is basically a lightweight attention mechanism with depthwise convolutions and gating:
\nclass LocalMambaBlock(nn.Module):\n def __init__(self, dim, kernel_size=5, dropout=0.1):\n super().__init__()\n self.norm = nn.LayerNorm(dim)\n self.dwconv = nn.Conv1d(dim, dim, kernel_size, \n padding=kernel_size//2, groups=dim)\n self.gate = nn.Linear(dim, dim)\n self.proj = nn.Linear(dim, dim)\n self.drop = nn.Dropout(dropout)\n\n def forward(self, x):\n shortcut = x\n x = self.norm(x)\n g = torch.sigmoid(self.gate(x))\n x = x * g\n x = self.dwconv(x.transpose(1, 2)).transpose(1, 2)\n x = self.proj(x)\n return shortcut + self.drop(x)\nThe idea is simple: normalize, gate the information flow, apply local convolution, project back, and add residual. Two of these blocks stacked together gave me good feature fusion.
\nAfter fusion, I have three separate prediction heads for Green, Dead, and Clover. Then GDM and Total are computed from these (GDM = Green + Clover, Total = GDM + Dead). This respects the actual physical relationships between the targets.
\nHere is what worked for me:
\nFor augmentation I kept it simple:
\nThe loss function is weighted to match the competition metric. I used Huber loss in log space to handle the wide range of biomass values and the outliers:
\nweights = [0.1, 0.1, 0.1, 0.2, 0.5] # Green, Dead, Clover, GDM, Total\nloss = weighted_mean(HuberLoss(log1p(pred), log1p(target)))\nDINO alone got me to around 0.72-0.73. To push further, I combined it with SigLIP.
\nSigLIP is a vision-language model. I extracted embeddings from images and computed similarity scores against agricultural text concepts like \"bare soil\", \"dense pasture\", \"dead grass\", \"white clover\" etc. These semantic features were then fed into a GBDT ensemble (HistGradientBoosting, GradientBoosting, CatBoost, LightGBM averaged together).
\nThe final prediction is 75% DINO and 25% SigLIP for most targets. For Clover specifically, I use 100% DINO because SigLIP was not good at predicting it.
\nSmall calibrations that helped:
\n# Clover tends to overpredict\nDry_Clover_g = Dry_Clover_g * 0.8\n\n# Dead material adjustments based on predicted range\nif Dry_Dead_g > 20:\n Dry_Dead_g = Dry_Dead_g * 1.1\nelif Dry_Dead_g < 10:\n Dry_Dead_g = Dry_Dead_g * 0.9\n\n# Enforce mass balance\nGDM_g = Dry_Green_g + Dry_Clover_g\nDry_Total_g = GDM_g + Dry_Dead_g\nCross-validation scores:
\n| Fold | \nR2 Score | \n
|---|---|
| 0 | \n0.82 | \n
| 1 | \n0.85 | \n
| 2 | \n0.81 | \n
| 3 | \n0.84 | \n
| Mean | \n0.83 | \n
Leaderboard:
\n| Stage | \nScore | \nRank | \n
|---|---|---|
| Public LB | \n0.73 | \n105th | \n
| Private LB | \n0.74 | \n33rd | \n
The shakeup was real. Trusting CV over public LB paid off.
\nI am sharing my complete solution:
\nTraining Code: A clean single-cell notebook with the full DINO training pipeline. I ran this privately so the notebook does not show my actual training outputs, but the code is exactly what produced my 0.74 submission.
\nhttps://www.kaggle.com/code/ibrahimqasimi/csiro-biomass-33rd-rank-training-0-74-0-63
\nInference Code: The ensemble inference that combines DINO and SigLIP. This is what generates the final submission.csv file.
\nhttps://www.kaggle.com/code/ibrahimqasimi/csiro-biomass-33rd-rank-inference-0-74-0-63
\nModel Checkpoints: The trained model weights (4 folds) that you can use directly for inference.
\nhttps://www.kaggle.com/datasets/ibrahimqasimi/dino-huge-retrain-checkpoints-zul
\nGitHub Repository: Complete code repository with training and inference scripts.
\nhttps://github.com/muhammadibrahim313/CSIRO-Image2Biomass-33rd-Place-Silver-Kaggle-Comp
\nThank you for reading. This was my first silver medal and it means a lot to me. If you have any questions about the solution, feel free to ask in the comments.
\nPS: i am now kaggle Competitions Expert :)
\nMuhammad Ibrahim Qasmi
@ibrahimqasimi Congratulations on your first silver medal. 🥈\n33rd out of 3000+ teams is impressive. Great work and well deserved. Congrats on becoming a Kaggle Competitions Expert!
", "rawMarkdown": "ibrahimqasimi Congratulations on your first silver medal. 🥈\n33rd out of 3000+ teams is impressive. Great work and well deserved. Congrats on becoming a Kaggle Competitions Expert!", "votes": null }, { "id": "3399829", "postDate": "01/31/2026 11:47:33", "content": "Congratulations. @ibrahimqasimi
", "rawMarkdown": "Congratulations. @ibrahimqasimi", "votes": null }, { "id": "3402860", "postDate": "02/07/2026 03:11:57", "content": "Congrats 🎉🎉.\nWhat resources did you use for training and what was the total train time.
", "rawMarkdown": "Congrats 🎉🎉.\nWhat resources did you use for training and what was the total train time.", "votes": null }, { "id": "3404250", "postDate": "02/10/2026 05:35:29", "content": "Amazing work,keep it up.
", "rawMarkdown": "Amazing work,keep it up.", "votes": null } ], "comments": [ { "id": 3399348, "author_name": "mehakiftikhar", "author_url": "", "post_date": "01/30/2026 17:36:11", "content": "@ibrahimqasimi Congratulations on your first silver medal. 🥈\n33rd out of 3000+ teams is impressive. Great work and well deserved. Congrats on becoming a Kaggle Competitions Expert!
", "votes": null, "replies": [] }, { "id": 3399829, "author_name": "", "author_url": "", "post_date": "01/31/2026 11:47:33", "content": "Congratulations. @ibrahimqasimi
", "votes": null, "replies": [] }, { "id": 3402860, "author_name": "cartoonistbeard", "author_url": "", "post_date": "02/07/2026 03:11:57", "content": "Congrats 🎉🎉.\nWhat resources did you use for training and what was the total train time.
", "votes": null, "replies": [] }, { "id": 3404250, "author_name": "", "author_url": "", "post_date": "02/10/2026 05:35:29", "content": "Amazing work,keep it up.
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "3399332": "Honestly, I still cannot believe this. My first silver medal. 33rd out of 3000+ teams. I had to refresh the leaderboard three times to make sure it was real :)\n\nFirst of all, thank you to CSIRO and everyone involved in organizing this competition. Predicting pasture biomass from images is such a real-world problem and I learned so much working on it.\n\nA quick shoutout to the public notebooks and datasets that helped me along the way - the SigLIP model, the data split notebook, and the baseline approaches that many of us built upon. The Kaggle community is genuinely amazing.\n\n\n## My Journey\n\nI joined this competition about a month ago but if I am being honest, most of my serious work happened in the last 15 days. For the first few weeks I was just reading discussions, running public notebooks, trying to understand what works and what does not.\n\nThe public leaderboard was stuck around 0.72-0.73 for many people including me. I kept trying small things here and there but nothing was really moving the needle. Then I started thinking differently.\n\nThe main idea that changed everything for me was this: instead of using frozen DINO embeddings like everyone else, what if I actually retrain the DINO model from scratch on this specific dataset? And not just any DINO, but the huge version with 1.1 billion parameters.\n\nThis was risky because training such a massive model on only 357 images could easily overfit. But with proper regularization and early stopping, it worked. I went from 0.73 to 0.74 and jumped from 106th on public LB to 33rd on private LB.\n\n\n\n## What I Actually Did\n\n### The Model\n\nI used vit_huge_plus_patch16_dinov3 as my backbone. This is a Vision Transformer with about 1.1 billion parameters pretrained with DINOv3 self-supervised learning.\n\nEach pasture image is 2000x1000 pixels. I split it vertically into left half and right half, each 1000x1000. Both halves go through the same backbone (shared weights), then I concatenate the features together.\n\nFor fusing these features, I used something called Local Mamba Blocks. This is basically a lightweight attention mechanism with depthwise convolutions and gating:\n\n```python\nclass LocalMambaBlock(nn.Module):\n def __init__(self, dim, kernel_size=5, dropout=0.1):\n super().__init__()\n self.norm = nn.LayerNorm(dim)\n self.dwconv = nn.Conv1d(dim, dim, kernel_size, \n padding=kernel_size//2, groups=dim)\n self.gate = nn.Linear(dim, dim)\n self.proj = nn.Linear(dim, dim)\n self.drop = nn.Dropout(dropout)\n\n def forward(self, x):\n shortcut = x\n x = self.norm(x)\n g = torch.sigmoid(self.gate(x))\n x = x * g\n x = self.dwconv(x.transpose(1, 2)).transpose(1, 2)\n x = self.proj(x)\n return shortcut + self.drop(x)\n```\n\nThe idea is simple: normalize, gate the information flow, apply local convolution, project back, and add residual. Two of these blocks stacked together gave me good feature fusion.\n\nAfter fusion, I have three separate prediction heads for Green, Dead, and Clover. Then GDM and Total are computed from these (GDM = Green + Clover, Total = GDM + Dead). This respects the actual physical relationships between the targets.\n\n### Training Details\n\nHere is what worked for me:\n\n- Image size: 512x512\n- Batch size: 6\n- 4-fold cross validation using StratifiedGroupKFold\n- Learning rate: 1e-5 for backbone, 5e-4 for head\n- Weight decay: 1e-2\n- Dropout: 0.2\n- Early stopping patience: 15 epochs\n- Max epochs: 210 (but early stopping kicked in much earlier)\n- Cosine annealing scheduler with 2 epoch warmup\n- Mixed precision training (fp16)\n\nFor augmentation I kept it simple:\n- Horizontal flip\n- Vertical flip\n- Random rotate 90\n- Shift scale rotate\n- Color jitter (mild)\n\nThe loss function is weighted to match the competition metric. I used Huber loss in log space to handle the wide range of biomass values and the outliers:\n\n```python\nweights = [0.1, 0.1, 0.1, 0.2, 0.5] # Green, Dead, Clover, GDM, Total\nloss = weighted_mean(HuberLoss(log1p(pred), log1p(target)))\n```\n\n### The Ensemble\n\nDINO alone got me to around 0.72-0.73. To push further, I combined it with SigLIP.\n\nSigLIP is a vision-language model. I extracted embeddings from images and computed similarity scores against agricultural text concepts like \"bare soil\", \"dense pasture\", \"dead grass\", \"white clover\" etc. These semantic features were then fed into a GBDT ensemble (HistGradientBoosting, GradientBoosting, CatBoost, LightGBM averaged together).\n\nThe final prediction is 75% DINO and 25% SigLIP for most targets. For Clover specifically, I use 100% DINO because SigLIP was not good at predicting it.\n\n### Post-Processing\n\nSmall calibrations that helped:\n\n```python\n# Clover tends to overpredict\nDry_Clover_g = Dry_Clover_g * 0.8\n\n# Dead material adjustments based on predicted range\nif Dry_Dead_g > 20:\n Dry_Dead_g = Dry_Dead_g * 1.1\nelif Dry_Dead_g < 10:\n Dry_Dead_g = Dry_Dead_g * 0.9\n\n# Enforce mass balance\nGDM_g = Dry_Green_g + Dry_Clover_g\nDry_Total_g = GDM_g + Dry_Dead_g\n```\n\n\n\n## Results\n\nCross-validation scores:\n\n| Fold | R2 Score |\n|------|----------|\n| 0 | 0.82 |\n| 1 | 0.85 |\n| 2 | 0.81 |\n| 3 | 0.84 |\n| Mean | 0.83 |\n\nLeaderboard:\n\n| Stage | Score | Rank |\n|-------|-------|------|\n| Public LB | 0.73 | 105th |\n| Private LB | 0.74 | 33rd |\n\nThe shakeup was real. Trusting CV over public LB paid off.\n\n\n\n## What I Learned\n\n1. Retraining beats frozen features - Fine-tuning DINO on this specific domain was the biggest gain\n2. Respect the target relationships - GDM and Total are derived quantities, predictions should maintain this\n3. Simple augmentation works - Heavy augmentation hurt more than helped on this small dataset\n4. Regularization is essential - Dropout, weight decay, and early stopping prevented overfitting on 357 images\n5. Trust your CV - Public LB was misleading, private LB matched my CV much better\n\n\n\n## Code and Resources\n\nI am sharing my complete solution:\n\n**Training Code**: A clean single-cell notebook with the full DINO training pipeline. I ran this privately so the notebook does not show my actual training outputs, but the code is exactly what produced my 0.74 submission.\n\nhttps://www.kaggle.com/code/ibrahimqasimi/csiro-biomass-33rd-rank-training-0-74-0-63\n\n**Inference Code**: The ensemble inference that combines DINO and SigLIP. This is what generates the final submission.csv file.\n\nhttps://www.kaggle.com/code/ibrahimqasimi/csiro-biomass-33rd-rank-inference-0-74-0-63\n\n**Model Checkpoints**: The trained model weights (4 folds) that you can use directly for inference.\n\nhttps://www.kaggle.com/datasets/ibrahimqasimi/dino-huge-retrain-checkpoints-zul\n\n**GitHub Repository**: Complete code repository with training and inference scripts.\n\nhttps://github.com/muhammadibrahim313/CSIRO-Image2Biomass-33rd-Place-Silver-Kaggle-Comp\n\n\n\nThank you for reading. This was my first silver medal and it means a lot to me. If you have any questions about the solution, feel free to ask in the comments.\n\nPS: i am now kaggle Competitions Expert :) \nMuhammad Ibrahim Qasmi", "3399348": "ibrahimqasimi Congratulations on your first silver medal. 🥈\n33rd out of 3000+ teams is impressive. Great work and well deserved. Congrats on becoming a Kaggle Competitions Expert!", "3399829": "Congratulations. @ibrahimqasimi", "3402860": "Congrats 🎉🎉.\nWhat resources did you use for training and what was the total train time.", "3404250": "Amazing work,keep it up." }, "source": "meta" }