![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F18908513%2Fa13e6b6e61001440d967ec88f50218a0%2Fpart%20-%201.png?generation=1769666215659267&alt=media\")
First, a huge thank you to the organizers for hosting this challenge and to my fellow competitors. Sharath and I are thrilled to achieve the 4th position (Gold Medal). Our solution relies on a heavy Vision Transformer backbone initialized with DINOv3 weights, a multi-modal fusion strategy combining images with tabular data, and a critical data cleaning pipeline.
\nBefore touching the model architecture, we realized a significant portion of the image data contained irrelevant noise—specifically, the cardboard backing used in the data collection process.
\nManual Cropping: My teammate and I manually reviewed the dataset and cropped out the cardboard edges from the pasture images. This ensured the model focused purely on the biomass (grass/clover) rather than learning artifacts from the background.
Impact: This step was crucial. It provided a clear boost of +0.01 on both leaderboards:
\nPublic LB: 0.74 →0.75
Private LB: 0.64 →0.65
My main approach was a Multi-Modal Regression Network. I treated the problem as a fusion of visual data (pasture images) and physical measurements (Height/NDVI).
\n
Backbone: vit_huge_plus_patch16_dinov3.lvd1689m weights. The self-supervised nature of DINOv3 provided superior feature extraction for grass textures compared to standard supervised weights with first half layers frozen.
Fusion Strategy: I did not rely solely on images. Pre_GSHH_NDVI and Height_Ave_cm were encoded via a 2-layer MLP and concatenated with the ViT global average pooling features before the final regression heads.
Loss Function: WeightedSmoothL1Loss. I used custom weights [0.1, 0.1, 0.1, 0.2, 0.5] to prioritize the \"Total\" and \"GDM\" biomass targets.
A major boost came from a secondary training stage where I repurposed the trained backbone to predict the tabular features from the images.
\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F18908513%2F4e9dd4df598970a80cc532db250f5c56%2Fpart%20-%202.png?generation=1769666274910752&alt=media\")
Logic: By forcing the model to predict NDVI and Height solely from the RGB image, the backbone learned robust features correlated with plant health and density that the primary regression heads might have missed.
Initialization: This model was initialized with the best weights from Fold 0 of the main training loop, essentially acting as a domain-specific fine-tuner.
Impact: This auxiliary training provided the final push for the Gold Medal, adding another +0.01 to both scores:
\nFinal Public LB: 0.76
Final Private LB: 0.66
Despite the success, many ideas were left on the cutting room floor.
\nLoss Functions: Pure MSE, Quantile Loss, Log Loss, and direct R2 optimization all failed to beat SmoothL1.
Target Transforms: Log transformation of targets and Scaled Sigmoid outputs reduced performance.
Image Size: Input sizes larger than 800x800 offered diminishing returns and OOM errors.
Complex Input: Attempting to process two images simultaneously (complex dual-input) did not yield better results.
I experimented with a custom pooling layer designed to reconstruct spatial grids from ViT tokens and pool across the height dimension (simulating a scan). While the validation loss looked very promising, it did not generalize as well to the private leaderboard.
\nPython
\nclass HybridTexturePooling(nn.Module):\n def __init__(self, embed_dim=1280, patch_size=16, num_extra_tokens=5):\n super().__init__()\n self.ps = patch_size\n self.num_extra_tokens = num_extra_tokens\n # Projection layer: maps ViT embedding to pixel-space (ps**2) * 2\n self.projection = nn.Linear(embed_dim, (patch_size ** 2) * 2)\n\n def forward(self, x, h, w):\n # x: [BS, N_tokens, Dim]\n # 1. Slice off CLS + Register tokens\n patch_tokens = x[:, self.num_extra_tokens:, :] \n\n # 2. Project to pixel-like space\n x = self.projection(patch_tokens) \n\n bs = x.shape[0]\n h_patches, w_patches = h // self.ps, w // self.ps\n\n # 3. Reconstruct Spatial Grid\n # Reshape to (BS, H_p, W_p, PS, PS, 2)\n x = x.view(bs, h_patches, w_patches, self.ps, self.ps, 2)\n # Permute to (BS, H_p, PS, W_p, PS, 2)\n x = x.permute(0, 1, 3, 2, 4, 5).contiguous()\n # Flatten to (BS, H, W * 2)\n x = x.view(bs, h, w * 2)\n\n # 4. Mean across the Height dimension\n x = x.mean(dim=1) # Result: [BS, 1600] for 800px input\n return x\nWe utilized a two-stage training pipeline. Below are the specific hyperparameters used for each stage.
\nModel: vit_huge_plus_patch16_dinov3.lvd1689m (Weights frozen for first 50% of layers)
Image Size: 800× 800
Batch Size: 10
Optimizer: AdamW
Scheduler: CosineAnnealingWarmRestarts(optimizer, T_0=10, T_mult=2, eta_min=1e-6)
Loss Function: WeightedSmoothL1Loss
[0.1, 0.1, 0.1, 0.2, 0.5] (Heavily penalizing Total Biomass errors)Validation Strategy: 5-Fold Cross-Validation (Seed 42)
Objective: Predict Tabular Features (NDVI, Height) from Images
Initialization: Loaded best weights from Stage 1 (Fold 0)
Batch Size: 8
Optimizer: AdamW
Scheduler: ReduceLROnPlateau(optimizer, mode='min', factor=0.5, patience=4)
Loss Function: MSELoss
Validation Strategy: 5-Fold Cross-Validation (Seed 44)
Note: All regression targets and auxiliary features (tabular inputs) were normalized using StandardScaler prior to training to ensure stable convergence.
\nThanks again to the community! I hope this write-up and the diagrams help explain the structure of the solution.
", "messages": [ { "id": "3398421", "postDate": "01/29/2026 05:44:24", "content": "First, a huge thank you to the organizers for hosting this challenge and to my fellow competitors. Sharath and I are thrilled to achieve the 4th position (Gold Medal). Our solution relies on a heavy Vision Transformer backbone initialized with DINOv3 weights, a multi-modal fusion strategy combining images with tabular data, and a critical data cleaning pipeline.
\nBefore touching the model architecture, we realized a significant portion of the image data contained irrelevant noise—specifically, the cardboard backing used in the data collection process.
\nManual Cropping: My teammate and I manually reviewed the dataset and cropped out the cardboard edges from the pasture images. This ensured the model focused purely on the biomass (grass/clover) rather than learning artifacts from the background.
Impact: This step was crucial. It provided a clear boost of +0.01 on both leaderboards:
\nPublic LB: 0.74 →0.75
Private LB: 0.64 →0.65
My main approach was a Multi-Modal Regression Network. I treated the problem as a fusion of visual data (pasture images) and physical measurements (Height/NDVI).
\n
Backbone: vit_huge_plus_patch16_dinov3.lvd1689m weights. The self-supervised nature of DINOv3 provided superior feature extraction for grass textures compared to standard supervised weights with first half layers frozen.
Fusion Strategy: I did not rely solely on images. Pre_GSHH_NDVI and Height_Ave_cm were encoded via a 2-layer MLP and concatenated with the ViT global average pooling features before the final regression heads.
Loss Function: WeightedSmoothL1Loss. I used custom weights [0.1, 0.1, 0.1, 0.2, 0.5] to prioritize the \"Total\" and \"GDM\" biomass targets.
A major boost came from a secondary training stage where I repurposed the trained backbone to predict the tabular features from the images.
\n
Logic: By forcing the model to predict NDVI and Height solely from the RGB image, the backbone learned robust features correlated with plant health and density that the primary regression heads might have missed.
Initialization: This model was initialized with the best weights from Fold 0 of the main training loop, essentially acting as a domain-specific fine-tuner.
Impact: This auxiliary training provided the final push for the Gold Medal, adding another +0.01 to both scores:
\nFinal Public LB: 0.76
Final Private LB: 0.66
Despite the success, many ideas were left on the cutting room floor.
\nLoss Functions: Pure MSE, Quantile Loss, Log Loss, and direct R2 optimization all failed to beat SmoothL1.
Target Transforms: Log transformation of targets and Scaled Sigmoid outputs reduced performance.
Image Size: Input sizes larger than 800x800 offered diminishing returns and OOM errors.
Complex Input: Attempting to process two images simultaneously (complex dual-input) did not yield better results.
I experimented with a custom pooling layer designed to reconstruct spatial grids from ViT tokens and pool across the height dimension (simulating a scan). While the validation loss looked very promising, it did not generalize as well to the private leaderboard.
\nPython
\nclass HybridTexturePooling(nn.Module):\n def __init__(self, embed_dim=1280, patch_size=16, num_extra_tokens=5):\n super().__init__()\n self.ps = patch_size\n self.num_extra_tokens = num_extra_tokens\n # Projection layer: maps ViT embedding to pixel-space (ps**2) * 2\n self.projection = nn.Linear(embed_dim, (patch_size ** 2) * 2)\n\n def forward(self, x, h, w):\n # x: [BS, N_tokens, Dim]\n # 1. Slice off CLS + Register tokens\n patch_tokens = x[:, self.num_extra_tokens:, :] \n\n # 2. Project to pixel-like space\n x = self.projection(patch_tokens) \n\n bs = x.shape[0]\n h_patches, w_patches = h // self.ps, w // self.ps\n\n # 3. Reconstruct Spatial Grid\n # Reshape to (BS, H_p, W_p, PS, PS, 2)\n x = x.view(bs, h_patches, w_patches, self.ps, self.ps, 2)\n # Permute to (BS, H_p, PS, W_p, PS, 2)\n x = x.permute(0, 1, 3, 2, 4, 5).contiguous()\n # Flatten to (BS, H, W * 2)\n x = x.view(bs, h, w * 2)\n\n # 4. Mean across the Height dimension\n x = x.mean(dim=1) # Result: [BS, 1600] for 800px input\n return x\nWe utilized a two-stage training pipeline. Below are the specific hyperparameters used for each stage.
\nModel: vit_huge_plus_patch16_dinov3.lvd1689m (Weights frozen for first 50% of layers)
Image Size: 800× 800
Batch Size: 10
Optimizer: AdamW
Scheduler: CosineAnnealingWarmRestarts(optimizer, T_0=10, T_mult=2, eta_min=1e-6)
Loss Function: WeightedSmoothL1Loss
[0.1, 0.1, 0.1, 0.2, 0.5] (Heavily penalizing Total Biomass errors)Validation Strategy: 5-Fold Cross-Validation (Seed 42)
Objective: Predict Tabular Features (NDVI, Height) from Images
Initialization: Loaded best weights from Stage 1 (Fold 0)
Batch Size: 8
Optimizer: AdamW
Scheduler: ReduceLROnPlateau(optimizer, mode='min', factor=0.5, patience=4)
Loss Function: MSELoss
Validation Strategy: 5-Fold Cross-Validation (Seed 44)
Note: All regression targets and auxiliary features (tabular inputs) were normalized using StandardScaler prior to training to ensure stable convergence.
\nThanks again to the community! I hope this write-up and the diagrams help explain the structure of the solution.
", "rawMarkdown": "# ViT-Huge DINOv3 & Multi-Modal Feature Fusion with AUX prediction\n\nFirst, a huge thank you to the organizers for hosting this challenge and to my fellow competitors. Sharath and I are thrilled to achieve the **4th position (Gold Medal)**. Our solution relies on a heavy Vision Transformer backbone initialized with DINOv3 weights, a multi-modal fusion strategy combining images with tabular data, and a critical data cleaning pipeline.\n\n## 1. Data Preprocessing: The \"Cardboard\" Cleanup\n\nBefore touching the model architecture, we realized a significant portion of the image data contained irrelevant noise—specifically, the cardboard backing used in the data collection process.\n\n- **Manual Cropping:** My teammate and I manually reviewed the dataset and cropped out the cardboard edges from the pasture images. This ensured the model focused purely on the biomass (grass/clover) rather than learning artifacts from the background.\n \n- **Impact:** This step was crucial. It provided a clear boost of **+0.01** on both leaderboards:\n \n - **Public LB:** 0.74 →**0.75**\n \n - **Private LB:** 0.64 →**0.65**\n \n\n## 2. Main Model Architecture\n\nMy main approach was a **Multi-Modal Regression Network**. I treated the problem as a fusion of visual data (pasture images) and physical measurements (Height/NDVI).\n\n### The Architecture Diagram\n\n\n\n### Key Components\n\n- **Backbone:** `vit_huge_plus_patch16_dinov3.lvd1689m` weights. The self-supervised nature of DINOv3 provided superior feature extraction for grass textures compared to standard supervised weights with first half layers frozen.\n \n- **Fusion Strategy:** I did not rely solely on images. `Pre_GSHH_NDVI` and `Height_Ave_cm` were encoded via a 2-layer MLP and concatenated with the ViT global average pooling features before the final regression heads.\n \n- **Loss Function:** `WeightedSmoothL1Loss`. I used custom weights `[0.1, 0.1, 0.1, 0.2, 0.5]` to prioritize the \"Total\" and \"GDM\" biomass targets.\n \n\n## 3. The \"Secret Sauce\": Auxiliary Task Training\n\nA major boost came from a secondary training stage where I repurposed the trained backbone to predict the _tabular features_ from the images.\n\n### Auxiliary Pipeline Diagram\n\n\n\n- **Logic:** By forcing the model to predict `NDVI` and `Height` solely from the RGB image, the backbone learned robust features correlated with plant health and density that the primary regression heads might have missed.\n \n- **Initialization:** This model was initialized with the **best weights from Fold 0** of the main training loop, essentially acting as a domain-specific fine-tuner.\n \n- **Impact:** This auxiliary training provided the final push for the Gold Medal, adding another **+0.01** to both scores:\n \n - **Final Public LB:** 0.76\n \n - **Final Private LB:** 0.66\n \n\n## 4. What Didn't Work (The Graveyard)\n\nDespite the success, many ideas were left on the cutting room floor.\n\n- **Loss Functions:** Pure MSE, Quantile Loss, Log Loss, and direct R2 optimization all failed to beat SmoothL1.\n \n- **Target Transforms:** Log transformation of targets and Scaled Sigmoid outputs reduced performance.\n \n- **Image Size:** Input sizes larger than 800x800 offered diminishing returns and OOM errors.\n \n- **Complex Input:** Attempting to process two images simultaneously (complex dual-input) did not yield better results.\n \n\n### The \"Promising but Failed\" Architecture: Hybrid Texture Pooling\n\nI experimented with a custom pooling layer designed to reconstruct spatial grids from ViT tokens and pool across the height dimension (simulating a scan). While the validation loss looked very promising, it did not generalize as well to the private leaderboard.\n\nPython\n\n```\nclass HybridTexturePooling(nn.Module):\n def __init__(self, embed_dim=1280, patch_size=16, num_extra_tokens=5):\n super().__init__()\n self.ps = patch_size\n self.num_extra_tokens = num_extra_tokens\n # Projection layer: maps ViT embedding to pixel-space (ps**2) * 2\n self.projection = nn.Linear(embed_dim, (patch_size ** 2) * 2)\n\n def forward(self, x, h, w):\n # x: [BS, N_tokens, Dim]\n # 1. Slice off CLS + Register tokens\n patch_tokens = x[:, self.num_extra_tokens:, :] \n \n # 2. Project to pixel-like space\n x = self.projection(patch_tokens) \n \n bs = x.shape[0]\n h_patches, w_patches = h // self.ps, w // self.ps\n \n # 3. Reconstruct Spatial Grid\n # Reshape to (BS, H_p, W_p, PS, PS, 2)\n x = x.view(bs, h_patches, w_patches, self.ps, self.ps, 2)\n # Permute to (BS, H_p, PS, W_p, PS, 2)\n x = x.permute(0, 1, 3, 2, 4, 5).contiguous()\n # Flatten to (BS, H, W * 2)\n x = x.view(bs, h, w * 2)\n \n # 4. Mean across the Height dimension\n x = x.mean(dim=1) # Result: [BS, 1600] for 800px input\n return x\n```\n\n### 5. Training Configuration\n\nWe utilized a two-stage training pipeline. Below are the specific hyperparameters used for each stage.\n\n#### **Stage 1: Main Biomass Regression**\n\n- **Model:** `vit_huge_plus_patch16_dinov3.lvd1689m` (Weights frozen for first 50% of layers)\n \n- **Image Size:** 800× 800\n \n- **Batch Size:** 10\n \n- **Optimizer:** AdamW\n \n- **Scheduler:** CosineAnnealingWarmRestarts(optimizer, T_0=10, T_mult=2, eta_min=1e-6)\n \n- **Loss Function:** `WeightedSmoothL1Loss`\n \n - Weights: `[0.1, 0.1, 0.1, 0.2, 0.5]` (Heavily penalizing Total Biomass errors)\n \n- **Validation Strategy:** 5-Fold Cross-Validation (Seed 42)\n \n\n#### **Stage 2: Auxiliary Feature Pre-training**\n\n- **Objective:** Predict Tabular Features (`NDVI`, `Height`) from Images\n \n- **Initialization:** Loaded best weights from **Stage 1 (Fold 0)**\n \n- **Batch Size:** 8\n \n- **Optimizer:** AdamW\n \n- **Scheduler:** ReduceLROnPlateau(optimizer, mode='min', factor=0.5, patience=4)\n \n- **Loss Function:** `MSELoss`\n \n- **Validation Strategy:** 5-Fold Cross-Validation (Seed 44)\n\nNote: All regression targets and auxiliary features (tabular inputs) were normalized using StandardScaler prior to training to ensure stable convergence.\n\n\nThanks again to the community! I hope this write-up and the diagrams help explain the structure of the solution.", "votes": null }, { "id": "3398430", "postDate": "01/29/2026 06:02:18", "content": "Very nice work. and I have a question: during the model inference phase, do you use a one-stage model to predict NDVI and Height first, and then fuse them with image features?
", "rawMarkdown": "Very nice work. and I have a question: during the model inference phase, do you use a one-stage model to predict NDVI and Height first, and then fuse them with image features?", "votes": null }, { "id": "3398432", "postDate": "01/29/2026 06:22:26", "content": "What CV strategy do you use?
", "rawMarkdown": "What CV strategy do you use?", "votes": null }, { "id": "3398434", "postDate": "01/29/2026 06:25:26", "content": "first aux prediction then main targets
", "rawMarkdown": "first aux prediction then main targets", "votes": null }, { "id": "3398436", "postDate": "01/29/2026 06:27:43", "content": "normal k-fold, other strategy didn't work for me
", "rawMarkdown": "normal k-fold, other strategy didn't work for me", "votes": null }, { "id": "3398455", "postDate": "01/29/2026 07:05:27", "content": "Great implementation. I tried training the model directly to predict Height and NVDI but failed. \nMy hypothesis is that, learning the original targets using Height and NVDI must have forced/encouraged the model to use shared features, which then made it possible to finetune the same model to predict Height and NVDI from the images alone. Amazing.
\nI got a few questions, if you can answer it would be great:
\nThanks Komil great questions.
\nOn the 256 bottleneck, yes, 256 resolution will not capture image features (since we are doing texture level regression task), I used 800*800 image resolution.
\nThat said, direct multi-task training didn’t completely fail for me it just underperformed. I just used my main model's weight and trained the aux model on uncleaned data.
\nRegarding train–test mismatch due to cleaning- Even though the test data isn’t cleaned, performance didn’t degrade. My intuition is that cleaning during training actually helps the model unlearn background shortcuts. In other words, by seeing “vegetation without distractions”, the model becomes less sensitive to non-vegetative artifacts at test time rather than more.
\nOn contrastive learning- yes, I think there’s real scope there. A setup where the model is encouraged to produce similar predictions (or embeddings) for original vs. cleaned versions of the same image could explicitly teach invariance to background noise. I didn’t explore it due to time constraints, but conceptually it fits very well with this problem.
", "rawMarkdown": "Thanks Komil great questions.\n\nOn the 256 bottleneck, yes, 256 resolution will not capture image features (since we are doing texture level regression task), I used 800*800 image resolution.\n\nThat said, direct multi-task training didn’t completely fail for me it just underperformed. I just used my main model's weight and trained the aux model on uncleaned data.\n\nRegarding train–test mismatch due to cleaning- Even though the test data isn’t cleaned, performance didn’t degrade. My intuition is that cleaning during training actually helps the model unlearn background shortcuts. In other words, by seeing “vegetation without distractions”, the model becomes less sensitive to non-vegetative artifacts at test time rather than more.\n\nOn contrastive learning- yes, I think there’s real scope there. A setup where the model is encouraged to produce similar predictions (or embeddings) for original vs. cleaned versions of the same image could explicitly teach invariance to background noise. I didn’t explore it due to time constraints, but conceptually it fits very well with this problem.", "votes": null }, { "id": "3398488", "postDate": "01/29/2026 08:10:05", "content": "Congratulations！ Very clear and elegant solution!
", "rawMarkdown": "Congratulations！ Very clear and elegant solution!", "votes": null }, { "id": "3398499", "postDate": "01/29/2026 08:24:43", "content": "Dumb question regarding the auxilliary part. How did you leverage that to predict the actual targets?
", "rawMarkdown": "Dumb question regarding the auxilliary part. How did you leverage that to predict the actual targets?", "votes": null }, { "id": "3398507", "postDate": "01/29/2026 08:38:19", "content": "By using those value in my main model, I was using mean value before training the auxiliary values while predicting main bio mass heads.\nstep 1 - input ( Image (800*800) ) AUX MODEL -> Predictions (Height,NDVI)\nstep 2- input ( Image, [Height,NDVI] ) Main Model -> Predictions ('Dry_Green_g', 'Dry_Dead_g', 'Dry_Clover_g', 'GDM_g', 'Dry_Total_g')
", "rawMarkdown": "By using those value in my main model, I was using mean value before training the auxiliary values while predicting main bio mass heads.\nstep 1 - input ( Image (800*800) ) AUX MODEL -> Predictions (Height,NDVI)\nstep 2- input ( Image, [Height,NDVI] ) Main Model -> Predictions ('Dry_Green_g', 'Dry_Dead_g', 'Dry_Clover_g', 'GDM_g', 'Dry_Total_g')", "votes": null }, { "id": "3398586", "postDate": "01/29/2026 10:53:51", "content": "Faker is coming😆
", "rawMarkdown": "Faker is coming😆", "votes": null }, { "id": "3399123", "postDate": "01/30/2026 08:37:27", "content": "So you were not using the second stage training (the one on aux feats) to perform a fine tuning of the same backbone used by the main model (the model trained in the first stage), but fine tuning two distinct instances of the backbone (one for the main model and one for the aux model), right?
", "rawMarkdown": "So you were not using the second stage training (the one on aux feats) to perform a fine tuning of the same backbone used by the main model (the model trained in the first stage), but fine tuning two distinct instances of the backbone (one for the main model and one for the aux model), right?", "votes": null }, { "id": "3399128", "postDate": "01/30/2026 08:46:02", "content": "I used main model's first fold weight for training aux model.
", "rawMarkdown": "I used main model's first fold weight for training aux model.", "votes": null }, { "id": "3399861", "postDate": "01/31/2026 12:15:37", "content": "What is your final rank, can't see your team on the leaderboard?
", "rawMarkdown": "What is your final rank, can't see your team on the leaderboard?", "votes": null }, { "id": "3399871", "postDate": "01/31/2026 12:31:16", "content": "Hi Amir,
\nWe ended up in 4th place on the private leaderboard, but unfortunately, our team was removed. I’ve already sent an email to Kaggle about the issue, but I think the leaderboard results are final, so it’s unlikely the decision will be overturned.
", "rawMarkdown": "Hi Amir,\n\nWe ended up in 4th place on the private leaderboard, but unfortunately, our team was removed. I’ve already sent an email to Kaggle about the issue, but I think the leaderboard results are final, so it’s unlikely the decision will be overturned.", "votes": null } ], "comments": [ { "id": 3398430, "author_name": "playwithme", "author_url": "", "post_date": "01/29/2026 06:02:18", "content": "Very nice work. and I have a question: during the model inference phase, do you use a one-stage model to predict NDVI and Height first, and then fuse them with image features?
", "votes": null, "replies": [ { "id": 3398434, "author_name": "", "author_url": "", "post_date": "01/29/2026 06:25:26", "content": "first aux prediction then main targets
", "votes": null, "replies": [] }, { "id": 3398586, "author_name": "pursueml", "author_url": "", "post_date": "01/29/2026 10:53:51", "content": "Faker is coming😆
", "votes": null, "replies": [] } ] }, { "id": 3398432, "author_name": "wukaiqi", "author_url": "", "post_date": "01/29/2026 06:22:26", "content": "What CV strategy do you use?
", "votes": null, "replies": [ { "id": 3398436, "author_name": "", "author_url": "", "post_date": "01/29/2026 06:27:43", "content": "normal k-fold, other strategy didn't work for me
", "votes": null, "replies": [] } ] }, { "id": 3398455, "author_name": "komilparmar", "author_url": "", "post_date": "01/29/2026 07:05:27", "content": "Great implementation. I tried training the model directly to predict Height and NVDI but failed. \nMy hypothesis is that, learning the original targets using Height and NVDI must have forced/encouraged the model to use shared features, which then made it possible to finetune the same model to predict Height and NVDI from the images alone. Amazing.
\nI got a few questions, if you can answer it would be great:
\nThanks Komil great questions.
\nOn the 256 bottleneck, yes, 256 resolution will not capture image features (since we are doing texture level regression task), I used 800*800 image resolution.
\nThat said, direct multi-task training didn’t completely fail for me it just underperformed. I just used my main model's weight and trained the aux model on uncleaned data.
\nRegarding train–test mismatch due to cleaning- Even though the test data isn’t cleaned, performance didn’t degrade. My intuition is that cleaning during training actually helps the model unlearn background shortcuts. In other words, by seeing “vegetation without distractions”, the model becomes less sensitive to non-vegetative artifacts at test time rather than more.
\nOn contrastive learning- yes, I think there’s real scope there. A setup where the model is encouraged to produce similar predictions (or embeddings) for original vs. cleaned versions of the same image could explicitly teach invariance to background noise. I didn’t explore it due to time constraints, but conceptually it fits very well with this problem.
", "votes": null, "replies": [] } ] }, { "id": 3398488, "author_name": "forgakki", "author_url": "", "post_date": "01/29/2026 08:10:05", "content": "Congratulations！ Very clear and elegant solution!
", "votes": null, "replies": [] }, { "id": 3398499, "author_name": "nymfree", "author_url": "", "post_date": "01/29/2026 08:24:43", "content": "Dumb question regarding the auxilliary part. How did you leverage that to predict the actual targets?
", "votes": null, "replies": [ { "id": 3398507, "author_name": "", "author_url": "", "post_date": "01/29/2026 08:38:19", "content": "By using those value in my main model, I was using mean value before training the auxiliary values while predicting main bio mass heads.\nstep 1 - input ( Image (800*800) ) AUX MODEL -> Predictions (Height,NDVI)\nstep 2- input ( Image, [Height,NDVI] ) Main Model -> Predictions ('Dry_Green_g', 'Dry_Dead_g', 'Dry_Clover_g', 'GDM_g', 'Dry_Total_g')
", "votes": null, "replies": [ { "id": 3399123, "author_name": "vabite", "author_url": "", "post_date": "01/30/2026 08:37:27", "content": "So you were not using the second stage training (the one on aux feats) to perform a fine tuning of the same backbone used by the main model (the model trained in the first stage), but fine tuning two distinct instances of the backbone (one for the main model and one for the aux model), right?
", "votes": null, "replies": [ { "id": 3399128, "author_name": "", "author_url": "", "post_date": "01/30/2026 08:46:02", "content": "I used main model's first fold weight for training aux model.
", "votes": null, "replies": [] } ] } ] } ] }, { "id": 3399861, "author_name": "amir78pgd", "author_url": "", "post_date": "01/31/2026 12:15:37", "content": "What is your final rank, can't see your team on the leaderboard?
", "votes": null, "replies": [ { "id": 3399871, "author_name": "", "author_url": "", "post_date": "01/31/2026 12:31:16", "content": "Hi Amir,
\nWe ended up in 4th place on the private leaderboard, but unfortunately, our team was removed. I’ve already sent an email to Kaggle about the issue, but I think the leaderboard results are final, so it’s unlikely the decision will be overturned.
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "3398421": "# ViT-Huge DINOv3 & Multi-Modal Feature Fusion with AUX prediction\n\nFirst, a huge thank you to the organizers for hosting this challenge and to my fellow competitors. Sharath and I are thrilled to achieve the **4th position (Gold Medal)**. Our solution relies on a heavy Vision Transformer backbone initialized with DINOv3 weights, a multi-modal fusion strategy combining images with tabular data, and a critical data cleaning pipeline.\n\n## 1. Data Preprocessing: The \"Cardboard\" Cleanup\n\nBefore touching the model architecture, we realized a significant portion of the image data contained irrelevant noise—specifically, the cardboard backing used in the data collection process.\n\n- **Manual Cropping:** My teammate and I manually reviewed the dataset and cropped out the cardboard edges from the pasture images. This ensured the model focused purely on the biomass (grass/clover) rather than learning artifacts from the background.\n \n- **Impact:** This step was crucial. It provided a clear boost of **+0.01** on both leaderboards:\n \n - **Public LB:** 0.74 →**0.75**\n \n - **Private LB:** 0.64 →**0.65**\n \n\n## 2. Main Model Architecture\n\nMy main approach was a **Multi-Modal Regression Network**. I treated the problem as a fusion of visual data (pasture images) and physical measurements (Height/NDVI).\n\n### The Architecture Diagram\n\n\n\n### Key Components\n\n- **Backbone:** `vit_huge_plus_patch16_dinov3.lvd1689m` weights. The self-supervised nature of DINOv3 provided superior feature extraction for grass textures compared to standard supervised weights with first half layers frozen.\n \n- **Fusion Strategy:** I did not rely solely on images. `Pre_GSHH_NDVI` and `Height_Ave_cm` were encoded via a 2-layer MLP and concatenated with the ViT global average pooling features before the final regression heads.\n \n- **Loss Function:** `WeightedSmoothL1Loss`. I used custom weights `[0.1, 0.1, 0.1, 0.2, 0.5]` to prioritize the \"Total\" and \"GDM\" biomass targets.\n \n\n## 3. The \"Secret Sauce\": Auxiliary Task Training\n\nA major boost came from a secondary training stage where I repurposed the trained backbone to predict the _tabular features_ from the images.\n\n### Auxiliary Pipeline Diagram\n\n\n\n- **Logic:** By forcing the model to predict `NDVI` and `Height` solely from the RGB image, the backbone learned robust features correlated with plant health and density that the primary regression heads might have missed.\n \n- **Initialization:** This model was initialized with the **best weights from Fold 0** of the main training loop, essentially acting as a domain-specific fine-tuner.\n \n- **Impact:** This auxiliary training provided the final push for the Gold Medal, adding another **+0.01** to both scores:\n \n - **Final Public LB:** 0.76\n \n - **Final Private LB:** 0.66\n \n\n## 4. What Didn't Work (The Graveyard)\n\nDespite the success, many ideas were left on the cutting room floor.\n\n- **Loss Functions:** Pure MSE, Quantile Loss, Log Loss, and direct R2 optimization all failed to beat SmoothL1.\n \n- **Target Transforms:** Log transformation of targets and Scaled Sigmoid outputs reduced performance.\n \n- **Image Size:** Input sizes larger than 800x800 offered diminishing returns and OOM errors.\n \n- **Complex Input:** Attempting to process two images simultaneously (complex dual-input) did not yield better results.\n \n\n### The \"Promising but Failed\" Architecture: Hybrid Texture Pooling\n\nI experimented with a custom pooling layer designed to reconstruct spatial grids from ViT tokens and pool across the height dimension (simulating a scan). While the validation loss looked very promising, it did not generalize as well to the private leaderboard.\n\nPython\n\n```\nclass HybridTexturePooling(nn.Module):\n def __init__(self, embed_dim=1280, patch_size=16, num_extra_tokens=5):\n super().__init__()\n self.ps = patch_size\n self.num_extra_tokens = num_extra_tokens\n # Projection layer: maps ViT embedding to pixel-space (ps**2) * 2\n self.projection = nn.Linear(embed_dim, (patch_size ** 2) * 2)\n\n def forward(self, x, h, w):\n # x: [BS, N_tokens, Dim]\n # 1. Slice off CLS + Register tokens\n patch_tokens = x[:, self.num_extra_tokens:, :] \n \n # 2. Project to pixel-like space\n x = self.projection(patch_tokens) \n \n bs = x.shape[0]\n h_patches, w_patches = h // self.ps, w // self.ps\n \n # 3. Reconstruct Spatial Grid\n # Reshape to (BS, H_p, W_p, PS, PS, 2)\n x = x.view(bs, h_patches, w_patches, self.ps, self.ps, 2)\n # Permute to (BS, H_p, PS, W_p, PS, 2)\n x = x.permute(0, 1, 3, 2, 4, 5).contiguous()\n # Flatten to (BS, H, W * 2)\n x = x.view(bs, h, w * 2)\n \n # 4. Mean across the Height dimension\n x = x.mean(dim=1) # Result: [BS, 1600] for 800px input\n return x\n```\n\n### 5. Training Configuration\n\nWe utilized a two-stage training pipeline. Below are the specific hyperparameters used for each stage.\n\n#### **Stage 1: Main Biomass Regression**\n\n- **Model:** `vit_huge_plus_patch16_dinov3.lvd1689m` (Weights frozen for first 50% of layers)\n \n- **Image Size:** 800× 800\n \n- **Batch Size:** 10\n \n- **Optimizer:** AdamW\n \n- **Scheduler:** CosineAnnealingWarmRestarts(optimizer, T_0=10, T_mult=2, eta_min=1e-6)\n \n- **Loss Function:** `WeightedSmoothL1Loss`\n \n - Weights: `[0.1, 0.1, 0.1, 0.2, 0.5]` (Heavily penalizing Total Biomass errors)\n \n- **Validation Strategy:** 5-Fold Cross-Validation (Seed 42)\n \n\n#### **Stage 2: Auxiliary Feature Pre-training**\n\n- **Objective:** Predict Tabular Features (`NDVI`, `Height`) from Images\n \n- **Initialization:** Loaded best weights from **Stage 1 (Fold 0)**\n \n- **Batch Size:** 8\n \n- **Optimizer:** AdamW\n \n- **Scheduler:** ReduceLROnPlateau(optimizer, mode='min', factor=0.5, patience=4)\n \n- **Loss Function:** `MSELoss`\n \n- **Validation Strategy:** 5-Fold Cross-Validation (Seed 44)\n\nNote: All regression targets and auxiliary features (tabular inputs) were normalized using StandardScaler prior to training to ensure stable convergence.\n\n\nThanks again to the community! I hope this write-up and the diagrams help explain the structure of the solution.", "3398430": "Very nice work. and I have a question: during the model inference phase, do you use a one-stage model to predict NDVI and Height first, and then fuse them with image features?", "3398432": "What CV strategy do you use?", "3398434": "first aux prediction then main targets", "3398436": "normal k-fold, other strategy didn't work for me", "3398455": "Great implementation. I tried training the model directly to predict Height and NVDI but failed. \nMy hypothesis is that, learning the original targets using Height and NVDI must have forced/encouraged the model to use shared features, which then made it possible to finetune the same model to predict Height and NVDI from the images alone. Amazing. \n\nI got a few questions, if you can answer it would be great:\n1. Do you think the small 256 bottleneck dimension is the key here to force shared representations? That would verify the above (if true).\n2. You conducted manual cleanup for training data, but what about test data? If the model saw more cleaned images during training, which are not available on test, shouldn't it perform worse comparitively?\n3. Does some contrastive learning have scope here? Like training the model to match the predictions from the original train image and cleaned trained image so as to train the model to ignore those details?", "3398480": "Thanks Komil great questions.\n\nOn the 256 bottleneck, yes, 256 resolution will not capture image features (since we are doing texture level regression task), I used 800*800 image resolution.\n\nThat said, direct multi-task training didn’t completely fail for me it just underperformed. I just used my main model's weight and trained the aux model on uncleaned data.\n\nRegarding train–test mismatch due to cleaning- Even though the test data isn’t cleaned, performance didn’t degrade. My intuition is that cleaning during training actually helps the model unlearn background shortcuts. In other words, by seeing “vegetation without distractions”, the model becomes less sensitive to non-vegetative artifacts at test time rather than more.\n\nOn contrastive learning- yes, I think there’s real scope there. A setup where the model is encouraged to produce similar predictions (or embeddings) for original vs. cleaned versions of the same image could explicitly teach invariance to background noise. I didn’t explore it due to time constraints, but conceptually it fits very well with this problem.", "3398488": "Congratulations！ Very clear and elegant solution!", "3398499": "Dumb question regarding the auxilliary part. How did you leverage that to predict the actual targets?", "3398507": "By using those value in my main model, I was using mean value before training the auxiliary values while predicting main bio mass heads.\nstep 1 - input ( Image (800*800) ) AUX MODEL -> Predictions (Height,NDVI)\nstep 2- input ( Image, [Height,NDVI] ) Main Model -> Predictions ('Dry_Green_g', 'Dry_Dead_g', 'Dry_Clover_g', 'GDM_g', 'Dry_Total_g')", "3398586": "Faker is coming😆", "3399123": "So you were not using the second stage training (the one on aux feats) to perform a fine tuning of the same backbone used by the main model (the model trained in the first stage), but fine tuning two distinct instances of the backbone (one for the main model and one for the aux model), right?", "3399128": "I used main model's first fold weight for training aux model.", "3399861": "What is your final rank, can't see your team on the leaderboard?", "3399871": "Hi Amir,\n\nWe ended up in 4th place on the private leaderboard, but unfortunately, our team was removed. I’ve already sent an email to Kaggle about the issue, but I think the leaderboard results are final, so it’s unlikely the decision will be overturned." }, "source": "meta" }