{ "id": 670691, "title": "Rank 83rd Silver Solution: DINOv2/3 + Depth Fusion with Uncertainty Calibration", "url": "/competitions/csiro-biomass/discussion/670691", "author_name": "Chakkrit Termritthikun", "post_date": "2026-01-29T07:21:54.951000", "votes": 9, "comment_count": 0, "views": 0, "content": "

Rank 83rd Solution: DINOv2 + Depth Fusion with Uncertainty Calibration

\n

Author: Chakkrit Termritthikun

\n

Affiliation: School of Renewable Energy and Smart Grid Technology (SGtech), Naresuan University, Thailand

\n

Performance: Private LB: 0.62 | Public LB: 0.74

\n

\"\"

\n

1. Core Architecture: Two-Stream Fusion

\n

My solution uses a Two-Stream Fusion model (TwoStreamDINOTiled) that integrates semantic RGB features with geometric depth cues.

\n

\"\"

\n

Backbones & Encoders

\n\n
class FeatureExtractor(nn.Module):\n    def __init__(self, model_name, pretrained, **kwargs):\n        super().__init__()\n        self.model = timm.create_model(model_name, pretrained=pretrained, \n                                       num_classes=0, global_pool='', **kwargs)\n        self.num_features = self.model.num_features\n        self.num_prefix_tokens = getattr(self.model, 'num_prefix_tokens', 1)\n\n    def forward(self, x):\n        x = self.model.forward_features(x)\n        if x.dim() > 3:\n            x = x.flatten(1, x.dim()-2)\n        if self.num_prefix_tokens > 0:\n            x = x[:, self.num_prefix_tokens:, :] # Remove CLS/Registers\n        return x\n
\n

I applied LoRA (Low-Rank Adaptation) (r=4, alpha=8) to the qkv modules, allowing the model to adapt to high-resolution biomass patterns while preserving pretrained weights.

\n\n
self.depth_encoder = timm.create_model(\n    'tf_efficientnet_b0.ns_jft_in1k', \n    pretrained=True, \n    num_classes=0, \n    global_pool='avg'\n)\ndepth_dim = self.depth_encoder.num_features\n
\n
\n

Key Finding: During experimentation, I observed that using larger depth encoders increased performance on the training set but led to a decrease in Public Leaderboard (LB) scores. The tf_efficientnet_b0 (B0) variant provided the best balance, offering superior generalization on the test set.

\n \n
\n
class TargetNormalizer:\n    def fit(self, targets: np.ndarray):\n        self.mean = torch.tensor(np.mean(targets, axis=0), dtype=torch.float32)\n        self.std = torch.tensor(np.std(targets, axis=0), dtype=torch.float32) + 1e-6\n\n    def transform(self, targets: torch.Tensor):\n        return (targets - self.mean) / self.std\n\n    def inverse_transform(self, targets: torch.Tensor):\n        return targets * self.std + self.mean\n
\n\n

Gated Fusion Logic

\n
class GatedFusion(nn.Module):\n    def __init__(self, dim):\n        super().__init__()\n        self.gate = nn.Sequential(\n            nn.Linear(dim * 2, dim),\n            nn.Sigmoid()\n        )\n    def forward(self, x, y):\n        g = self.gate(torch.cat([x, y], dim=-1))\n        return g * x + (1 - g) * y\n
\n\n
class AttentionPooling(nn.Module):\n    def __init__(self, in_dim):\n        super().__init__()\n        self.attention = nn.Sequential(\n            nn.Linear(in_dim, in_dim),\n            nn.Tanh(),\n            nn.Linear(in_dim, 1)\n        )\n\n    def forward(self, x):\n        w = self.attention(x)\n        w = F.softmax(w, dim=1)\n        return torch.sum(x * w, dim=1)\n
\n\n
self.head_body = nn.Sequential(ß\n    nn.Linear(head_in_dim, 512),\n    nn.GELU(),\n    nn.Dropout(dropout),\n    nn.Linear(512, 256),\n    nn.GELU(),\n    nn.Dropout(dropout),\n)\n# Head A: The actual regression prediction (Mu)\nself.fc_mu = nn.Linear(256, num_classes)\n# Head B: The uncertainty prediction (Log Variance)\nself.fc_var = nn.Linear(256, num_classes)\n
\n

2. Advanced Training Strategy

\n

Loss: Uncertainty-Aware NLL

\n

I used a custom UncertaintyRegressionLoss designed to handle noise in biomass labels. It minimizes the Negative Log-Likelihood (NLL) with a prior regularization on the variance to prevent instability:\n$$Loss = \\frac{1}{2}\\exp(-\\log \\sigma^2) \\cdot (\\mu - y)^2 + \\frac{1}{2}\\log \\sigma^2 + \\lambda \\cdot (\\log \\sigma^2)^2$$

\n
def forward(self, mu, target, log_var):\n    log_var = torch.clamp(log_var, self.log_var_min, self.log_var_max)\n    mse = (mu - target) ** 2\n    nll = 0.5 * torch.exp(-log_var) * mse + 0.5 * log_var\n    prior = self.prior_weight * log_var ** 2\n    return torch.mean(nll + prior)\n
\n\n

Optimization & CV

\n\n
lr_head = 1e-3       # Faster learning for the fresh head\nlr_backbone = 1e-5   # Fine-tune the pre-trained weights slowly\n
\n

5-Fold Cross-Validation Strategy

\n

I employed a 5-Fold Cross-Validation strategy to ensure model stability and a reliable estimation of leaderboard performance.

\n\n
# 5-Fold Stratified Bins Logic\nnum_bins = int(np.floor(1 + np.log2(len(agg_train_df))))\nagg_train_df['bins'] = pd.qcut(agg_train_df['Dry_Total_g'], q=num_bins, labels=False)\nskf = StratifiedKFold(n_splits=5, shuffle=True, random_state=42)\n\nfor fold, (train_idx, val_idx) in enumerate(skf.split(agg_train_df, agg_train_df['bins'])):\n    agg_train_df.loc[val_idx, 'fold'] = fold\n
\n

3. Metrics & Technical Implementation Nuances

\n

Metric Weighting & Score Calculation

\n

The competition utilized a Weighted R2 Score, which I implemented directly in our validation pipeline to ensure high correlation between local OOF and Leaderboard performance.

\n
def weighted_r2_score(y_true, y_pred):\n    # Competition weights for: [Clover, Dead, Green, Total, GMD]\n    weights = np.array([0.1, 0.1, 0.1, 0.5, 0.2])\n    r2_scores = []\n    for i in range(5):\n        y_t, y_p = y_true[:, i], y_pred[:, i]\n        ss_res = np.sum((y_t - y_p) ** 2)\n        ss_tot = np.sum((y_t - np.mean(y_t)) ** 2)\n        r2_scores.append(1 - ss_res / ss_tot if ss_tot > 0 else 0.0)\n\n    r2_scores = np.array(r2_scores)\n    return np.sum(r2_scores * weights) / np.sum(weights), r2_scores\n
\n\n

3. Data Pipeline & Augmentation

\n

Two-View Spatial Strategy

\n

A key design choice in my RegressionDataset was splitting each input image into two distinct views (Left and Right halves). This served two purposes:

\n
    \n
  1. Resolution Integrity: It allowed me to process high-resolution images without scaling them down to fit the ViT's input size (518x518), effectively doubling the effective spatial resolution.
    2. \n
  2. Data Augmentation: It acts as a natural augmentation, providing the model with localized views of the pasture.
    3. \n
\n
# Image splitting in RegressionDataset\nw, h = image.size\nleft, right = image.crop((0,0,w//2,h)), image.crop((w//2,0,w,h))\nd_left, d_right = depth.crop((0,0,w//2,h)), depth.crop((w//2,0,w,h))\n
\n\n

Test-Time BN Adaptation (Domain Alignment)

\n

One of the most effective techniques used in my solution was adapting the Batch Normalization (BN) layers to the test set distribution. By running a single forward pass over the test data in .train() mode while keeping weights frozen, I updated the running_mean and running_var of the encoders.

\n

This process aligns the model's internal feature normalization with the specific lighting, sensor characteristics, and environmental conditions of the test images, which is particularly beneficial when training data is limited or collected under different conditions.

\n
def adapt_bn_to_test_set(model, test_loader, device):\n    model.train()\n    model.to(device)\n    for param in model.parameters():\n        param.requires_grad = False\n\n    with torch.no_grad():\n        for images, depth, _ in test_loader:\n            images = images.to(device)\n            depth = depth.to(device)\n            _, _ = model(images, depth)\n    model.eval()\n    return model\n
\n

Training Hyperparameters

\n

To ensure reproductive stability, I used a consistent training configuration across all folds:

\n\n

Multi-Stage Ensemble Strategy

\n

The final submission was not based on a single model but an ensemble of multiple \"snapshots\" collected across cycles and folds:

\n
    \n
  1. Averaging: I perform a simple arithmetic mean of the calibrated predictions across all saved fold/cycle checkpoints.
    2. \n
  2. Inversion: Predictions are inverted from the Z-score normalized space back to original biomass units.
    3. \n
  3. Refinement: Manual scaling and physical consistency optimization are applied at the very end of the ensemble pipeline.
    4. \n
\n
config = LoraConfig(\n    r=4,\n    lora_alpha=8,\n    target_modules=[\"qkv\"],\n    lora_dropout=0.05,\n    bias=\"none\",\n)\nself.backbone = get_peft_model(self.backbone, config)\n
\n

4. Post-Processing: The Boost to Private LB

\n

The performance jump to Rank 5 came from a multi-stage inference pipeline:

\n
    \n
  1. Test-Time BN Adaptation: Before prediction, I updated the model's Batch Normalization statistics (running_mean, running_var) by passing test data through the model in train() mode with frozen weights. This aligned the model with the test set's exposure and lighting.
    2. \n
  2. Test-Time Augmentation (TTA): 4-view averaging (Original, H-Flip, V-Flip, Rot90).
    3. \n
  3. Post-Hoc Calibration: A LinearRegression calibrator was fitted on OOF predictions to correct residual scale and bias shifts.
    4. \n
\n
class PostHocCalibrator:\n    def fit(self, preds: np.ndarray, targets: np.ndarray):\n        for i in range(preds.shape[1]):\n            reg = LinearRegression()\n            reg.fit(preds[:, i].reshape(-1, 1), targets[:, i])\n            self.models.append(reg)\n\n    def predict(self, preds: np.ndarray):\n        calibrated = [reg.predict(preds[:, i].reshape(-1, 1)) \n                      for i, reg in enumerate(self.models)]\n        return np.stack(calibrated, axis=1)\n
\n
    \n
  1. Manual Scaling Logic:
    2. \n
  2. Physical Consistency Optimization: Enforced the law of parts-to-whole using scipy.optimize.minimize (L-BFGS-B). I trusted the Total and GDM heads more by applying a weight of 1.2 compared to 1.0 for individual components:
    3. \n
\n
def objective(x):\n    clover, dead, green = x\n    total = clover + dead + green\n    gdm = clover + green\n    x_full = np.array([clover, dead, green, total, gdm])\n    return np.sum(weights * (x_full - preds) ** 2)\n
\n

5. Final Ensemble Strategy

\n

My top-performing submission was a robust ensemble of two distinct model configurations, blending global context with fine-grained local details:

\n
    \n
  1. DINOv2 Large Model:
    2. \n
  2. DINOv3 Large Model:
    3. \n
\n

By combining the DINOv2 (focused on high-resolution global structure) with the DINOv3 (leveraging multi-tile local features).

\n

Acknowledgments

\n

I would like to express my sincere gratitude to the competition organizers for hosting such a challenging and meaningful event. I also want to thank the other teams and the community; many of the ideas and implementation strategies in this solution were inspired by their insights and shared knowledge.

", "messages": [ { "id": 3398465, "postDate": "2026-01-29T07:21:54.950Z", "content": "

Rank 83rd Solution: DINOv2 + Depth Fusion with Uncertainty Calibration

\n

Author: Chakkrit Termritthikun

\n

Affiliation: School of Renewable Energy and Smart Grid Technology (SGtech), Naresuan University, Thailand

\n

Performance: Private LB: 0.62 | Public LB: 0.74

\n

\"\"

\n

1. Core Architecture: Two-Stream Fusion

\n

My solution uses a Two-Stream Fusion model (TwoStreamDINOTiled) that integrates semantic RGB features with geometric depth cues.

\n

\"\"

\n

Backbones & Encoders

\n\n
class FeatureExtractor(nn.Module):\n    def __init__(self, model_name, pretrained, **kwargs):\n        super().__init__()\n        self.model = timm.create_model(model_name, pretrained=pretrained, \n                                       num_classes=0, global_pool='', **kwargs)\n        self.num_features = self.model.num_features\n        self.num_prefix_tokens = getattr(self.model, 'num_prefix_tokens', 1)\n\n    def forward(self, x):\n        x = self.model.forward_features(x)\n        if x.dim() > 3:\n            x = x.flatten(1, x.dim()-2)\n        if self.num_prefix_tokens > 0:\n            x = x[:, self.num_prefix_tokens:, :] # Remove CLS/Registers\n        return x\n
\n

I applied LoRA (Low-Rank Adaptation) (r=4, alpha=8) to the qkv modules, allowing the model to adapt to high-resolution biomass patterns while preserving pretrained weights.

\n\n
self.depth_encoder = timm.create_model(\n    'tf_efficientnet_b0.ns_jft_in1k', \n    pretrained=True, \n    num_classes=0, \n    global_pool='avg'\n)\ndepth_dim = self.depth_encoder.num_features\n
\n
\n

Key Finding: During experimentation, I observed that using larger depth encoders increased performance on the training set but led to a decrease in Public Leaderboard (LB) scores. The tf_efficientnet_b0 (B0) variant provided the best balance, offering superior generalization on the test set.

\n \n
\n
class TargetNormalizer:\n    def fit(self, targets: np.ndarray):\n        self.mean = torch.tensor(np.mean(targets, axis=0), dtype=torch.float32)\n        self.std = torch.tensor(np.std(targets, axis=0), dtype=torch.float32) + 1e-6\n\n    def transform(self, targets: torch.Tensor):\n        return (targets - self.mean) / self.std\n\n    def inverse_transform(self, targets: torch.Tensor):\n        return targets * self.std + self.mean\n
\n\n

Gated Fusion Logic

\n
class GatedFusion(nn.Module):\n    def __init__(self, dim):\n        super().__init__()\n        self.gate = nn.Sequential(\n            nn.Linear(dim * 2, dim),\n            nn.Sigmoid()\n        )\n    def forward(self, x, y):\n        g = self.gate(torch.cat([x, y], dim=-1))\n        return g * x + (1 - g) * y\n
\n\n
class AttentionPooling(nn.Module):\n    def __init__(self, in_dim):\n        super().__init__()\n        self.attention = nn.Sequential(\n            nn.Linear(in_dim, in_dim),\n            nn.Tanh(),\n            nn.Linear(in_dim, 1)\n        )\n\n    def forward(self, x):\n        w = self.attention(x)\n        w = F.softmax(w, dim=1)\n        return torch.sum(x * w, dim=1)\n
\n\n
self.head_body = nn.Sequential(ß\n    nn.Linear(head_in_dim, 512),\n    nn.GELU(),\n    nn.Dropout(dropout),\n    nn.Linear(512, 256),\n    nn.GELU(),\n    nn.Dropout(dropout),\n)\n# Head A: The actual regression prediction (Mu)\nself.fc_mu = nn.Linear(256, num_classes)\n# Head B: The uncertainty prediction (Log Variance)\nself.fc_var = nn.Linear(256, num_classes)\n
\n

2. Advanced Training Strategy

\n

Loss: Uncertainty-Aware NLL

\n

I used a custom UncertaintyRegressionLoss designed to handle noise in biomass labels. It minimizes the Negative Log-Likelihood (NLL) with a prior regularization on the variance to prevent instability:\n$$Loss = \\frac{1}{2}\\exp(-\\log \\sigma^2) \\cdot (\\mu - y)^2 + \\frac{1}{2}\\log \\sigma^2 + \\lambda \\cdot (\\log \\sigma^2)^2$$

\n
def forward(self, mu, target, log_var):\n    log_var = torch.clamp(log_var, self.log_var_min, self.log_var_max)\n    mse = (mu - target) ** 2\n    nll = 0.5 * torch.exp(-log_var) * mse + 0.5 * log_var\n    prior = self.prior_weight * log_var ** 2\n    return torch.mean(nll + prior)\n
\n\n

Optimization & CV

\n\n
lr_head = 1e-3       # Faster learning for the fresh head\nlr_backbone = 1e-5   # Fine-tune the pre-trained weights slowly\n
\n

5-Fold Cross-Validation Strategy

\n

I employed a 5-Fold Cross-Validation strategy to ensure model stability and a reliable estimation of leaderboard performance.

\n\n
# 5-Fold Stratified Bins Logic\nnum_bins = int(np.floor(1 + np.log2(len(agg_train_df))))\nagg_train_df['bins'] = pd.qcut(agg_train_df['Dry_Total_g'], q=num_bins, labels=False)\nskf = StratifiedKFold(n_splits=5, shuffle=True, random_state=42)\n\nfor fold, (train_idx, val_idx) in enumerate(skf.split(agg_train_df, agg_train_df['bins'])):\n    agg_train_df.loc[val_idx, 'fold'] = fold\n
\n

3. Metrics & Technical Implementation Nuances

\n

Metric Weighting & Score Calculation

\n

The competition utilized a Weighted R2 Score, which I implemented directly in our validation pipeline to ensure high correlation between local OOF and Leaderboard performance.

\n
def weighted_r2_score(y_true, y_pred):\n    # Competition weights for: [Clover, Dead, Green, Total, GMD]\n    weights = np.array([0.1, 0.1, 0.1, 0.5, 0.2])\n    r2_scores = []\n    for i in range(5):\n        y_t, y_p = y_true[:, i], y_pred[:, i]\n        ss_res = np.sum((y_t - y_p) ** 2)\n        ss_tot = np.sum((y_t - np.mean(y_t)) ** 2)\n        r2_scores.append(1 - ss_res / ss_tot if ss_tot > 0 else 0.0)\n\n    r2_scores = np.array(r2_scores)\n    return np.sum(r2_scores * weights) / np.sum(weights), r2_scores\n
\n\n

3. Data Pipeline & Augmentation

\n

Two-View Spatial Strategy

\n

A key design choice in my RegressionDataset was splitting each input image into two distinct views (Left and Right halves). This served two purposes:

\n
    \n
  1. Resolution Integrity: It allowed me to process high-resolution images without scaling them down to fit the ViT's input size (518x518), effectively doubling the effective spatial resolution.
    2. \n
  2. Data Augmentation: It acts as a natural augmentation, providing the model with localized views of the pasture.
    3. \n
\n
# Image splitting in RegressionDataset\nw, h = image.size\nleft, right = image.crop((0,0,w//2,h)), image.crop((w//2,0,w,h))\nd_left, d_right = depth.crop((0,0,w//2,h)), depth.crop((w//2,0,w,h))\n
\n\n

Test-Time BN Adaptation (Domain Alignment)

\n

One of the most effective techniques used in my solution was adapting the Batch Normalization (BN) layers to the test set distribution. By running a single forward pass over the test data in .train() mode while keeping weights frozen, I updated the running_mean and running_var of the encoders.

\n

This process aligns the model's internal feature normalization with the specific lighting, sensor characteristics, and environmental conditions of the test images, which is particularly beneficial when training data is limited or collected under different conditions.

\n
def adapt_bn_to_test_set(model, test_loader, device):\n    model.train()\n    model.to(device)\n    for param in model.parameters():\n        param.requires_grad = False\n\n    with torch.no_grad():\n        for images, depth, _ in test_loader:\n            images = images.to(device)\n            depth = depth.to(device)\n            _, _ = model(images, depth)\n    model.eval()\n    return model\n
\n

Training Hyperparameters

\n

To ensure reproductive stability, I used a consistent training configuration across all folds:

\n\n

Multi-Stage Ensemble Strategy

\n

The final submission was not based on a single model but an ensemble of multiple \"snapshots\" collected across cycles and folds:

\n
    \n
  1. Averaging: I perform a simple arithmetic mean of the calibrated predictions across all saved fold/cycle checkpoints.
    2. \n
  2. Inversion: Predictions are inverted from the Z-score normalized space back to original biomass units.
    3. \n
  3. Refinement: Manual scaling and physical consistency optimization are applied at the very end of the ensemble pipeline.
    4. \n
\n
config = LoraConfig(\n    r=4,\n    lora_alpha=8,\n    target_modules=[\"qkv\"],\n    lora_dropout=0.05,\n    bias=\"none\",\n)\nself.backbone = get_peft_model(self.backbone, config)\n
\n

4. Post-Processing: The Boost to Private LB

\n

The performance jump to Rank 5 came from a multi-stage inference pipeline:

\n
    \n
  1. Test-Time BN Adaptation: Before prediction, I updated the model's Batch Normalization statistics (running_mean, running_var) by passing test data through the model in train() mode with frozen weights. This aligned the model with the test set's exposure and lighting.
    2. \n
  2. Test-Time Augmentation (TTA): 4-view averaging (Original, H-Flip, V-Flip, Rot90).
    3. \n
  3. Post-Hoc Calibration: A LinearRegression calibrator was fitted on OOF predictions to correct residual scale and bias shifts.
    4. \n
\n
class PostHocCalibrator:\n    def fit(self, preds: np.ndarray, targets: np.ndarray):\n        for i in range(preds.shape[1]):\n            reg = LinearRegression()\n            reg.fit(preds[:, i].reshape(-1, 1), targets[:, i])\n            self.models.append(reg)\n\n    def predict(self, preds: np.ndarray):\n        calibrated = [reg.predict(preds[:, i].reshape(-1, 1)) \n                      for i, reg in enumerate(self.models)]\n        return np.stack(calibrated, axis=1)\n
\n
    \n
  1. Manual Scaling Logic:
    2. \n
  2. Physical Consistency Optimization: Enforced the law of parts-to-whole using scipy.optimize.minimize (L-BFGS-B). I trusted the Total and GDM heads more by applying a weight of 1.2 compared to 1.0 for individual components:
    3. \n
\n
def objective(x):\n    clover, dead, green = x\n    total = clover + dead + green\n    gdm = clover + green\n    x_full = np.array([clover, dead, green, total, gdm])\n    return np.sum(weights * (x_full - preds) ** 2)\n
\n

5. Final Ensemble Strategy

\n

My top-performing submission was a robust ensemble of two distinct model configurations, blending global context with fine-grained local details:

\n
    \n
  1. DINOv2 Large Model:
    2. \n
  2. DINOv3 Large Model:
    3. \n
\n

By combining the DINOv2 (focused on high-resolution global structure) with the DINOv3 (leveraging multi-tile local features).

\n

Acknowledgments

\n

I would like to express my sincere gratitude to the competition organizers for hosting such a challenging and meaningful event. I also want to thank the other teams and the community; many of the ideas and implementation strategies in this solution were inspired by their insights and shared knowledge.

", "rawMarkdown": "# Rank 83rd Solution: DINOv2 + Depth Fusion with Uncertainty Calibration\n\n### **Author:** Chakkrit Termritthikun \n### **Affiliation:** School of Renewable Energy and Smart Grid Technology (SGtech), Naresuan University, Thailand\n### **Performance:** Private LB: 0.62 | Public LB: 0.74\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F627068%2Fa81e4629ce921266664d71606adba10a%2Fkaggle.jpg?generation=1769669773305743&alt=media)\n\n## 1. Core Architecture: Two-Stream Fusion\nMy solution uses a **Two-Stream Fusion** model (`TwoStreamDINOTiled`) that integrates semantic RGB features with geometric depth cues.\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F627068%2F1cf565db00eae4ccfbf70f9e4d242088%2Fkaggle3.jpg?generation=1769670299583621&alt=media)\n\n### Backbones & Encoders\n* **RGB Stream:** `vit_large_patch14_dinov2.lvd142m` is used as the backbone via a custom `FeatureExtractor` to handle prefix tokens and spatial flattening.\n\n```python\nclass FeatureExtractor(nn.Module):\n def __init__(self, model_name, pretrained, **kwargs):\n super().__init__()\n self.model = timm.create_model(model_name, pretrained=pretrained, \n num_classes=0, global_pool='', **kwargs)\n self.num_features = self.model.num_features\n self.num_prefix_tokens = getattr(self.model, 'num_prefix_tokens', 1)\n\n def forward(self, x):\n x = self.model.forward_features(x)\n if x.dim() > 3:\n x = x.flatten(1, x.dim()-2)\n if self.num_prefix_tokens > 0:\n x = x[:, self.num_prefix_tokens:, :] # Remove CLS/Registers\n return x\n```\n\nI applied **LoRA (Low-Rank Adaptation)** (r=4, alpha=8) to the `qkv` modules, allowing the model to adapt to high-resolution biomass patterns while preserving pretrained weights.\n* **Depth Stream:** Pre-calculated depth maps are processed via a `tf_efficientnet_b0` encoder initialized with `global_pool='avg'`.\n\n```python\nself.depth_encoder = timm.create_model(\n 'tf_efficientnet_b0.ns_jft_in1k', \n pretrained=True, \n num_classes=0, \n global_pool='avg'\n)\ndepth_dim = self.depth_encoder.num_features\n```\n> **Key Finding:** During experimentation, I observed that using larger depth encoders increased performance on the training set but led to a decrease in Public Leaderboard (LB) scores. The `tf_efficientnet_b0` (B0) variant provided the best balance, offering superior generalization on the test set.\n* **Target Normalizer:** Targets were Z-score normalized during training and inverted using a stored `TargetNormalizer` during inference.\n\n```python\nclass TargetNormalizer:\n def fit(self, targets: np.ndarray):\n self.mean = torch.tensor(np.mean(targets, axis=0), dtype=torch.float32)\n self.std = torch.tensor(np.std(targets, axis=0), dtype=torch.float32) + 1e-6\n\n def transform(self, targets: torch.Tensor):\n return (targets - self.mean) / self.std\n\n def inverse_transform(self, targets: torch.Tensor):\n return targets * self.std + self.mean\n```\n* **Fusion Mechanism:** Features are combined through **Gated Fusion**. A learned sigmoid gate determines the contribution ratio between RGB features and projected depth features:\n $$F_{fuse} = \\text{gate} \\cdot F_{RGB} + (1 - \\text{gate}) \\cdot \\text{Proj}(F_{Depth})$$\n### Gated Fusion Logic\n\n```python\nclass GatedFusion(nn.Module):\n def __init__(self, dim):\n super().__init__()\n self.gate = nn.Sequential(\n nn.Linear(dim * 2, dim),\n nn.Sigmoid()\n )\n def forward(self, x, y):\n g = self.gate(torch.cat([x, y], dim=-1))\n return g * x + (1 - g) * y\n```\n* **Attention Pooling:** Instead of simple global averaging, I used **Attention Pooling** to aggregate spatial tokens, ensuring the model focuses on biomass-dense patches.\n\n```python\nclass AttentionPooling(nn.Module):\n def __init__(self, in_dim):\n super().__init__()\n self.attention = nn.Sequential(\n nn.Linear(in_dim, in_dim),\n nn.Tanh(),\n nn.Linear(in_dim, 1)\n )\n\n def forward(self, x):\n w = self.attention(x)\n w = F.softmax(w, dim=1)\n return torch.sum(x * w, dim=1)\n```\n* **Dual-Head Regression:** The model predicts two outputs per target:\n 1. **Mu:** The predicted biomass value (in normalized space).\n 2. **Log-Variance:** The predicted aleatoric uncertainty.\n\n```python\nself.head_body = nn.Sequential(ß\n nn.Linear(head_in_dim, 512),\n nn.GELU(),\n nn.Dropout(dropout),\n nn.Linear(512, 256),\n nn.GELU(),\n nn.Dropout(dropout),\n)\n# Head A: The actual regression prediction (Mu)\nself.fc_mu = nn.Linear(256, num_classes)\n# Head B: The uncertainty prediction (Log Variance)\nself.fc_var = nn.Linear(256, num_classes)\n```\n\n## 2. Advanced Training Strategy\n### Loss: Uncertainty-Aware NLL\nI used a custom `UncertaintyRegressionLoss` designed to handle noise in biomass labels. It minimizes the Negative Log-Likelihood (NLL) with a prior regularization on the variance to prevent instability:\n$$Loss = \\frac{1}{2}\\exp(-\\log \\sigma^2) \\cdot (\\mu - y)^2 + \\frac{1}{2}\\log \\sigma^2 + \\lambda \\cdot (\\log \\sigma^2)^2$$\n\n```python\ndef forward(self, mu, target, log_var):\n log_var = torch.clamp(log_var, self.log_var_min, self.log_var_max)\n mse = (mu - target) ** 2\n nll = 0.5 * torch.exp(-log_var) * mse + 0.5 * log_var\n prior = self.prior_weight * log_var ** 2\n return torch.mean(nll + prior)\n```\n* `log_var` is clamped to [-7, 3] for stability.\n* `prior_weight` (lambda) is set to `5e-3`.\n\n### Optimization & CV\n* **Snapshot Ensembling:** I utilized `CosineAnnealingWarmRestarts` with (2 cycles). This periodic reset of the learning rate allowed the model to escape local minima and capture highly diverse \"snapshots\" which were then saved as separate checkpoints for each cycle.\n* **Differential Learning Rates:** I used distinct learning rates for different parts of the architecture. The feature extractors (backbone and depth encoder) were trained at a much lower rate to preserve pretrained features, while the regression head used a higher rate.\n\n```python\nlr_head = 1e-3 # Faster learning for the fresh head\nlr_backbone = 1e-5 # Fine-tune the pre-trained weights slowly\n```\n### 5-Fold Cross-Validation Strategy\nI employed a **5-Fold Cross-Validation** strategy to ensure model stability and a reliable estimation of leaderboard performance. \n* **NFOLD:** Set to 5.\n* **Stratified Bins CV:** To handle the skewed biomass distribution, I used `pd.qcut` on the `Dry_Total_g` target to create balanced bins. This ensures that every fold is representative of the entire population's biomass range.\n\n```python\n# 5-Fold Stratified Bins Logic\nnum_bins = int(np.floor(1 + np.log2(len(agg_train_df))))\nagg_train_df['bins'] = pd.qcut(agg_train_df['Dry_Total_g'], q=num_bins, labels=False)\nskf = StratifiedKFold(n_splits=5, shuffle=True, random_state=42)\n\nfor fold, (train_idx, val_idx) in enumerate(skf.split(agg_train_df, agg_train_df['bins'])):\n agg_train_df.loc[val_idx, 'fold'] = fold\n```\n\n## 3. Metrics & Technical Implementation Nuances\n\n### Metric Weighting & Score Calculation\nThe competition utilized a **Weighted R2 Score**, which I implemented directly in our validation pipeline to ensure high correlation between local OOF and Leaderboard performance.\n\n```python\ndef weighted_r2_score(y_true, y_pred):\n # Competition weights for: [Clover, Dead, Green, Total, GMD]\n weights = np.array([0.1, 0.1, 0.1, 0.5, 0.2])\n r2_scores = []\n for i in range(5):\n y_t, y_p = y_true[:, i], y_pred[:, i]\n ss_res = np.sum((y_t - y_p) ** 2)\n ss_tot = np.sum((y_t - np.mean(y_t)) ** 2)\n r2_scores.append(1 - ss_res / ss_tot if ss_tot > 0 else 0.0)\n \n r2_scores = np.array(r2_scores)\n return np.sum(r2_scores * weights) / np.sum(weights), r2_scores\n```\n* **Strategic Focus:** The heavy 0.5 weight on `Dry_Total_g` informed our physical consistency logic to trust the \"Total\" predictions more during the final optimization step.\n\n### 3. Data Pipeline & Augmentation\n\n### Two-View Spatial Strategy\nA key design choice in my `RegressionDataset` was splitting each input image into two distinct views (Left and Right halves). This served two purposes:\n1. **Resolution Integrity:** It allowed me to process high-resolution images without scaling them down to fit the ViT's input size (518x518), effectively doubling the effective spatial resolution.\n2. **Data Augmentation:** It acts as a natural augmentation, providing the model with localized views of the pasture.\n\n```python\n# Image splitting in RegressionDataset\nw, h = image.size\nleft, right = image.crop((0,0,w//2,h)), image.crop((w//2,0,w,h))\nd_left, d_right = depth.crop((0,0,w//2,h)), depth.crop((w//2,0,w,h))\n```\n\n* **16-bit Depth Handling:** Depth values normalized from [0, 65535] to [0, 1].\n### Test-Time BN Adaptation (Domain Alignment)\nOne of the most effective techniques used in my solution was adapting the Batch Normalization (BN) layers to the test set distribution. By running a single forward pass over the test data in `.train()` mode while keeping weights frozen, I updated the `running_mean` and `running_var` of the encoders. \n\nThis process aligns the model's internal feature normalization with the specific lighting, sensor characteristics, and environmental conditions of the test images, which is particularly beneficial when training data is limited or collected under different conditions.\n\n```python\ndef adapt_bn_to_test_set(model, test_loader, device):\n model.train()\n model.to(device)\n for param in model.parameters():\n param.requires_grad = False\n\n with torch.no_grad():\n for images, depth, _ in test_loader:\n images = images.to(device)\n depth = depth.to(device)\n _, _ = model(images, depth)\n model.eval()\n return model\n```\n\n### Training Hyperparameters\nTo ensure reproductive stability, I used a consistent training configuration across all folds:\n* **Optimizer:** `AdamW` with `weight_decay = 1e-4`.\n* **Scheduler:** `CosineAnnealingWarmRestarts` (2 cycles).\n* **Batch Size:** 32 (standard) or adjusted based on available GPU memory.\n* **Epochs:** 100 per fold, saving the best checkpoint for each annealing cycle.\n* **Mixed Precision:** `torch.amp.autocast` (FP16) enabled to speed up processing and reduce memory usage.\n\n### Multi-Stage Ensemble Strategy\nThe final submission was not based on a single model but an ensemble of multiple \"snapshots\" collected across cycles and folds:\n1. **Averaging:** I perform a simple arithmetic mean of the calibrated predictions across all saved fold/cycle checkpoints.\n2. **Inversion:** Predictions are inverted from the Z-score normalized space back to original biomass units.\n3. **Refinement:** Manual scaling and physical consistency optimization are applied at the very end of the ensemble pipeline.\n\n```python\nconfig = LoraConfig(\n r=4,\n lora_alpha=8,\n target_modules=[\"qkv\"],\n lora_dropout=0.05,\n bias=\"none\",\n)\nself.backbone = get_peft_model(self.backbone, config)\n```\n\n## 4. Post-Processing: The Boost to Private LB\nThe performance jump to Rank 5 came from a multi-stage inference pipeline:\n\n1. **Test-Time BN Adaptation:** Before prediction, I updated the model's Batch Normalization statistics (`running_mean`, `running_var`) by passing test data through the model in `train()` mode with frozen weights. This aligned the model with the test set's exposure and lighting.\n2. **Test-Time Augmentation (TTA):** 4-view averaging (Original, H-Flip, V-Flip, Rot90).\n3. **Post-Hoc Calibration:** A `LinearRegression` calibrator was fitted on OOF predictions to correct residual scale and bias shifts.\n\n```python\nclass PostHocCalibrator:\n def fit(self, preds: np.ndarray, targets: np.ndarray):\n for i in range(preds.shape[1]):\n reg = LinearRegression()\n reg.fit(preds[:, i].reshape(-1, 1), targets[:, i])\n self.models.append(reg)\n\n def predict(self, preds: np.ndarray):\n calibrated = [reg.predict(preds[:, i].reshape(-1, 1)) \n for i, reg in enumerate(self.models)]\n return np.stack(calibrated, axis=1)\n```\n4. **Manual Scaling Logic:**\n * `Dry_Clover_g` scaled by `0.855`.\n * `Dry_Dead_g` adjusted upwards (+15.5%) if > 20 and downwards (-4.5%) if < 10.\n5. **Physical Consistency Optimization:** Enforced the law of parts-to-whole using `scipy.optimize.minimize` (L-BFGS-B). I trusted the `Total` and `GDM` heads more by applying a weight of **1.2** compared to **1.0** for individual components:\n\n```python\ndef objective(x):\n clover, dead, green = x\n total = clover + dead + green\n gdm = clover + green\n x_full = np.array([clover, dead, green, total, gdm])\n return np.sum(weights * (x_full - preds) ** 2)\n```\n\n## 5. Final Ensemble Strategy\nMy top-performing submission was a robust ensemble of two distinct model configurations, blending global context with fine-grained local details:\n\n1. **DINOv2 Large Model:** \n * **Resolution:** High-resolution 518x518 input.\n * **Tiling:** 2-Tile split (Left/Right) to preserve structural continuity while doubling effective resolution.\n2. **DINOv3 Large Model:** \n * **Resolution:** 256x256 patches.\n * **Tiling:** Aggressive 8-Tile split to capture high-frequency texture and localized biomass density variations.\n\nBy combining the **DINOv2** (focused on high-resolution global structure) with the **DINOv3** (leveraging multi-tile local features).\n\n## Acknowledgments\nI would like to express my sincere gratitude to the competition organizers for hosting such a challenging and meaningful event. I also want to thank the other teams and the community; many of the ideas and implementation strategies in this solution were inspired by their insights and shared knowledge.", "votes": 9 } ], "comments": [], "raw_markdown_by_id": { "3398465": "# Rank 83rd Solution: DINOv2 + Depth Fusion with Uncertainty Calibration\n\n### **Author:** Chakkrit Termritthikun \n### **Affiliation:** School of Renewable Energy and Smart Grid Technology (SGtech), Naresuan University, Thailand\n### **Performance:** Private LB: 0.62 | Public LB: 0.74\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F627068%2Fa81e4629ce921266664d71606adba10a%2Fkaggle.jpg?generation=1769669773305743&alt=media)\n\n## 1. Core Architecture: Two-Stream Fusion\nMy solution uses a **Two-Stream Fusion** model (`TwoStreamDINOTiled`) that integrates semantic RGB features with geometric depth cues.\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F627068%2F1cf565db00eae4ccfbf70f9e4d242088%2Fkaggle3.jpg?generation=1769670299583621&alt=media)\n\n### Backbones & Encoders\n* **RGB Stream:** `vit_large_patch14_dinov2.lvd142m` is used as the backbone via a custom `FeatureExtractor` to handle prefix tokens and spatial flattening.\n\n```python\nclass FeatureExtractor(nn.Module):\n def __init__(self, model_name, pretrained, **kwargs):\n super().__init__()\n self.model = timm.create_model(model_name, pretrained=pretrained, \n num_classes=0, global_pool='', **kwargs)\n self.num_features = self.model.num_features\n self.num_prefix_tokens = getattr(self.model, 'num_prefix_tokens', 1)\n\n def forward(self, x):\n x = self.model.forward_features(x)\n if x.dim() > 3:\n x = x.flatten(1, x.dim()-2)\n if self.num_prefix_tokens > 0:\n x = x[:, self.num_prefix_tokens:, :] # Remove CLS/Registers\n return x\n```\n\nI applied **LoRA (Low-Rank Adaptation)** (r=4, alpha=8) to the `qkv` modules, allowing the model to adapt to high-resolution biomass patterns while preserving pretrained weights.\n* **Depth Stream:** Pre-calculated depth maps are processed via a `tf_efficientnet_b0` encoder initialized with `global_pool='avg'`.\n\n```python\nself.depth_encoder = timm.create_model(\n 'tf_efficientnet_b0.ns_jft_in1k', \n pretrained=True, \n num_classes=0, \n global_pool='avg'\n)\ndepth_dim = self.depth_encoder.num_features\n```\n> **Key Finding:** During experimentation, I observed that using larger depth encoders increased performance on the training set but led to a decrease in Public Leaderboard (LB) scores. The `tf_efficientnet_b0` (B0) variant provided the best balance, offering superior generalization on the test set.\n* **Target Normalizer:** Targets were Z-score normalized during training and inverted using a stored `TargetNormalizer` during inference.\n\n```python\nclass TargetNormalizer:\n def fit(self, targets: np.ndarray):\n self.mean = torch.tensor(np.mean(targets, axis=0), dtype=torch.float32)\n self.std = torch.tensor(np.std(targets, axis=0), dtype=torch.float32) + 1e-6\n\n def transform(self, targets: torch.Tensor):\n return (targets - self.mean) / self.std\n\n def inverse_transform(self, targets: torch.Tensor):\n return targets * self.std + self.mean\n```\n* **Fusion Mechanism:** Features are combined through **Gated Fusion**. A learned sigmoid gate determines the contribution ratio between RGB features and projected depth features:\n $$F_{fuse} = \\text{gate} \\cdot F_{RGB} + (1 - \\text{gate}) \\cdot \\text{Proj}(F_{Depth})$$\n### Gated Fusion Logic\n\n```python\nclass GatedFusion(nn.Module):\n def __init__(self, dim):\n super().__init__()\n self.gate = nn.Sequential(\n nn.Linear(dim * 2, dim),\n nn.Sigmoid()\n )\n def forward(self, x, y):\n g = self.gate(torch.cat([x, y], dim=-1))\n return g * x + (1 - g) * y\n```\n* **Attention Pooling:** Instead of simple global averaging, I used **Attention Pooling** to aggregate spatial tokens, ensuring the model focuses on biomass-dense patches.\n\n```python\nclass AttentionPooling(nn.Module):\n def __init__(self, in_dim):\n super().__init__()\n self.attention = nn.Sequential(\n nn.Linear(in_dim, in_dim),\n nn.Tanh(),\n nn.Linear(in_dim, 1)\n )\n\n def forward(self, x):\n w = self.attention(x)\n w = F.softmax(w, dim=1)\n return torch.sum(x * w, dim=1)\n```\n* **Dual-Head Regression:** The model predicts two outputs per target:\n 1. **Mu:** The predicted biomass value (in normalized space).\n 2. **Log-Variance:** The predicted aleatoric uncertainty.\n\n```python\nself.head_body = nn.Sequential(ß\n nn.Linear(head_in_dim, 512),\n nn.GELU(),\n nn.Dropout(dropout),\n nn.Linear(512, 256),\n nn.GELU(),\n nn.Dropout(dropout),\n)\n# Head A: The actual regression prediction (Mu)\nself.fc_mu = nn.Linear(256, num_classes)\n# Head B: The uncertainty prediction (Log Variance)\nself.fc_var = nn.Linear(256, num_classes)\n```\n\n## 2. Advanced Training Strategy\n### Loss: Uncertainty-Aware NLL\nI used a custom `UncertaintyRegressionLoss` designed to handle noise in biomass labels. It minimizes the Negative Log-Likelihood (NLL) with a prior regularization on the variance to prevent instability:\n$$Loss = \\frac{1}{2}\\exp(-\\log \\sigma^2) \\cdot (\\mu - y)^2 + \\frac{1}{2}\\log \\sigma^2 + \\lambda \\cdot (\\log \\sigma^2)^2$$\n\n```python\ndef forward(self, mu, target, log_var):\n log_var = torch.clamp(log_var, self.log_var_min, self.log_var_max)\n mse = (mu - target) ** 2\n nll = 0.5 * torch.exp(-log_var) * mse + 0.5 * log_var\n prior = self.prior_weight * log_var ** 2\n return torch.mean(nll + prior)\n```\n* `log_var` is clamped to [-7, 3] for stability.\n* `prior_weight` (lambda) is set to `5e-3`.\n\n### Optimization & CV\n* **Snapshot Ensembling:** I utilized `CosineAnnealingWarmRestarts` with (2 cycles). This periodic reset of the learning rate allowed the model to escape local minima and capture highly diverse \"snapshots\" which were then saved as separate checkpoints for each cycle.\n* **Differential Learning Rates:** I used distinct learning rates for different parts of the architecture. The feature extractors (backbone and depth encoder) were trained at a much lower rate to preserve pretrained features, while the regression head used a higher rate.\n\n```python\nlr_head = 1e-3 # Faster learning for the fresh head\nlr_backbone = 1e-5 # Fine-tune the pre-trained weights slowly\n```\n### 5-Fold Cross-Validation Strategy\nI employed a **5-Fold Cross-Validation** strategy to ensure model stability and a reliable estimation of leaderboard performance. \n* **NFOLD:** Set to 5.\n* **Stratified Bins CV:** To handle the skewed biomass distribution, I used `pd.qcut` on the `Dry_Total_g` target to create balanced bins. This ensures that every fold is representative of the entire population's biomass range.\n\n```python\n# 5-Fold Stratified Bins Logic\nnum_bins = int(np.floor(1 + np.log2(len(agg_train_df))))\nagg_train_df['bins'] = pd.qcut(agg_train_df['Dry_Total_g'], q=num_bins, labels=False)\nskf = StratifiedKFold(n_splits=5, shuffle=True, random_state=42)\n\nfor fold, (train_idx, val_idx) in enumerate(skf.split(agg_train_df, agg_train_df['bins'])):\n agg_train_df.loc[val_idx, 'fold'] = fold\n```\n\n## 3. Metrics & Technical Implementation Nuances\n\n### Metric Weighting & Score Calculation\nThe competition utilized a **Weighted R2 Score**, which I implemented directly in our validation pipeline to ensure high correlation between local OOF and Leaderboard performance.\n\n```python\ndef weighted_r2_score(y_true, y_pred):\n # Competition weights for: [Clover, Dead, Green, Total, GMD]\n weights = np.array([0.1, 0.1, 0.1, 0.5, 0.2])\n r2_scores = []\n for i in range(5):\n y_t, y_p = y_true[:, i], y_pred[:, i]\n ss_res = np.sum((y_t - y_p) ** 2)\n ss_tot = np.sum((y_t - np.mean(y_t)) ** 2)\n r2_scores.append(1 - ss_res / ss_tot if ss_tot > 0 else 0.0)\n \n r2_scores = np.array(r2_scores)\n return np.sum(r2_scores * weights) / np.sum(weights), r2_scores\n```\n* **Strategic Focus:** The heavy 0.5 weight on `Dry_Total_g` informed our physical consistency logic to trust the \"Total\" predictions more during the final optimization step.\n\n### 3. Data Pipeline & Augmentation\n\n### Two-View Spatial Strategy\nA key design choice in my `RegressionDataset` was splitting each input image into two distinct views (Left and Right halves). This served two purposes:\n1. **Resolution Integrity:** It allowed me to process high-resolution images without scaling them down to fit the ViT's input size (518x518), effectively doubling the effective spatial resolution.\n2. **Data Augmentation:** It acts as a natural augmentation, providing the model with localized views of the pasture.\n\n```python\n# Image splitting in RegressionDataset\nw, h = image.size\nleft, right = image.crop((0,0,w//2,h)), image.crop((w//2,0,w,h))\nd_left, d_right = depth.crop((0,0,w//2,h)), depth.crop((w//2,0,w,h))\n```\n\n* **16-bit Depth Handling:** Depth values normalized from [0, 65535] to [0, 1].\n### Test-Time BN Adaptation (Domain Alignment)\nOne of the most effective techniques used in my solution was adapting the Batch Normalization (BN) layers to the test set distribution. By running a single forward pass over the test data in `.train()` mode while keeping weights frozen, I updated the `running_mean` and `running_var` of the encoders. \n\nThis process aligns the model's internal feature normalization with the specific lighting, sensor characteristics, and environmental conditions of the test images, which is particularly beneficial when training data is limited or collected under different conditions.\n\n```python\ndef adapt_bn_to_test_set(model, test_loader, device):\n model.train()\n model.to(device)\n for param in model.parameters():\n param.requires_grad = False\n\n with torch.no_grad():\n for images, depth, _ in test_loader:\n images = images.to(device)\n depth = depth.to(device)\n _, _ = model(images, depth)\n model.eval()\n return model\n```\n\n### Training Hyperparameters\nTo ensure reproductive stability, I used a consistent training configuration across all folds:\n* **Optimizer:** `AdamW` with `weight_decay = 1e-4`.\n* **Scheduler:** `CosineAnnealingWarmRestarts` (2 cycles).\n* **Batch Size:** 32 (standard) or adjusted based on available GPU memory.\n* **Epochs:** 100 per fold, saving the best checkpoint for each annealing cycle.\n* **Mixed Precision:** `torch.amp.autocast` (FP16) enabled to speed up processing and reduce memory usage.\n\n### Multi-Stage Ensemble Strategy\nThe final submission was not based on a single model but an ensemble of multiple \"snapshots\" collected across cycles and folds:\n1. **Averaging:** I perform a simple arithmetic mean of the calibrated predictions across all saved fold/cycle checkpoints.\n2. **Inversion:** Predictions are inverted from the Z-score normalized space back to original biomass units.\n3. **Refinement:** Manual scaling and physical consistency optimization are applied at the very end of the ensemble pipeline.\n\n```python\nconfig = LoraConfig(\n r=4,\n lora_alpha=8,\n target_modules=[\"qkv\"],\n lora_dropout=0.05,\n bias=\"none\",\n)\nself.backbone = get_peft_model(self.backbone, config)\n```\n\n## 4. Post-Processing: The Boost to Private LB\nThe performance jump to Rank 5 came from a multi-stage inference pipeline:\n\n1. **Test-Time BN Adaptation:** Before prediction, I updated the model's Batch Normalization statistics (`running_mean`, `running_var`) by passing test data through the model in `train()` mode with frozen weights. This aligned the model with the test set's exposure and lighting.\n2. **Test-Time Augmentation (TTA):** 4-view averaging (Original, H-Flip, V-Flip, Rot90).\n3. **Post-Hoc Calibration:** A `LinearRegression` calibrator was fitted on OOF predictions to correct residual scale and bias shifts.\n\n```python\nclass PostHocCalibrator:\n def fit(self, preds: np.ndarray, targets: np.ndarray):\n for i in range(preds.shape[1]):\n reg = LinearRegression()\n reg.fit(preds[:, i].reshape(-1, 1), targets[:, i])\n self.models.append(reg)\n\n def predict(self, preds: np.ndarray):\n calibrated = [reg.predict(preds[:, i].reshape(-1, 1)) \n for i, reg in enumerate(self.models)]\n return np.stack(calibrated, axis=1)\n```\n4. **Manual Scaling Logic:**\n * `Dry_Clover_g` scaled by `0.855`.\n * `Dry_Dead_g` adjusted upwards (+15.5%) if > 20 and downwards (-4.5%) if < 10.\n5. **Physical Consistency Optimization:** Enforced the law of parts-to-whole using `scipy.optimize.minimize` (L-BFGS-B). I trusted the `Total` and `GDM` heads more by applying a weight of **1.2** compared to **1.0** for individual components:\n\n```python\ndef objective(x):\n clover, dead, green = x\n total = clover + dead + green\n gdm = clover + green\n x_full = np.array([clover, dead, green, total, gdm])\n return np.sum(weights * (x_full - preds) ** 2)\n```\n\n## 5. Final Ensemble Strategy\nMy top-performing submission was a robust ensemble of two distinct model configurations, blending global context with fine-grained local details:\n\n1. **DINOv2 Large Model:** \n * **Resolution:** High-resolution 518x518 input.\n * **Tiling:** 2-Tile split (Left/Right) to preserve structural continuity while doubling effective resolution.\n2. **DINOv3 Large Model:** \n * **Resolution:** 256x256 patches.\n * **Tiling:** Aggressive 8-Tile split to capture high-frequency texture and localized biomass density variations.\n\nBy combining the **DINOv2** (focused on high-resolution global structure) with the **DINOv3** (leveraging multi-tile local features).\n\n## Acknowledgments\nI would like to express my sincere gratitude to the competition organizers for hosting such a challenging and meaningful event. I also want to thank the other teams and the community; many of the ideas and implementation strategies in this solution were inspired by their insights and shared knowledge." } }