# 76th Private/11th Public - CSIRO Biomass Prediction Pipeline

CSIRO Biomass Prediction Pipeline
Overview
This solution achieves
0.790 weighted R²
on the CSIRO Biomass prediction task by ensembling three complementary deep learning models with optimized per-target weights and physical constraints.
Architecture Overview
The pipeline consists of three diverse models combined through an optimized ensemble:
![](assets/24e59f83ca29a74196089da2.png)
Model 1: S5-DINO (ContextAwareMIL)
Backbone
: DINOv3 ViT-Huge (1,280-dim embeddings)
Multi-Scale Crops
: 3×3 grid (9 crops) + 2×2 grid (4 crops) =
13 crops
Physics Features
: Green Fraction (HSV-based) + Excess Green Index (ExG)
Sequence Model
: 4-layer S5 (State Space) blocks with residual connections
Context Aggregation
: Mean + Std pooling across crops
Output
: Softplus activation → Sum across crops (MIL aggregation)
Feature dimensions
: 13 crops × (1,280 DINO + 2 physics) =
16,666 features
Model 2: S5-SigLIP (ContextAwareMIL + Semantic)
Backbone
: DINOv3 ViT-Huge + SigLIP SO400M
Multi-Scale Crops
: Same 13-crop strategy
Semantic Features
: 8 concept groups via CLIP-style similarity:
bare
,
sparse
,
medium
,
dense
(coverage)
green
,
dead
(vitality)
clover
,
grass
(species)
Derived Ratios
: greenness, clover fraction, cover ratio
Feature dimensions
: 13 crops × (1,280 DINO + 11 SigLIP + 2 physics) =
16,809 features
Model 3: MIL Uncertainty (DINOv3 7B)
Backbone
: DINOv3 ViT-7B (4,096-dim embeddings)
Tile Strategy
: 3×6 patch-aligned grid =
18 tiles
Head
: MILUncertaintyHead with attention-based aggregation
Uncertainty
: Predicts both μ (mean) and σ² (variance) per target
Ensemble
: 5-fold × 3 members × 2 TTA (orig + hflip) =
30 components
Training Strategy
Cross-Validation
S5 Models
: 3-fold GroupKFold (grouped by image ID)
MIL Model
: 5-fold GroupKFold with 3 ensemble members per fold
Data Augmentation
S5 Models
: Crop shuffle TTA (seeds: None, 42, 123)
MIL Model
: Horizontal flip TTA
Normalization
Features
: MinMaxScaler(0, 1) fitted per fold
Targets
: MinMaxScaler(1e-5, 1) for stable gradients
Ensemble Optimization
Weight Search Strategy
Brute-force grid search with step=0.01 to find optimal weights:
Target
w_MIL
w_S5
w_SigLIP
R²
Dry_Clover_g
0.33
0.31
0.36
0.832
Dry_Dead_g
0.37
0.38
0.25
0.601
Dry_Green_g
0.31
0.62
0.07
0.850
Dry_Total_g
0.56
0.36
0.08
0.792
GDM_g
0.25
0.69
0.06
0.828
Physical Constraints
Post-processing to ensure biologically valid predictions:
Clip all values to [0, 200]
Enforce:
Dry_Total_g ≥ Dry_Green_g + Dry_Dead_g
Enforce:
GDM_g ≤ Dry_Total_g
📊 Results Summary
Method
Weighted R²
MIL Uncertainty (single)
0.7496
S5-SigLIP (single)
0.7556
S5-DINO (single)
0.7631
Simple Average (3-model)
0.7865
Global Optimal (3-model)
0.7880
Per-Target + Constraints
0.7901
✓