# 54th Place Solution: All I Need Is fastai, timm, and DINOv3

I would like to thank the competition organizers and Kaggle for creating this interesting challenge.
This competition was exactly the kind of image regression competition I had been waiting for.
This is my first writeup. I learned a lot from this competition, so I decided to share my solution.
I hope it will be helpful to everyone!
1. Summary
Final Results
Private LB Score: 0.63
Rank: 54
Framework
fastai🩷
timm🩷
Models Used
vit_huge_plus_patch16_dinov3.lvd1689m
vit_large_patch16_dinov3.lvd1689m
StratifiedGroupKFold (3 folds)
2. Validation Strategy
My approach for this competition was inspired by the previously held "PetFinder.my - Pawpularity Contest".
The Pawpularity competition also required quantifying latent features that are not clearly discernible to the human eye, making it similar to this biomass estimation task where judgment from images alone is difficult.
2.1. Kaggle Notebook Environment
Due to different start times, the Kaggle Notebook environment varied, so standardization was necessary
I confirmed that older versions of scikit-learn produced incorrect split results with StratifiedGroupKFold
[https://github.com/scikit-learn/scikit-learn/issues/32478](https://github.com/scikit-learn/scikit-learn/issues/32478)
[https://github.com/scikit-learn/scikit-learn/pull/32540](https://github.com/scikit-learn/scikit-learn/pull/32540)
2025-11-07 version
fastai: 2.8.5
scikit-learn: 1.2.2
timm: 1.0.19
torch: 2.6.0+cu124
2025-12-17 version
fastai: 2.8.4
scikit-learn: 1.6.1
timm: 1.0.20
torch: 2.8.0+cu126
Standardized environment
fastai: 2.8.4
scikit-learn: 1.8.0
timm: 1.0.24
torch: 2.8.0+cu126
![](assets/698334002281280a4b8bfb85.jpg)
2.2. Images
Removed the annotation issue (ID230058600) reported in the discussion
[https://www.kaggle.com/competitions/csiro-biomass/discussion/631335#3338190](https://www.kaggle.com/competitions/csiro-biomass/discussion/631335#3338190)
Confirmed distribution bias in Fescue
2015/10/6: 11 images
2015/2/25: 9 images
2015/5/7: 8 images
Confirmed distribution bias in Lucerne (alfalfa)
2015/1/15: 7 images
2015/4/1: 10 images
2015/5/7: 5 images
2.3. Split Strategy
Used 3-fold split to properly handle Fescue and Lucerne (alfalfa)
CV Strategy
StratifiedGroupKFold (3 folds)
stratify:
stratify_label
groups:
Sampling_Date
+
State
# Create `stratify_label` from species
def get_stratify_label(species):
    if 'Lucerne' in species:
        return 'Lucerne'
    elif 'Fescue' in species:
        return 'Fescue'
    elif 'Phalaris' in species:
        return 'Phalaris'
    elif 'Clover' in species:
        return 'Clover'
    else:
        return 'Other'
Average
Dry_Total_g
distribution for each seed
SEEDS = [42, 1, 777, 2024]
stratify_label
fold
42
1
777
2024
std
Clover
0
38.488704
40.031428
40.821862
40.821862
1.099880
Clover
1
36.034956
34.881416
34.090982
33.604583
1.060977
Clover
2
34.221660
33.740706
33.740706
34.221660
0.277679
Fescue
0
76.177778
76.177778
75.376471
76.177778
0.400654
Fescue
1
46.550000
91.121053
103.227273
46.550000
29.642796
Fescue
2
91.121053
46.550000
46.550000
91.121053
25.733109
Lucerne
0
42.160000
42.160000
50.920000
42.160000
4.380000
Lucerne
1
75.171429
50.920000
42.160000
75.171429
16.912791
Lucerne
2
50.920000
75.171429
75.171429
50.920000
14.001569
Other
0
39.602041
39.602041
39.602041
39.602041
0.000000
Other
1
40.413157
31.913777
31.913777
40.413157
4.907119
Other
2
27.551414
36.985586
36.985586
27.551414
5.446822
Phalaris
0
38.461538
38.461538
38.461538
38.461538
0.000000
Phalaris
1
76.023600
50.312000
61.631600
64.704000
10.571392
Phalaris
2
46.888000
72.599600
61.280000
58.207600
10.571392
2.4. CV and LB Correlation
There was a very good correlation between CV and LB. This allowed me to confidently trust the CV scores.
LB
CV
r2_weighted, [Clover, Dead, Green, Total, GDM]
Seed
Target
DINOv3
0.75
0.764281, [0.68012, 0.510887, 0.752103, 0.675727, 0.724613]
42
3 primary + Height
Huge
0.74
0.753265, [0.659145, 0.495417, 0.730635, 0.661692, 0.713109]
42
5 biomass + Height
Huge
0.74
0.754347, [0.684125, 0.503932, 0.724692, 0.666095, 0.705388]
42
3 primary + Height (-Noise)
Huge
0.75
0.756209, [0.662654, 0.512846, 0.741057, 0.665265, 0.712921]
42
3 primary + Height + NDVI
Huge
0.74
0.745794, [0.703269, 0.471936, 0.717405, 0.656409, 0.686196]
777
3 primary + Height + NDVI
Huge
0.75
0.733871, [0.705841, 0.484615, 0.714099, 0.640872, 0.657992]
1
3 primary + Height + NDVI
Huge
0.74
0.744295, [0.718311, 0.469911, 0.729706, 0.652764, 0.680162]
2024
3 primary + Height + NDVI
Huge
0.74
0.756737, [0.697973, 0.494104, 0.738189, 0.669619, 0.702361]
777
3 primary + Height
Huge
0.74
0.720886, [0.700602, 0.451035, 0.709695, 0.61784, 0.653728]
1
3 primary + Height
Huge
0.74
0.74205, [0.724016, 0.455966, 0.725739, 0.647846, 0.68428]
2024
3 primary + Height
Huge
0.74
0.758067, [0.723696, 0.538679, 0.725656, 0.672485, 0.69996]
42
3 primary + Height
Large
0.74
0.779158, [0.698054, 0.547103, 0.735667, 0.705507, 0.728854]
42
3 primary + Height + NDVI
Large
0.74
0.779295, [0.694276, 0.526245, 0.740259, 0.706725, 0.726529]
42
3 primary + Height + NDVI (drop_rate=0.3, wd=0.01)
Large
0.71
0.767606, [0.618705, 0.479375, 0.741914, 0.684672, 0.734104]
42
5 biomass + Height + NDVI (drop_rate=0.3, wd=0.01)
Large
3. Model Architecture
I adopted DINOv3 pre-trained models.
Image sizes were determined considering A100 memory constraints.
vit_huge_plus_patch16_dinov3.lvd1689m
Input: 448×896×3
Epochs: 50
vit_large_patch16_dinov3.lvd1689m
Input: 512×1024×3
Epochs: 40
Common settings
Loss: RMSELossFlat
Learning rate: 0.0015
Batch size: 16
Freeze epochs: 4
Callbacks: SaveModelCallback, ReduceLROnPlateau (monitor='rmse')
4. Training
4.1. Multi-target Learning Strategy
To improve representation learning, I added auxiliary tasks in addition to the submission target variables.
# Final configuration (7 targets)
targets = [
    'Dry_Clover_g', 'Dry_Dead_g', 'Dry_Green_g',  # Primary (for submission)
    'Dry_Total_g', 'GDM_g',                       # Auxiliary biomass
    'Height_Ave_cm', 'Pre_GSHH_NDVI'              # Auxiliary measurements
]
Configurations tested:
3 primary + Height
3 primary + Height + NDVI
5 biomass + Height
5 biomass + Height + NDVI
4.2. Data Augmentation
I used the fastai pipeline.
# Resize to target resolution
item_tfms = Resize(input_size, method='squish')

# Training augmentations
batch_tfms = \
    setup_aug_tfms([
        Brightness(max_lighting=0.1),
        Contrast(max_lighting=0.1),
        Saturation(max_lighting=0.1),
        Hue(max_hue=0.02)
    ]) + aug_transforms(do_flip=True, flip_vert=True)
5. Ensemble & Post-processing
5.1. Multi-seed Training
I trained with multiple seeds to ensure ensemble diversity.
24 models
vit_huge_plus_patch16_dinov3
4 seeds
3 folds
2 targets (3 primary + Height, 3 primary + Height + NDVI)
SEEDS = [42, 1, 777, 2024]
for seed in SEEDS:
    sgkf = StratifiedGroupKFold(n_splits=3, random_state=seed, shuffle=True)
    for fold, (train_index, valid_index) in enumerate(sgkf.split(df, df['stratify_label'], df['group_key'])):
        # Train and save model
        model = train_fold(train_index, valid_index, seed, fold)
5.2. Final Ensemble
The final submission was a simple average of 42 models.
Simple averaging of 24 models
Additional 18 models
5 biomass + Height: 3 models
3 primary + Height (-Noise): 3 models
vit_large_patch16_dinov3: 3 models x 4 targets = 12 models
5.3. Post-processing
Predictions were made only for ['Dry_Clover_g', 'Dry_Dead_g', 'Dry_Green_g'], and Total and other values were calculated from these.
['Dry_Clover_g', 'Dry_Dead_g', 'Dry_Green_g']
'Dry_Total_g' = 'Dry_Clover_g' + 'Dry_Dead_g' + 'Dry_Green_g'
'GDM_g' = 'Dry_Clover_g' + 'Dry_Green_g'
5.4. Submit LB and CV
vit_huge_plus_patch16_dinov3.lvd1689m
seed: 42 x 4 models
seed: 777 x 2 models
seed: 1 x 2 models
seed: 2024 x 2 models
vit_large_patch16_dinov3.lvd1689m
seed: 42 x 4 models
LB
CV
r2_weighted, [Clover, Dead, Green, Total, GDM]
Ensemble models
0.75
0.773948, [0.718483, 0.532946, 0.747799, 0.694633, 0.722592]
huge:[42x4, 777x2, 1x2, 2024x2] + large:[42x4]
6. What Didn't Work
Noise removal of "red spots, date-time strings, cardboard" mentioned in discussions had little effect
[https://www.kaggle.com/competitions/csiro-biomass/discussion/651525#3363233](https://www.kaggle.com/competitions/csiro-biomass/discussion/651525#3363233)
GradCAM visualization confirmed that the model does not focus much on these noise regions
![](assets/84eb906f0da1d2c7e08fa745.jpg)
![](assets/daf902433306872daa4ae880.jpg)
Splitting by Sampling_Date: Results were unstable and CV worsened.
[https://www.kaggle.com/competitions/csiro-biomass/discussion/615401](https://www.kaggle.com/competitions/csiro-biomass/discussion/615401)
TTA: CV worsened.
[https://www.kaggle.com/competitions/csiro-biomass/discussion/668066](https://www.kaggle.com/competitions/csiro-biomass/discussion/668066)
Tried various models but CV did not improve much
eva02_large_patch14_448
eva02_base_patch14_448
vit_so400m_patch14_siglip_378
vit_so150m2_patch16_reg1_gap_44
convnext_large_mlp
caformer_b36
naflexvit_base_patch16_par_gap
vit_base_patch14_reg4_dinov2
vit_7b_patch16_dinov3.sat493m
Other approaches that worsened CV:
SmoothL1Loss
MixUp()
CutMix()
7. Hardware & Environment
For early to mid-stage validation, I used lightweight models on Kaggle Notebook
T4 x2: 15GB
vit_small_plus_patch16_dinov3.lvd1689m
CV Avg: 0.69
15min/fold
For later-stage training, I used Google Colab Pro (approximately $25 cost)
A100: 80GB
vit_huge_plus_patch16_dinov3.lvd1689m
45min/fold
A100: 40GB
vit_large_patch16_dinov3.lvd1689m
25min/fold
8. Conclusion & Lessons Learned
8.1. DINOv3 is Amazing!
I used DINOv3 for the first time in this competition and was amazed by its high learning accuracy.
8.2. Domain Knowledge
I happened to watch an anime that featured alfalfa, which made me pay attention to the distribution bias in the images. I recommend everyone to watch it!
Hiromu Arakawa's "The Noble Farmer" (Hyakushō Kizoku, 百姓貴族) Episode 30 "Pasture"
[https://www.youtube.com/watch?v=ohJgdhWS218](https://www.youtube.com/watch?v=ohJgdhWS218)
8.3. AI and Community Participation
I was anxious about communicating in English and had not made any posts before.
However, with the advancement of AI translation technology in recent years, I have been able to obtain information from many participants in the Kaggle community.
I also realized that AI has been a great help not only technically but also in participating in international communities.
I plan to actively participate in the community and share information in the future.
Thank you, AI❗
Enjoy! Happy Kaggling!