Thanks to the organizers and the Kaggle community for an outstanding competition — I’m happy to share the core ideas behind my final submission.
\n\nTo generate a robust training dataset, I used a modified version of interpolate_and_split.py. This produced approximately ~38,000 additional samples, combined with the original samples from train.csv, yielding a total of ~40,000 training samples.
Missing values were filled with zeros.
I did not use any handcrafted features — early experiments with metadata (like geology_id) and traditional feature engineering didn’t show meaningful gains.
I used 5-fold cross-validation and trained one model per fold. Final predictions were averaged across all 5 models.
\nOptimizer: AdamW
Scheduler: Cosine Annealing (T_0 = 5, T_mult = 2)
Loss: Normalized MSE, where both output and target are standardized per batch:
loss = F.mse_loss(\n (output - target_mean) / target_std,\n (target - target_mean) / target_std\n)\nGradient clipping: max_norm = 1.0
Epochs: 30
Batch size: 256
Mixup augmentation applied during training with α = 0.4
Although seeds were fixed and deterministic flags enabled, I encountered reproducibility issues between environments:
\nFinal model was trained on a local RTX 3080 Ti
Re-running on Kaggle P100 produced slightly different weights (likely due to numerical precision and cuDNN backend differences)
To ensure the notebook’s predictions exactly match the final submission, the weights are loaded from a private Kaggle dataset.
\nThe full training code is included (commented) in the final notebook, and the GitHub repo contains all implementation files.
\nSeveral Kaggle participants shared ideas that shaped my approach:
\nEva Koroleva
\nOlSch
\nThis structure inspired me to apply residual connections within my LSTM stack:
\nx = Dense(UNITS, activation=ACTIVATION)(x)\nx = Dropout(DROPOUT)(x)\nx = x + skip \nThanks to everyone who shared their insights — it made this journey both fun and impactful.
\n✅ Full pipeline (training, inference, augmentation, evaluation):
\n📂 GitHub
\n📄 Licensed under BSD 2-Clause in accordance with the competition rules.
Thanks to the organizers and the Kaggle community for an outstanding competition — I’m happy to share the core ideas behind my final submission.
\n\nTo generate a robust training dataset, I used a modified version of interpolate_and_split.py. This produced approximately ~38,000 additional samples, combined with the original samples from train.csv, yielding a total of ~40,000 training samples.
Missing values were filled with zeros.
I did not use any handcrafted features — early experiments with metadata (like geology_id) and traditional feature engineering didn’t show meaningful gains.
I used 5-fold cross-validation and trained one model per fold. Final predictions were averaged across all 5 models.
\nOptimizer: AdamW
Scheduler: Cosine Annealing (T_0 = 5, T_mult = 2)
Loss: Normalized MSE, where both output and target are standardized per batch:
loss = F.mse_loss(\n (output - target_mean) / target_std,\n (target - target_mean) / target_std\n)\nGradient clipping: max_norm = 1.0
Epochs: 30
Batch size: 256
Mixup augmentation applied during training with α = 0.4
Although seeds were fixed and deterministic flags enabled, I encountered reproducibility issues between environments:
\nFinal model was trained on a local RTX 3080 Ti
Re-running on Kaggle P100 produced slightly different weights (likely due to numerical precision and cuDNN backend differences)
To ensure the notebook’s predictions exactly match the final submission, the weights are loaded from a private Kaggle dataset.
\nThe full training code is included (commented) in the final notebook, and the GitHub repo contains all implementation files.
\nSeveral Kaggle participants shared ideas that shaped my approach:
\nEva Koroleva
\nOlSch
\nThis structure inspired me to apply residual connections within my LSTM stack:
\nx = Dense(UNITS, activation=ACTIVATION)(x)\nx = Dropout(DROPOUT)(x)\nx = x + skip \nThanks to everyone who shared their insights — it made this journey both fun and impactful.
\n✅ Full pipeline (training, inference, augmentation, evaluation):
\n📂 GitHub
\n📄 Licensed under BSD 2-Clause in accordance with the competition rules.