# public 1st, private 2nd solution

Thanks to the organizers and the Kaggle community for an outstanding competition — I’m happy to share the core ideas behind my final submission.
[Final Notebook](https://www.kaggle.com/code/nikitashevyrev/geologyforecast-final-version-2nd-in-private)
Data Section
To generate a robust training dataset, I used a modified version of
interpolate_and_split.py
. This produced approximately
~38,000 additional samples
, combined with the original samples from
train.csv
, yielding a total of
~40,000 training samples
.
Missing values were filled with zeros.
I did not use any handcrafted features — early experiments with metadata (like
geology_id
) and traditional feature engineering didn’t show meaningful gains.
Training and Model
I used
5-fold cross-validation
and trained one model per fold. Final predictions were averaged across all 5 models.
Optimizer:
AdamW
Scheduler:
Cosine Annealing (
T_0 = 5
,
T_mult = 2
)
Loss:
Normalized MSE, where both output and target are standardized per batch:
loss = F.mse_loss(
    (output - target_mean) / target_std,
    (target - target_mean) / target_std
)
Gradient clipping:
max_norm = 1.0
Epochs:
30
Batch size:
256
Mixup augmentation
applied during training with α = 0.4
Reproducibility Notes
Although seeds were fixed and deterministic flags enabled, I encountered reproducibility issues between environments:
Final model was trained on a local
RTX 3080 Ti
Re-running on
Kaggle P100
produced slightly different weights (likely due to numerical precision and cuDNN backend differences)
To ensure the notebook’s predictions exactly match the final submission, the weights are loaded from a
private Kaggle dataset
.
The full training code is included (commented) in the final notebook, and the GitHub repo contains all implementation files.
Acknowledgements & Influences
Several Kaggle participants shared ideas that shaped my approach:
Eva Koroleva
Idea: Using
LSTM
as a base architecture
Notebook:
[LSTM KFold](https://www.kaggle.com/code/qmarva/lstm-kfold)
Profile:
[@qmarva](https://www.kaggle.com/qmarva)
OlSch
Idea: Applying
skip connections
inside dense networks
Notebook:
[Keras Geology Forecast Competition 58.4](https://www.kaggle.com/code/olasdsd/keras-geology-forecast-competition-58-4)
Profile:
[@olasdsd](https://www.kaggle.com/olasdsd)
This structure inspired me to apply residual connections within my LSTM stack:
x = Dense(UNITS, activation=ACTIVATION)(x)
x = Dropout(DROPOUT)(x)
x = x + skip
Thanks to everyone who shared their insights — it made this journey both fun and impactful.
GitHub Repository
✅ Full pipeline (training, inference, augmentation, evaluation):
📂
[GitHub](https://github.com/NikitaShevyrev/Geology-Forecast-Challenge)
📄 Licensed under BSD 2-Clause in accordance with the competition rules.