# 1st Place Solution | Single NN - 3 Random Seed

Final Submission Code:
[here](https://www.kaggle.com/code/masayakawamata/bj-deep-neural-network?)
Acknowledgements
First, I'd like to sincerely thank
[@andreipaulavets](https://www.kaggle.com/andreipaulavets)
for organizing this competition and for providing thoughtful feedback throughout. Your comments helped shape and sharpen many of my experiments. Thanks also to
[@ravi20076](https://www.kaggle.com/ravi20076)
for introducing such a great learning opportunity.
TL;DR
Best Model:
Deep Neural Network (DNN), trained and averaged over 3 different seeds.
Evaluation:
10-fold KFold cross-validation.
Final DNN Architecture:
A deep feedforward network without dropout.
Performance:
CV:
~0.0000000200
Public LB:
0.0000000092
Private LB:
0.00000000109
Dataset Characteristics & Model Performance
Data Overview:
The dataset captures complex, non-linear relationships between card removal patterns and expected value (EV) changes in a 6-deck Blackjack shoe. Models capable of learning these non-linearities, like Neural Networks and GBDTs, performed well.
Model Comparisons:
GBDT Models:
CatBoost initially performed very well, outperforming XGBoost and LightGBM in early trials. Parameter tuning was critical, with CatBoost requiring deeper trees (depth=6) for best results.
Neural Networks:
Neural Networks ultimately outperformed all other models. After noticing that NN-based models consistently ranked high in AutoGluon's leaderboard, I explored different architectures through a grid-style search. The most effective architecture was:
nn_model.add(Dense(, activation=, input_dim=X_train_scaled.shape[]))
nn_model.add(Dense(, activation=))
nn_model.add(Dense(, activation=))
nn_model.add(Dense(, activation=))
nn_model.add(Dense(, activation=))
nn_model.add(Dense(, activation=))
nn_model.add(Dense(, activation=))
nn_model.(optimizer=, loss=)
This deep, dropout-free network achieved extremely low error. To reduce prediction variance, I trained three models with different seeds and averaged their predictions.
Feature Engineering
While earlier trials showed that engineered features like card counts, averages, and ratios helped GBDT models, the final DNN model performed best using only the original features. This suggests the DNN was able to internally capture the structure and relationships in the data without explicit feature engineering—thanks in part to the clean and low-noise nature of the dataset.
Conclusion
This competition was a valuable lesson in questioning common assumptions. While GBDTs are often considered best for tabular data, this challenge showed that deep neural networks can outperform them in settings where the data is clean and highly structured. Careful tuning of architecture and training strategies enabled the DNN to reach top-tier performance.