# Private Leaderboard 3442 - A Beginner's Simple Solution and Key Takeaways

Hi everyone!
Don't think this would be useful to anyone with experience on Kaggle but I thought I would share my rather basic solution. As this was my first actual project, I mostly ended up trying out all the things I had learned from tutorials in a real project rather than making a significant model. It proved a good experience and I look forward to becoming better at DS using the next steps for education identified through this competition.
My best performing solution's notebook is available
[here](https://www.kaggle.com/code/shahilap96/private-leaderboard-3442-my-best-solution)
for anyone that wishes to take a look.
Final model:
Denoised
[dataset](https://www.kaggle.com/datasets/raddar/amex-data-integer-dtypes-parquet-format)
by
raddar
.
Last two transactions per customer as suggested by this
[notebook](https://www.kaggle.com/code/junjitakeshima/amex-try-to-improve-lgbm-starter-eng)
. I tried all variations and found that two worked best for me.
Dropped Customer_ID and S_2 columns. I found that keeping all other columns (including those with high number of nulls gave better results.
scikit-learn's imputation and standard scaling.
Two models -> LGBM (n_estimators=300) and CatBoost (default parameters).
Averaged my LGBM and CatBoost prediction probabilities for final submission. This gave the best solution.
Things I tried:
XGBoost - didn't perform as well as LGBM and CatBoost alone and when averaged with their scores.
Simple NN - last minute trial, didn't perform well, probably due to lack of tuning.
Random Forest, Decision Tree and other similar ML models.
Removed columns with high number of nulls (50% null, 66% null, 75% null).
Cross validation for measuring performance.
Dask and Vaex for working with the large volume of data initially.
Hyperparameter tuning using GridSearchCV.
Things I need to look at next:
Denoising data
EDA
Feature engineering
Understanding higher ranking participants solutions
Handling large volume of data when training models.
I definitely have a long way to go but this competition was a great starting point and the community around it made it an awesome experience.
Happy Kaggling!