# 260th place solution

Thank to AMEX for this competition and thank to
[@raddar](https://www.kaggle.com/raddar)
for his wonedful data compression notebook.
My model is the enesemble of DART and Neural networks.
Neural Network models: Tabular data(same used for Tree learners), Gru, Transformer and Stack Model(Gru+Tabular Model).
Following are the scenarios where neural nets struggled compared to the tree based learners.
1 . Outlier numbers in some features.(lightgbm process it as quantiles, hence roboust)
2 . Understanding if the value is missing; even if we fix it as -1, still will be difficult to process.
3 . Dominant easy examples.(Gradients are not getting propogated to hard/varied examples as most examples are easy to classify).
1. Preprocessing:
Clip the values of the features to 95th or 99th percentile of the feature.
[Tabular Data Clip values](https://www.kaggle.com/code/narendra/amex-tabular-nn-data-clip-map)
,
[Sequential Data Clip values](https://www.kaggle.com/code/narendra/amex-sequence-data-clip-map)
2. Add Embeddings to the Missing values
For both FFN (tabular) and sequential data(GRU, transformer), for each feature given more information that the values are missing or not. Adding this information helps the model to converge faster and better generalization.
Tabular Model:
[here](https://www.kaggle.com/code/narendra/amex-nn-ranking-train-1024#model)
Gru Model:
[here](https://www.kaggle.com/code/narendra/amex-gru-with-tab-embedds-train-v3#Sequence-Model)
Stack Model:
[here](https://www.kaggle.com/code/narendra/amex-model-stack)
Transformer Model:
[here](https://www.kaggle.com/code/narendra/amex-transformer-train-v3)