# 4 th place solution

Hi,
Given the age of this competition, I'm not sure anyone is reading these notes. But we never know. :-)
This competition was a bit special because the public leaderboard was calculated on the whole test set. And therefore, no suspense on the result of the private leaderboard which is identical to the public.
My approach was:
removal of irrelevant variables using
sklearn.feature_selection.mutual_info_classif
Training on several models (
SvcSvm, KNeighborsClassifier, HistGradientBoostingClassifier, RidgeClassifierCV, MLPClassifier, RandomForestClassifier,ExtraTreesClassifier,AdaBoostClassifier, GradientBoostingClassifier, LogisticRegression, CatBoostClassifier,LGBMClassifier and XGBClassifier
)
Using Optuna to find the best settings of each
At the end of this step, I had several trained models with their respective oof files.
With these new files (predictions + oof ), I generate a new train and test set of 156 features. It is quite a lot.
I select from these new sets, the best features with
RFECV
(Feature ranking with recursive feature elimination). So I keep 9 features from the 156.
Finally, I trained a model of type LogisticRegression for the final prediction.
Because the private and public leaderboards were the same, I was able to launch the treatments several times without fear of overfitting.
Thank to Bharat Rai for organizing this very very long competition
Regards
Thierry