# One week to Silver - LB 86 Approach 

Hello all, firstly congratulation to the winners and top performers, your effort and sharing are great and I have learnt a lot in a very short space of time thanks to you.  Also, I want to say well done for people who had been working on this from the beginning, in the end, tenacity will prevail in the form of knowledge gained, medals and good ranking.
Basically, I have treated this competition as a week-long hackathon and spent one intensive week (50h+) on it.  I am very lucky to be ranked within top 100. This wouldn't have been possible without all the sharing. So big thanks from me. Here is the overall sketch of my approach to achieving top 100 ranking in this short space of time:
Shared Features:
Features from LB0.3805009 baseline by 🐳鲲(China)
Features from LB0.4029970 baseline
[Sh1ng](https://www.kaggle.com/c/instacart-market-basket-analysis/discussion/37697#latest-213541)
Product W2V PCA features by
[Omar Essam](https://www.kaggle.com/omarito/word2vec-for-products-analysis-0-01-lb)
, I used the top 5 components.
Customer Segment features by
[Andrea](https://www.kaggle.com/asindico/customer-segments)
Order streak by
[Faron](https://www.kaggle.com/mmueller/order-streaks-feature)
My own features:
 6. Lag 1 features - User total products, and user reorder sum in n-1 order (last order in prior)
 7. Lag difference - User total products difference between orders: n-1 & n-2,  n-2 & n-3, n-3 & n-4
 8. Skew statistic on user product state: so let' say a user had 5 orders,  for a given product, the product purchase states in these orders were - for argument sake - [0 0 1 1 1 ]. so I capture this series and calculate the Skew and IPR measure - but only the skew measure was useful.
These yields just above 100 features.
Modelling & CV Approach:
I used GBM tools from LightGBM, XGBoost and Arboretum - reused most parameters from the original sharing with minor changes at learning rate.  Time limit means I didn't have time to do any series tuning.   I had also use Keras to build NN model, but the result wasn't good enough to be featured in the final blend
I tried to setup 3-fold OOF pipeline for this. I must have made some silly mistake on setting up user_id into folds or aggregating prediction values. My LB score from the OOF pipeline was way off the normal range, and I decided that I didn't have enough time to get it right (by this time 4 days left). So bye-bye OOF, bye-bye stacking and bye-bye K-fold CV.   I then switched to simply validating on the same 25% of train orders, took the risk of overfitting - I was hoping the amount of data in this competition would help to address the risk of overfitting.
Post-processing - Optimising F1 Score:
Here is the part that I spent most energy on getting to grip with. I had no idea so much academia efforts (like
[this](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4442797/)
&
[this](assets/233b613c1c425946e2a615f0.pdf)
) had been done behind such a seemingly simple metric!  So I took some long time to read those papers, and tried to understand as much as possible the code share by
[Faron](https://www.kaggle.com/mmueller/f1-score-expectation-maximization-in-o-n)
and
[Sh1ng](https://github.com/sh1ng/imba)
.  I then integrate the Faron's F1 optimisation routine in ways very similar to what is shared by
[Ivan](https://www.kaggle.com/c/instacart-market-basket-analysis/discussion/37874)
Final Blending: 
A blending of ARB*0.7+XGB*0.15+LGB*0.15, this ration was checked and valided in the 25 validation data I left out. For each of these individual model I also used a lower learning rate in the last day of the competition to retrain - turned out this gave by far my best private LB results (+0.0002) better than anything else.
My final thoughts on this competition: 
 - It is fun to treat it as 1-week hackathon - I had to force myself not to dwell on small details, and move as quick as I can along the ML pipeline. 
 - I also had too made some tough trade-off - i.e. giving up on fixing my OOF routine, and work on other aspects
- By the end, I half-anticipated some strong shake-up. I notice that some of my models with similar local log-loss were giving some significant difference in the LB via MF1. So in the last two days, I decided to include some features that were not improving my validation logloss which I still believed to be reasonable features.  Would be very interesting to see how other's logloss correlate with their LBs :) 
 - I have benefited a lot from sharing in this competition, but I have also faced significant challenges by late sharing, as discussed in-depth in this post by
[CPMP](https://www.kaggle.com/c/quora-question-pairs/discussion/33801)
here on the Quora competition. My view is simply no matter what, don't lose heart. In the long run, most of us would benefit from learning from such sharing.
That is it! not a top solution, but quick & dirty to get some decent result, thanks again!