# 4-th Place Tips

My solution is some ad-hoc feature engineering, which is rather boring and trivial compared to sjv's clean and cool neural network solution. Here are some tips:
Uncertainty and Correlation of Estimated Probabilities:
Faron's O(n^2) F1-optimization script is sub-optimal because it does not address correlation. Uncertainty is also an issue: when all of the estimated probabilities have high uncertainty, it is better to choose all the products to optimize F1. I have two ways to address this issue:
Multiply the probability by a factor that is negatively correlated with the conditional basket size. I tried some parametric function, and I chose 1.6^(1/n^2), where n is the average conditional basket size: conditional on user purchasing a certain product, how many products he purchase in this order. For "None", n=1.
Exact F1 Measure, based on the paper An Exact Algorithm for F-Measure Maximization. The algorithm is O(n^3) if we supply the joint probability matrix(n*n) of a product being purchased and the corresponding basket size of the order. When the purchase probability are independent, it is easy to compute such theoretical matrix using O(n^2) and the result would be the same to Faron's F1 optimization. I constructed an empirical distribution of conditional basket size and take a weighted average between the probability matrix with independent assumption and this empirical one. 
I gained about 0.001-0.0015 from the above optimization.
catboost
: catboost is my best single model, with a 0.4086 public LB and 0.4070 private LB. It is about 0.0003-0.0005 better than lightgbm and xgboost. I turned on the "accurate" option of xgboost, and it is 0.0015 better than the approximate one.
LDA and NMF
: I also tried to use LDA and NMF features, which gives me 0.0005-0.001 increase.
What did not worked for me
:
I ended up not using lag1 data because they did not perform well based on my feature engineering. It's possible that when I was constructing summary statistics such as product average, product standard deviation, they got messed up. It would be great to see how others are using the lag1 data.