# 6th place solution overview

My approach is good old "feature engineer till you drop" so I suppose it's not so interesting after all but I hope somebody will find something. :) Here is brief overview of approach.
Solution follows standard scheme. Train logloss-model for user-product pairs, train logloss-model for None, use probabilities for F1-optimisation.
Let us denote current user as U, last order of U as LO, current product as P and its aisle and department as A and D. Let us denote order as "A order" if it contains at least one product from aisle A (same for P and D).
Product only features:
P is Low Fat/Organic :)
200 binary "P_is_X_product" features for top-200 most popular products
binary "A_is_X_aisle" and "D_is_X_department" features for all X aisles and departments
average day/count interval between all P orders
User only features
7+24 binary day of week and hour of day features for O
Ratio of reordered products in last N orders/last M days orders of U
Average similarity of product sets for sequential orders (e.g. Jaccard measure of such sets)
Average similarity of product sets for same day orders (it could be useful in case LO has days_since == 0)
User-product features:
Number of days/orders between LO and most recent order of U with P/A/D
Number of orders between most recent order with P and most recent order with A (trying to see if user switched products inside A)
Binary "it's been X orders since last order with P" features where X is from 1 to 10
Average day/count interval between all U's orders with P/A/D
Share of P products/orders among all/A/D products/orders bought by U (how ofter U buys P)
Same share but only for orders since first P order (how ofter U buys P since he began to do so)
Same share but for last N orders
Same share but for orders during last M days
Average position of P in orders of U (raw or normalized by size of order)
Ratio of average interval between U's orders with P to average interval between all orders with P
Ratio of number of days/orders since most recent U's P order to average interval between all orders with P
Let's assume it's been N days/orders since last P order. Using all prior orders for each product P we can calculate probability to reorder P if it's been N days/orders since last P order.
Same could be done using only U's prior orders (but it's much more noisy of course). I've tried to smooth one probability with another buy couldn't achieve good results.
Product similarity features. Let's pretend that users often switch from product A to product B (for example Bananas and Organic Bananas). How could we try to detect such product pairs. We could assume that buying product B should considerably lower probability of buying product A in the same order. Let's consider ratio = P(A in current order|A in previous orders, B in current order) / P(A in current order|A in previous orders). For unrelated products this ratio should be close to 1. To similar products this ratio should be close to 0. Such probabilities could be easily calculated using prior orders. For example top-10 pairs with lowest ratio (only for pairs with enough statistics) are:
Large Brown Eggs - Organic Large Brown Grade AA Cage Free Eggs
Large Brown Eggs - Organic Large Grade AA Brown Eggs
Green Seedless Grapes - Grape White/Green Seedless
Organic Large Grade AA Brown Eggs - Large Brown Eggs
Organic Broccoli - Organic Broccoli Crowns
Banana - Organic Banana
Organic Banana - Banana
Organic Milk Reduced Fat - Organic Reduced Fat Milk
Banana - Bag of Organic Bananas
Half And Half Cream - Organic Half & Half
Using these pairs we can generate several features like "how many products similar to P user U bought since last P order". If such number is great it could mean user switched from product P to similar product Q.
Average share of P aggregated by months.
Linear extrapolation of mentioned above month aggregated share.
P orders streak length of user U
I've expanded training set using last prior order for all test users with at least 5 prior orders.
Main user-product probability model is average of 8 CatBoost models. I've used product ID as categorical feature for CatBoost (yeah, that's why CatBoost has such name :))
For None-classifier I've used the same training users. I've used my per-user features from main model and used features based on predictions of main model. It's a little bit not very clean and tidy approach because main model predictions are a little bit overfitted for training users (so features on train distributed in slightly another manner in comparison to features on test) but this approach gave me good results, and I was unable to invent better approach. :) Splitting train users for main/none classifiers gave me considerably worse results. So, None-classifier is a single CatBoost model.
F1-optimization. At first I used a Monte-Carlo simulation using predicted probabilities to determine solution with max expected F1 for each user. But Faron's approach gave me considerably better results. So I've used it instead of Monte-Carlo. :)
Thanks!