# 1st place solution

Thanks to H&M and Kaggle Team giving us such a wonderful competition.Thanks my team mate
[@h4211819](https://www.kaggle.com/h4211819)
finally we got 1st place.This competition has no leak,stable cv-lb correlation,and infinite possibilities to improve, we really enjoyed exploring the accuracy boundaries of  infinite possibilities.
Overview
The most interesting part of this competition is we need to generate train and test by ourself, the candinate generation strategy is the key to beyond limitation of accuracy ,good feature engineering or modeling could be close to the limitation .Our solution is using various retrieval strategies + feature engineering + GBDT, seems simple but powerful.
We mainly generate recent popular items because fashion changing fast and has seasonality,tried to add cold start items but they will never be ranked to top12 due to lacking of interaction information.
user and item interaction information are always the most important of recommendation problem,the features we created are almost interaction features,image and text features didn't help but should be useful for cold start problem.
Almost 50% users have no transactions in recent 3months,so we created many cumulative features for them,and last week,last month,last season features for active users.
We use 6 weeks data as train,last week as valid,retrieve 100 caninates for each user,it has stable cv-lb correlation.We focus on improving single lightgbm model to the last week, cv is 0.0430 and lb is 0.0362.At last week I have to rent gcp's big memory server and vast.ai's gpu server to run bigger models to get higher accuracy.
[](https://postimg.cc/xXLGHd2r)
Retrieval(aka: candidate generation | recall)
At early stage,we focus on increasing the hit number of retrieved 100 candidates,try various strategies to cover more positive sample.
WeekNo
HitNum@100
2020-09-16
39142
2020-09-09
38427
2020-09-02
41019
Ranking
Feature Engineering
basiclly,features are created base on retrieval strategies,create user-item interaction for repurchase,create collaborative filtering score for itemcf,create similarity for embedding retrieval,create item count for popularity……
Type
Description
Count
user-item, user-category of last week/month/season/same week of last year/all, time weighted count…
Time
first,last days of tranactions…
Mean/Max/Min
aggregation of age,price,sales_channel_id…
Difference/Ratio
difference between age and mean age of who purchased item, ratio of one user's purchased item count and the item's count
Similarity
collaborative filtering score of  item2item, cosine similarity of item2item(word2vec), cosine similarity of user2item(ProNE)
Downsampling
If we retrieve 100~500 candidates for each user,the negative samples number is very big,so negative downsampling is must,we found 1 million ~ 2 million negative samples for each week has better performace .
neg_samples = 1000000;seed = 42
train[train['label']>0].append(train[train['label']==0].sample(neg_samples, random_state=seed))
Model
Our best single model is a lightgbm(cv:0.0441,lb:0.0367),finally we trained 5 lightgbm classifier and 7 catboost classifier for ensemble(lb:0.0371).catboost's lb score is much worse than lightgbm,lightgbm has very stable cv-lb correlation.
[](https://postimg.cc/ZBmJRVLt)
Optimization
we use TreeLite to accelerate lightgbm inference speed (2X faster),catboost-gpu is 30X faster than lightgbm-cpu inference.
transform all the categorical features(including two way) to label encoding,use reduce_mem_usage,
create a
feature store
,save intermediate features files to dictionary , final features to feather,exsiting features will not be create again
split all the users to 28 group,inference simultaneously with multiple servers.
Machine
I have a desktop with 128G RAM,64 vcore CPU,TITAN RTX GPU.It's enough to run most of our models to win.My teammate has a desktop with 64G RAM.At last week ,we use 300G RAM gcp instances.