# 4th place solution

Thanks to H&M and Kaggle Team for organizing such a great recommendation competition.
Thanks
[@paweljankiewicz](https://www.kaggle.com/paweljankiewicz)
,
[@titericz](https://www.kaggle.com/titericz)
,
[@cdeotte](https://www.kaggle.com/cdeotte)
and many Kagglers for their great discussions and notebooks.
I hope to get a solo gold metal this time, so I decided to do this competition by myself in the beginning.
Overview
I followed the same pipeline as
[@paweljankiewicz](https://www.kaggle.com/paweljankiewicz)
suggested in
[Addressing common questions and what the competition is really about](kaggle.com/competitions/h-and-m-personalized-fashion-recommendations/discussion/307288)
. First, generate candidates using different recall models for each customers. Second, build a ranking model that ranks the candidates within customer.
[](https://postimg.cc/PNQsRDhB)
CV setup
I mainly followed
[How To Setup Local CV](https://www.kaggle.com/competitions/h-and-m-personalized-fashion-recommendations/discussion/308919)
, I created 5 folds and used 3 folds as training data. I also tried more training data, but it didn’t increase the local CV and LB.
Recall model
I developed 4 recall models in total.
Item2item CF: the relationship between the pair of items (the user who bought Y, also bought Z).
repurchase: latest 20 purchased products by each customers.
popular: popular ranking in last week.
Two Tower MMoE: I mainly focused on improving the performance of this model. Because it can generate candidates of arbitrary length for all users, and the user/item embeddings (similarity) can also be used as an important feature for ranking model.
I used a gating network to make sure that the user tower can learn well by using different experts for recent active customers and non active customers.
[](https://postimg.cc/phyh6v5S)
[](https://postimg.cc/crGCQwh2)
[](https://postimg.cc/kRQnjYsZ)
Ranking model
I mainly used lightgbm with lambda-rank objective. I also implement DCN model described in
[https://arxiv.org/abs/2008.13535](https://arxiv.org/abs/2008.13535)
. But I didn’t have enough time to tune it. The score of lightgbm is much better.
Cold-start users
Two Tower MMoE can also generate candidates using user demographic features for customers without purchase log.
Cold-start items
Besides product meta features, I also extracted the following features from text and image.
Text: Extract TF-IDF features from product description, cluster products using SVD + K-means. Then use cluster id as feature.
Image: Extract image vectors using pre-trained tf_efficientnet_b3_ns backbone, cluster products using PCA + K-means.  Then use cluster id as feature.
Submission
best local CV is 0.039, LB is 0.0349.
My final submission combined my previous submissions by
[h-m-ensembling-how-to](https://www.kaggle.com/code/titericz/h-m-ensembling-how-to)
to be updated