# 5th place solution

First of all, thanks to the competition organizers to host such an interesting competition and my great teammates
[@rendongltt](https://www.kaggle.com/rendongltt)
(who is a huge fan of online-shopping, which gives us a lot of help😂) and
[@jiaqizhang35](https://www.kaggle.com/jiaqizhang35)
(who is my friend, without whom I didn't even join this competition). And also I would like to thank
[@paweljankiewicz](https://www.kaggle.com/paweljankiewicz)
, without his great posts, we will not reach this far. And also you (and also a lot of other nice guys, who love sharing) motivated me to do more sharing with the community.
Overview
Overall we love this competition so much because it did provide infinite ways to solve this problem especially the possible recall methods. Talking to feature engineering, even though we're quite confident about our skills, but we may have no shot to get a gold medal compared to other grandmasters. So we focused ourselves on the recall methods from the very start. And our recall methods is deeply dependent on article embeddings (image, txt, word2vec, tfidf) and the amazing python package faiss to find the similar articles and customers. As to the feature engineering, we didn't spend too much time on it but using different days window to create aggreated stat features of articles and customers seperatly and customer-article cross aggreated stat features and different cosine similarity feaures using embeddings aforementioned.
Recall methods
The recall methods comprised of those parts:
Customer's last buckets(and same product code to customers' last bucket's articles), recent bought articles
U2I: User based collaborative filtering.
I2i: Item based collaborative filtering
word2vec: to predict customer's next possible purchase articles after last buckets
Articles bought together. I modified this nice
[notebook](https://www.kaggle.com/code/titericz/article-id-pairs-in-3s-using-cudf)
from
[@titericz](https://www.kaggle.com/titericz)
Popular articles with different customers' attributes like age bin, gender etc.
So overall we use 21 recall methods to create article candidates.
Embedding methods
We created article embdding using different data and methods
For article images, we use swin transformer to extract the embedding features.
For the article texts, first we concatnate all text values of each article and then I use SentenceTransformer to extract the embeddings
Using this great
[notebook](https://www.kaggle.com/code/aerdem4/h-m-rapids-article2vec)
from
[@aerdem4](https://www.kaggle.com/aerdem4)
you can easily get article embeddings using tfidf method
The last but also the most useful method to get article embeddings are word2vec method, for which we use gensim to ease the process.
Recall method calibration
Because different recall methods have different positive ratio, so it's better to use some method to calibrate all the recall methods. I use the method which I shared
[here](https://www.kaggle.com/competitions/h-and-m-personalized-fashion-recommendations/discussion/321672#:~:text=I%20would%20sugguest,goes%20like%20this.)
to do the preprocess.
Feature engineering
Nothing special about this part but I found some method which will hugely accelerate the computing time for feature computing.
Basicly we divide the features into 3 parts:
articles features, which will be constant to all customers once you fixed the overvation time
similarly, customer features will be contant to all candidate articles once you fixed the overvation date
customer-article features, like cosine similarity between each pair of customer and article, customers' bought history aggreated features of this particular article.
For part 1 and 2, you could do the computation and save them once and merged with the features of part 3 later. This method really save me a lot of time especially during inference time.
Modeling
Because we spend most of the time to devolop recall strategies, we use only lightgbm ranker model for training and don't even have time to try other methods like catboost and xgboost. So basically our score is single model's score.
Remaining question
We have one question that we still couldn't solve it right now, which is about the word2vec model. We use all transaction data to train the word2vec model, which actually leaked the future info a little bit.
Imagne that for observation date 20200915 you compute the customer-article cosine similarity using the word2vec model trained using all transaction data. The word2vec embedings of two article ids aid_1, aid_2, which maybe mainly appeared duing week 20200916-20200922, are used on the observation date 20200915.
The effect is everytime I use those embeddings to create candidates or compute features, the local CV will improve a lot while the LB improves only a little bit. For my final ranking model, my map@12 score for last week is 0.0441, but the LB is only 0.0350, the gap is much larger than the ones mentioned by others. But if we don't use this feature, both CV and LB score are dropping. So we don't really know how to do it to make the most use of this method. What do you think?