# 63th place solution

63th Place Solution
Thanks to my teammates
[@ootake](https://www.kaggle.com/ootake)
,
[@ryoichi0917](https://www.kaggle.com/ryoichi0917)
,
[@xia464](https://www.kaggle.com/xia464)
, especially
[@tyonemoto](https://www.kaggle.com/tyonemoto)
for inviting me.
Code (updated in 2022-07-12)
I publish the entire code.
code:
[https://github.com/Y-Haneji/kaggle-handm](https://github.com/Y-Haneji/kaggle-handm)
Last Submission
(lr=0.1) : Private LB: 0.02813, Public LB: 0.02826, Validation: 0.033623
(lr=0.02): Private LB: 0.02772, Public LB: 0.0277, Validation: 0.033814
Summary
I think that the key points of this competition are these.
Stable validation.
Two stage: generating candidates by some strategies, then ranking them.
Better strategies and scores from them.
Using full information like not only transactions but also detail descriptions, images, and so on.
Ranking Model (me)
Model
LightGBM lambdarank
Dataset
This competition gave no training datasets, only transactions datas and some metadatas. I needed to generate training datasets by myself. The dataset was like this.
customer_id
article_id (candidate)
score from generative strategy (may not just one score)
flags wheter the candidate included each strategy
other features of customers
other features of articles
label (0 or 1)
The features of customers and articles are purchase and sales of …
each week
agregation (sum, mean, max, min) of all weeks
difference and division between each week and aggregation
Label is whether the article was bought target_week.
The details of strategies to generate candidates is written in below.
Training
5 target weeks, this means 5 folds training dataset.
For training, I only generated candidates for the customers who had some purchases in the target week.
Model Validation: NDCG@12. The dataset of next target week was validation dataset. The last target week's training dataset had no validation datasets.
Submission Validation: MAP@12 for last week's transactions.
Parameter
objective: lambdarank
lr: 0.1 or 0.02
feature_fraction: 0.8
baggin_fraction: 0.8
early_stopping: 10 or 50 rounds (depend on lr)
Other parameters is default. I tried Optuna but it didn't work well for lambdarank because it attempted to LOWER the MAP.
Prediction (Submission)
For prediction, I should generate candidates for all the customers, which causes a memory problem.
Split the customers into batches.
Ensemble 5 target week's models.
Generally take longer time than training
If validation was not good, I didn't compute submission.
Candidates Strategies
Worked (included last submission)
trending (original:
[here](https://www.kaggle.com/code/byfone/h-m-trending-products-weekly)
)
recently purchased
bin popular
last week for each customer
also bought this (original:
[here](https://www.kaggle.com/code/cdeotte/customers-who-bought-this-frequently-buy-this)
)
collabo
t-SNE KNN
Albert KNN
sequence Word2Vec
Not Worked
Bert KNN
VGG KNN
Gru4Rec, Bert4Rec (LSTM) (original:
[here](https://www.kaggle.com/code/astrung/lstm-model-with-item-infor-fix-missing-last-item)
)
Implicit ALS
Details of Some Strategies
TO BE UPDATED: Some strategies will be written in some days.
bin popular (by me and by
[@tyonemoto](https://www.kaggle.com/tyonemoto)
)
To recommend popular item was effecitve especially for cold start customers. Different kind of customers should have different preference. So we split customers into some groups by their features, like age, sex, and so on.
In the experiment phase, we used age, price, and transaction channel for binning. The number of splitting is 10 for age, 20 for price, and 3 for channels. Recall of this strategy was 0.06750.
Because of the deadline, we have no choice but to adopt only age bins.
collabo (by
[@tyonemoto](https://www.kaggle.com/tyonemoto)
)
The collaboration articles with Kangol was launched a few weeks ago from the submission week. We recommended popular collaboration articles to customers who had bought some collaboration articles. Recall of this strategy was 0.00094.
t-SNE (by
[@tyonemoto](https://www.kaggle.com/tyonemoto)
)
The metadatas of articles from the file of articles.csv and embedding vectors of detail descriptions by tf-idf and bert are compressed to two dimensions by t-SNE. We used count encoding method to deal with the category metadatas columns.
Albert (by
[@ryoichi0917](https://www.kaggle.com/ryoichi0917)
)
sequence Word2Vec (by
[@tyonemoto](https://www.kaggle.com/tyonemoto)
)
Bert (by
[@tyonemoto](https://www.kaggle.com/tyonemoto)
and
[@ootake](https://www.kaggle.com/ootake)
)
We used bert pretrained model named "bert-uncased", "MRPC", and "SQuAD" to extract features from articles' detail description.
VGG (by me)
I extracted features from images using VGG because their vectors were relatively low dimensions, which can avoid a memory problem. But even they reached 10GB files (when compressed to .npy).
VGG KNN doesn't worked, but I wonder if I could use other bigger models to extract features.
Implicit ALS (by
[@ootake](https://www.kaggle.com/ootake)
)
Comparision
The true purpose of this competition is not to explore strategies which have better Precision and Recall, but to obtain better MAP@12 for submission, so I compared each strategy based on MAP@12.
Strategy
MAP@12 change
trending
base lines
recently
base lines
popular
base lines
last week
-0.000131
t-SNE
+0.00053
umap
-0.0001
t-SNE & umap
-0.000898
also bought this
+0.000533
albert
+0.000241
collaboration
+0.000134
Bert
+0.000051 (didn't correlate with LB)
sequence Word2Vec w/ compression
+0.000367
bin popular
+0.00058
bert4rec
-0.00037
gru4rec
-0.000347
Implicit ALS
+0.000142
sequence Word2Vec
-0.000563
VGG
-0.000375
Note: This is the PROCESS of adding or subtracting the strategies rather than the result of comparision experiment.
Computing Resources
Untill 2 weeks from deadlines: MacBook Air (M1, 2020) (8 core CPU, 16 GM RAM, no GPU for deep learning).
Last 2 weeks: GCP instances to deal with more candidates. (48 core CPU, 384 GB RAM)
Some Story
This is my first medal in Kaggle, so I'm very happy with my place!
I am new to a recommend problem, but my teammates helped me a lot, and
[@paweljankiewicz](https://www.kaggle.com/paweljankiewicz)
and
[@lihaorocky](https://www.kaggle.com/lihaorocky)
was so kind to answer my questions in discussion.