# 52nd place with 20 minute Kaggle Notebook

I was able to generate my final submission - including candidate retrieval, feature engineering, and training/predicting with two models - in just 20 minutes.
Partially, it was from using cudf for whatever I could, and it's really amazing how much time it saves.
But I think the main reason was that I used a light-weight candidate retrieval method that gave me a good result without needing many candidates per customer, and without needing collaborative filtering or image/text processing.
I had 33 million candidates (which comes out to only ~25 per customer on average).
I started with
[@cdeotte](https://www.kaggle.com/cdeotte)
's notebooks -
[customers who bought this frequently bought this](https://www.kaggle.com/code/cdeotte/customers-who-bought-this-frequently-buy-this)
and
[Recommend Items Purchased Together](https://www.kaggle.com/code/cdeotte/recommend-items-purchased-together-0-021)
.
I refactored his first notebook in
[this notebook](https://www.kaggle.com/code/jacob34/cdeotte-pairs-in-3-minutes)
, so that I could get the same result in 3 minutes.
Then I made several changes:
I had it only look for "pairs" from articles that were sold in the past week
I had it find 5 "pairs" for each article, instead of just 1
I ignored any article that wasn't sold within the past week
I ignored any "pair" purchased by less than 2 customers
We get a very strong retrieval method, a sort of manual, time-aware collaborative filtering method - what customers with similar purchase interests were purchased
in the past week
- so it includes trend information as well.
For the second part, I followed his method of recommending "pairs" of items a customer had purchased in their last few weeks of history.
And I also kept features to feed to LGBMRanker about:
the strength of the "match" (i.e. how many/what percentage of customers the "pair" was based on)
the "source" of the pair (i.e. how recently the original article was purchased, how many times it was purchased)
Of course, I also generated the regular, standard candidates: past purchases, and 12 most popular items (I generated those based on age group).
I think this retrieval method was the main reason I was able to get a good result with relatively few candidates/resources.
Update - code
Some people asked for it, so I made a clean notebook (moved nearly all code to the repo).
Here's
[the notebook](https://www.kaggle.com/jacob34/clear-n-simple-final-shared-notebook)
, here's
[the repo](https://github.com/JacobCP/kaggle-handm-helpers)
, and here's the
[kaggle dataset](https://www.kaggle.com/datasets/jacob34/handmhelpers)
that syncs with the repo.
The notebook runs in 20 minutes, and gets the same private LB score (it varies a bit per run).
My workflow was:
I'd work with a function in the notebook (for example, a candidate generator)
Once the function was working, I'd move it to the repo and point the notebook there.
Any parameters of the function (for example, candidate threshold), I'd add to my
params
dictionary in the notebook.
Each function had a
**kwargs
argument, so I could pass the same single
params
dictionary to every function, and the function would use the arguments it needed.
If I needed to revisit a function, or work with the values generated in middle, I'd move the function/s back to the notebook.
Update #2 - cuML's FIL
Afterwards, I refactored my code to use cuML's FIL for inference, and that brought notebook time down from 20 minutes to just 12 minutes!