# 83rd place solution

Github repo:
[https://github.com/btrotta/kaggle-google-quest](https://github.com/btrotta/kaggle-google-quest)
My code relies heavily on the following public notebook:
[https://www.kaggle.com/akensert/bert-base-tf2-0-now-huggingface-transformer](https://www.kaggle.com/akensert/bert-base-tf2-0-now-huggingface-transformer)
My BERT modelling code is almost identical to that kernel; the only change I made was to insert a special token between the question title and body. (I'm not sure whether this had any real effect.) As in the script, I did 5 folds and 3 epochs.
To optimise the Spearman rho metric, I found it was helpful to limit the predicted scores to a smaller number of distinct values. The way I did this was by post-processing using a LightGBM model with small leaf size (3) and few iterations (20). The model just takes a single predictor (the output prediction of the BERT model) and optimises the cross-entropy loss. This gives a boost of around 0.02 on the private leaderboard, for a final score of 0.39507.