# 23th place solution

First, thanks to my teammate @agatan .
Pre-processing
apply html unescape to sentences
sentances longer than maximum length was trimmed at the head and tail parts
Modeling
We trained Question Bert-base model and Question and Anser Bert-base model separetely on kernel
Question model predicts question-targets and Q-A model predicts all targets
3fold and 3epochs with GroupKFold
BCE + margin ranking loss
Post-processing
used lightgbm as second-stage stacking model
max_depth=1 and lr=0.1 was best from my experiments
Input meta features such as text-length to lightgbm addition to predicted values
Best model
CV: 0.3851
Public: 0.45979
Private: 0.41440
We shared our kernel, please see if you like.
[https://www.kaggle.com/shuheigoda/23th-place-solusion](https://www.kaggle.com/shuheigoda/23th-place-solusion)