# silver solution (44th) 

Congratulations to winners! And Thank you to Kaggle for hosting this.
My Bert model had USE features + Crawl embeddings.  USE features improved my score quite a bit. Crawl not so much. I had answers and questions model separate, a little later I realized I could have saved lot of time by having QA branch in the model itself (too late).  My post processing was from public kernel. (All PyTorch)
Roberta was surprisingly good pretty close to Bert on public LB, considering the amount of time I spent on Roberta vs Bert. This morning I realized that my Roberta model isn't using  USE Features - it was too late, had to let it go.
Preprocessing:
None for Bert, For crawl I used the preprocessing from public kernel.
Tokenization
Head + Tail approach. 
Custom loss function
I'll have my training code on GitHub and I'll make my inference kernel also public.
Look forward to reading better solutions !
Edit:
[Inference Kernel](https://www.kaggle.com/rashmibanthia/googlequest-inference?scriptVersionId=28474231)