# 36th Place - Trust Your CV - Optimize PP

Thank you Kaggle and host for a fun and  interesting competition.
Thank you
[@kishalmandal](https://www.kaggle.com/kishalmandal)
for the wonderful public notebooks. I joined a few days ago and immediately studied Kishal's top scoring public notebooks.
Final Submission 1 - Best LB
My first final submission was my maximum
public LB of 0.795
. I took Kishal's LB 0.792 public notebook
[here](https://www.kaggle.com/kishalmandal/5-folds-infer-combined-model-0-792)
and changed the weights to
start_logits = (start_logits1 + 0.85*start_logits2 + 0.85*start_logits3
       + start_logits4+ 1.7*start_logits5 )/5.4
end_logits = (end_logits1 + 0.85*end_logits2 + 0.85*end_logits3 
       + end_logits4 + 1.7*end_logits5)/5.4
This achieved public LB 0.795 which was 60th place on public LB. But when i calculated the local CV score, it was only
CV 0.677
.  This was very concerning. (And as it turns out, this submission had private LB 0.718 and had 728th place 😮 so it was overfitting public LB)
Final Submission 2 - Best CV
For my second final submission, I downloaded
[@kishalmandal](https://www.kaggle.com/kishalmandal)
train notebooks
[here](https://www.kaggle.com/kishalmandal/chaii-fit-2-epochs-mlqa-xquad-chaii/)
and
[here](https://www.kaggle.com/kishalmandal/chaii-fit-7-epochs-extra-tamil-data/)
. I optimized the hyperparameters, learning rates, and learning schedules and achieved local
CV 0.697
.
Modify Boost Post Process +0.004!
Next, I adjusted the popular public post process. I changed the first line which uses logits to the second line which uses the Bayesian product of probabilities (and gives start position a slight emphasis over end position).
"score": start_logits[start_index] + end_logits[end_index]

from scipy.special import expit  #THIS IS SIGMOID FUNCTION
"score": expit(1.2*start_logits[start_index]) \
                  * expit(end_logits[end_index])
This boosted CV by +0.003, public LB by +0.003, and private LB by +0.004!
Best Final Submission
My best final submission turned out to be my second submission which was my best CV and not my best LB. This model had CV 0.700, Public LB 0.784, and Private LB 0.744. Once again, "trust you CV" was the best strategy!
UPDATE -  Post Comp Reflection
My solution is only XLM-Roberta. After reading top solutions, it appears that ensemble with
Muril-Large
(and maybe
Rembert
) was the secret sauce to achieving a higher LB score! And TydiQA external dataset.