# 3rd Place Solution

Muril-Large + Magic
First of all, I would like to thank Kaggle and the competition host for hosting this challenge.
The third place came as a surprise to me and I was not expecting it at all, since my public LB score was not high.
Muril is all I have. I have found that xlm-r is more efficient when finetuning on the squad 2.0 dataset, so I tried it similarly with muril-large.
Data
competition data, MLQA, XQUAD, SQUAD v2
Script to finetune muril on squad 2.0:
[https://www.kaggle.com/nguyenduongthanh/fine-tune-muril-squad-v2](https://www.kaggle.com/nguyenduongthanh/fine-tune-muril-squad-v2)
Muril-Large-Squad-v2 Checkpoint
[https://www.kaggle.com/nguyenduongthanh/muril-large-squad-v2](https://www.kaggle.com/nguyenduongthanh/muril-large-squad-v2)
Fine-tuning:
I just fine-tune on 2 epochs each fold
MuRIL Large 10 folds: max-length=384, doc_stride=128, lr=1e-5, gradient accumulation=1
Ensemble
Simple average of 10 folds
Notebook
Train:
[https://www.kaggle.com/nguyenduongthanh/simple-train](https://www.kaggle.com/nguyenduongthanh/simple-train)
Inference :
[https://www.kaggle.com/nguyenduongthanh/simple-10-folds-of-muril-large](https://www.kaggle.com/nguyenduongthanh/simple-10-folds-of-muril-large)