# 10th Place Solution

Data
In my final submission I only used models which were trained on
[MLQA](https://github.com/facebookresearch/MLQA)
,
[xquad](https://github.com/deepmind/xquad)
and the competition data.
I experimented a little bit with data augmentation but I was not able to improve my score.
Models
My final submission was an ensemble of finetuned
XLM-Roberta
(pretrained on squad 2) and
MuRIL
Training
I split the external data + the competition data into
5 folds
and trained both models on the five folds.
During training I ussed
early stopping
: after each epoch I checked the Jaccard Similarity on the oof split and afterwards used the weights of the epoch where JS was highest (usually this was after the second epoch).
I used
Kaggle TPUs
for training.
Ensembling
I first ensembled by taking the mean over the 5 folds for both xlm-rob and muril. I then determined for both models the top n predicted strings based on the sum of the start and end logits.
In order to get the final prediction I then used the following algorithm:
`
For each model:
Store the top n postprocessed predicted strings in a list.
Sort the list (from low to high) based on the score of the predicted string (where the score is the sum of the start and end char logit).
Assign a new score to each predicted string, by simply taking its index in the list.
For each predicted string:
Sum up the new scores from each model.
Pick the string with the highest sum as the final prediction.
`
I also tried mapping the token logits back to the characters but I was able to achieve better results using the algorithm above.
Implementation
I used
tensorflow
and uploaded my code
[here](https://github.com/drehero/kaggle-solutions/tree/main/chaii-hindi-and-tamil-question-answering-10th-place)
.
Other
I also trained
RemBert
, but unfortionately I was not able to ensemble all three models because I always ran into "Notebook Exceeded Allowed Compute" errors during submission.
Acknowledgements
Thanks to the community. You shared interesting ideas in discussions and notebooks and I was able to learn a lot from you. The following notebooks where especially helpful to me:
[https://www.kaggle.com/rhtsingh/chaii-qa-5-fold-xlmroberta-torch-fit](https://www.kaggle.com/rhtsingh/chaii-qa-5-fold-xlmroberta-torch-fit)
[https://www.kaggle.com/rhtsingh/external-data-mlqa-xquad-preprocessing](https://www.kaggle.com/rhtsingh/external-data-mlqa-xquad-preprocessing)
[https://www.kaggle.com/thedrcat/chaii-eda-baseline](https://www.kaggle.com/thedrcat/chaii-eda-baseline)
[https://www.kaggle.com/kishalmandal/5-folds-infer-combined-model-0-792](https://www.kaggle.com/kishalmandal/5-folds-infer-combined-model-0-792)