{ "id": 288001, "title": "10th Place Solution", "content": "

Data

\n

In my final submission I only used models which were trained on MLQA, xquad and the competition data.
\nI experimented a little bit with data augmentation but I was not able to improve my score.

\n

Models

\n

My final submission was an ensemble of finetuned

\n\n

Training

\n

I split the external data + the competition data into 5 folds and trained both models on the five folds.

\n

During training I ussed early stopping: after each epoch I checked the Jaccard Similarity on the oof split and afterwards used the weights of the epoch where JS was highest (usually this was after the second epoch).

\n

I used Kaggle TPUs for training.

\n

Ensembling

\n

I first ensembled by taking the mean over the 5 folds for both xlm-rob and muril. I then determined for both models the top n predicted strings based on the sum of the start and end logits.

\n

In order to get the final prediction I then used the following algorithm:

\n

`
\nFor each model:

\n
    \n
  1. Store the top n postprocessed predicted strings in a list.
    2. \n
  2. Sort the list (from low to high) based on the score of the predicted string (where the score is the sum of the start and end char logit).
    3. \n
  3. Assign a new score to each predicted string, by simply taking its index in the list.
    4. \n
\n

For each predicted string:

\n
    \n
  1. Sum up the new scores from each model.
    2. \n
  2. Pick the string with the highest sum as the final prediction.
    \n`
    3. \n
\n

I also tried mapping the token logits back to the characters but I was able to achieve better results using the algorithm above.

\n

Implementation

\n

I used tensorflow and uploaded my code here.

\n

Other

\n

I also trained RemBert, but unfortionately I was not able to ensemble all three models because I always ran into \"Notebook Exceeded Allowed Compute\" errors during submission.

\n

Acknowledgements

\n

Thanks to the community. You shared interesting ideas in discussions and notebooks and I was able to learn a lot from you. The following notebooks where especially helpful to me:

\n", "comments": [ { "id": 1584833, "author_name": "Darek Kłeczek", "author_url": "", "post_date": "2021-11-16T19:48:55.867000", "content": "

Congratulations! Interesting ensembling approach, thanks for sharing!

", "votes": 1, "replies": [] }, { "id": 1584638, "author_name": "Raghawendra Singh", "author_url": "", "post_date": "2021-11-16T16:39:16.177000", "content": "

Congrats for solo gold…

", "votes": 1, "replies": [] }, { "id": 1584606, "author_name": "Kiran Chaudhary", "author_url": "", "post_date": "2021-11-16T16:15:30.730000", "content": "

Congratulations ….

", "votes": 1, "replies": [] } ], "comment_count": 3, "messages": [ { "id": 1584113, "postDate": "2021-11-16T09:29:34.333Z", "content": "

Data

\n

In my final submission I only used models which were trained on MLQA, xquad and the competition data.
\nI experimented a little bit with data augmentation but I was not able to improve my score.

\n

Models

\n

My final submission was an ensemble of finetuned

\n\n

Training

\n

I split the external data + the competition data into 5 folds and trained both models on the five folds.

\n

During training I ussed early stopping: after each epoch I checked the Jaccard Similarity on the oof split and afterwards used the weights of the epoch where JS was highest (usually this was after the second epoch).

\n

I used Kaggle TPUs for training.

\n

Ensembling

\n

I first ensembled by taking the mean over the 5 folds for both xlm-rob and muril. I then determined for both models the top n predicted strings based on the sum of the start and end logits.

\n

In order to get the final prediction I then used the following algorithm:

\n

`
\nFor each model:

\n
    \n
  1. Store the top n postprocessed predicted strings in a list.
    2. \n
  2. Sort the list (from low to high) based on the score of the predicted string (where the score is the sum of the start and end char logit).
    3. \n
  3. Assign a new score to each predicted string, by simply taking its index in the list.
    4. \n
\n

For each predicted string:

\n
    \n
  1. Sum up the new scores from each model.
    2. \n
  2. Pick the string with the highest sum as the final prediction.
    \n`
    3. \n
\n

I also tried mapping the token logits back to the characters but I was able to achieve better results using the algorithm above.

\n

Implementation

\n

I used tensorflow and uploaded my code here.

\n

Other

\n

I also trained RemBert, but unfortionately I was not able to ensemble all three models because I always ran into \"Notebook Exceeded Allowed Compute\" errors during submission.

\n

Acknowledgements

\n

Thanks to the community. You shared interesting ideas in discussions and notebooks and I was able to learn a lot from you. The following notebooks where especially helpful to me:

\n", "rawMarkdown": "## Data\n\nIn my final submission I only used models which were trained on [MLQA](https://github.com/facebookresearch/MLQA), [xquad](https://github.com/deepmind/xquad) and the competition data.\nI experimented a little bit with data augmentation but I was not able to improve my score.\n\n## Models\n\nMy final submission was an ensemble of finetuned\n* **XLM-Roberta** (pretrained on squad 2) and\n* **MuRIL**\n\n## Training\n\nI split the external data + the competition data into **5 folds** and trained both models on the five folds.\n\nDuring training I ussed **early stopping**: after each epoch I checked the Jaccard Similarity on the oof split and afterwards used the weights of the epoch where JS was highest (usually this was after the second epoch).\n\nI used **Kaggle TPUs** for training.\n\n## Ensembling\n\nI first ensembled by taking the mean over the 5 folds for both xlm-rob and muril. I then determined for both models the top n predicted strings based on the sum of the start and end logits.\n\nIn order to get the final prediction I then used the following algorithm:\n\n`\nFor each model:\n1. Store the top n postprocessed predicted strings in a list.\n2. Sort the list (from low to high) based on the score of the predicted string (where the score is the sum of the start and end char logit).\n3. Assign a new score to each predicted string, by simply taking its index in the list.\n\nFor each predicted string:\n1. Sum up the new scores from each model.\n2. Pick the string with the highest sum as the final prediction.\n`\n\nI also tried mapping the token logits back to the characters but I was able to achieve better results using the algorithm above.\n\n## Implementation\n\nI used **tensorflow** and uploaded my code [here](https://github.com/drehero/kaggle-solutions/tree/main/chaii-hindi-and-tamil-question-answering-10th-place).\n\n## Other\n\nI also trained **RemBert**, but unfortionately I was not able to ensemble all three models because I always ran into \"Notebook Exceeded Allowed Compute\" errors during submission.\n\n## Acknowledgements\n\nThanks to the community. You shared interesting ideas in discussions and notebooks and I was able to learn a lot from you. The following notebooks where especially helpful to me:\n\n* https://www.kaggle.com/rhtsingh/chaii-qa-5-fold-xlmroberta-torch-fit\n* https://www.kaggle.com/rhtsingh/external-data-mlqa-xquad-preprocessing\n* https://www.kaggle.com/thedrcat/chaii-eda-baseline\n* https://www.kaggle.com/kishalmandal/5-folds-infer-combined-model-0-792\n\n\n", "votes": 13 }, { "id": 1584833, "postDate": "2021-11-16T19:48:55.867Z", "content": "

Congratulations! Interesting ensembling approach, thanks for sharing!

", "rawMarkdown": "Congratulations! Interesting ensembling approach, thanks for sharing!", "votes": 1 }, { "id": 1584638, "postDate": "2021-11-16T16:39:16.177Z", "content": "

Congrats for solo gold…

", "rawMarkdown": "Congrats for solo gold...", "votes": 1 }, { "id": 1584606, "postDate": "2021-11-16T16:15:30.730Z", "content": "

Congratulations ….

", "rawMarkdown": "Congratulations ....", "votes": 1 } ], "complete": true, "source": "live", "fetched_at": "2026-10-19T16:19:15Z", "competition_id": 30060, "competition_slug": "chaii-hindi-and-tamil-question-answering", "team_id": 7346669, "solution_url": "/competitions/chaii-hindi-and-tamil-question-answering/writeups/drehero-10th-place-solution" }