{
  "id": 288365,
  "title": "37th Place Solution [Sad MT5 Vibes]",
  "content": "<p>I'd like to thank Google research and Kaggle for hosting this competition through this competition I have gained a lot of experience and the field of NLP.  this was my first NLP competition on Kaggle and I'm glad to have scored a silver medal which would tentatively make me a Kaggle competitions expert. 😃</p>\n<h2>Approach 1 : Question answering</h2>\n<p>I trained the following models namely XLM-Roberta-large , MURIL , REMBERT , BERT trained on XQUAD. All models here have been trained for 5 folds with extra data. MLQA, Hindi xQuad, Tamil translated SQUAD.</p>\n<table>\n<thead>\n<tr>\n<th>Model Name</th>\n<th>H &amp; T Split</th>\n<th>Public LB Score</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>XLM Roberta Large</td>\n<td>No</td>\n<td>0.771</td>\n</tr>\n<tr>\n<td>XLM Roberta Large</td>\n<td>Yes</td>\n<td>0.754</td>\n</tr>\n<tr>\n<td>MURIL</td>\n<td>Yes</td>\n<td>0.738</td>\n</tr>\n<tr>\n<td>Rembert</td>\n<td>No</td>\n<td>0.788</td>\n</tr>\n<tr>\n<td>BERT</td>\n<td>No</td>\n<td>0.642</td>\n</tr>\n<tr>\n<td>InfoXLM</td>\n<td>No</td>\n<td>0.714</td>\n</tr>\n</tbody>\n</table>\n<p><em>My final model was a single five-fold ensemble of from Rembert trained on the extra data available from MLQA and XQUAD and Tamil translated SQUAD</em></p>\n<p>I would have tried ensembling models but averaging scores did note seem to work and as i had started seriously getting into the competition only in the last week I did not have time to figure out a voting pipeline as many other solutions have done.</p>\n<h2>Approach 2: Seq2Seq (Incomplete) 😭</h2>\n<p>the second approach I attempted was by building a text to text transformer model using the mT5-base transformer. This approach showed promise as it was the only one in which all the correct letters were being predicted but in this I face the unique and weird problem that none of the spaces between the words were being predicted. hence I could not properly post process the outputs or this model to get a get a proper submission. I would be really grateful if someone from the Kaggle community would be able to help me out with this problem.</p>\n<p>The links to my notebooks are as follows:</p>\n<ul>\n<li>Training Notebook : <a href=\"https://www.kaggle.com/aishikai/chaii-seq2seq-mt5-fit\" target=\"_blank\">Click here</a></li>\n<li>Seq2Seq Training notebook : <a href=\"https://www.kaggle.com/aishikai/chaii-rembert-fit\" target=\"_blank\">Click Here</a></li>\n<li>Inference Notebook : <a href=\"https://www.kaggle.com/aishikai/chaii-inference\" target=\"_blank\">Click Here</a></li>\n<li>H&amp;T Split Inference Notebook :  <a href=\"https://www.kaggle.com/aishikai/chaii-inference-ht-split\" target=\"_blank\">Click Here</a></li>\n</ul>",
  "comments": [
    {
      "id": 1587806,
      "author_name": "MINGOOK KIM",
      "author_url": "",
      "post_date": "2021-11-19T00:15:13.800000",
      "content": "<p>Thank you for sharing ^O^!</p>",
      "votes": 1,
      "replies": []
    },
    {
      "id": 1585638,
      "author_name": "Aayush Sharma",
      "author_url": "",
      "post_date": "2021-11-17T12:01:10.773000",
      "content": "<p>Thanks for sharing, really helpful!</p>",
      "votes": 1,
      "replies": []
    },
    {
      "id": 1585581,
      "author_name": "Shreya Sajal",
      "author_url": "",
      "post_date": "2021-11-17T11:05:27.743000",
      "content": "<p>Thanks for sharing aishikkk,you rock!🥳</p>",
      "votes": 1,
      "replies": []
    }
  ],
  "comment_count": 3,
  "messages": [
    {
      "id": 1585546,
      "postDate": "2021-11-17T10:20:52.353Z",
      "content": "<p>I'd like to thank Google research and Kaggle for hosting this competition through this competition I have gained a lot of experience and the field of NLP.  this was my first NLP competition on Kaggle and I'm glad to have scored a silver medal which would tentatively make me a Kaggle competitions expert. 😃</p>\n<h2>Approach 1 : Question answering</h2>\n<p>I trained the following models namely XLM-Roberta-large , MURIL , REMBERT , BERT trained on XQUAD. All models here have been trained for 5 folds with extra data. MLQA, Hindi xQuad, Tamil translated SQUAD.</p>\n<table>\n<thead>\n<tr>\n<th>Model Name</th>\n<th>H &amp; T Split</th>\n<th>Public LB Score</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>XLM Roberta Large</td>\n<td>No</td>\n<td>0.771</td>\n</tr>\n<tr>\n<td>XLM Roberta Large</td>\n<td>Yes</td>\n<td>0.754</td>\n</tr>\n<tr>\n<td>MURIL</td>\n<td>Yes</td>\n<td>0.738</td>\n</tr>\n<tr>\n<td>Rembert</td>\n<td>No</td>\n<td>0.788</td>\n</tr>\n<tr>\n<td>BERT</td>\n<td>No</td>\n<td>0.642</td>\n</tr>\n<tr>\n<td>InfoXLM</td>\n<td>No</td>\n<td>0.714</td>\n</tr>\n</tbody>\n</table>\n<p><em>My final model was a single five-fold ensemble of from Rembert trained on the extra data available from MLQA and XQUAD and Tamil translated SQUAD</em></p>\n<p>I would have tried ensembling models but averaging scores did note seem to work and as i had started seriously getting into the competition only in the last week I did not have time to figure out a voting pipeline as many other solutions have done.</p>\n<h2>Approach 2: Seq2Seq (Incomplete) 😭</h2>\n<p>the second approach I attempted was by building a text to text transformer model using the mT5-base transformer. This approach showed promise as it was the only one in which all the correct letters were being predicted but in this I face the unique and weird problem that none of the spaces between the words were being predicted. hence I could not properly post process the outputs or this model to get a get a proper submission. I would be really grateful if someone from the Kaggle community would be able to help me out with this problem.</p>\n<p>The links to my notebooks are as follows:</p>\n<ul>\n<li>Training Notebook : <a href=\"https://www.kaggle.com/aishikai/chaii-seq2seq-mt5-fit\" target=\"_blank\">Click here</a></li>\n<li>Seq2Seq Training notebook : <a href=\"https://www.kaggle.com/aishikai/chaii-rembert-fit\" target=\"_blank\">Click Here</a></li>\n<li>Inference Notebook : <a href=\"https://www.kaggle.com/aishikai/chaii-inference\" target=\"_blank\">Click Here</a></li>\n<li>H&amp;T Split Inference Notebook :  <a href=\"https://www.kaggle.com/aishikai/chaii-inference-ht-split\" target=\"_blank\">Click Here</a></li>\n</ul>",
      "rawMarkdown": "I'd like to thank Google research and Kaggle for hosting this competition through this competition I have gained a lot of experience and the field of NLP.  this was my first NLP competition on Kaggle and I'm glad to have scored a silver medal which would tentatively make me a Kaggle competitions expert. 😃\n## Approach 1 : Question answering\nI trained the following models namely XLM-Roberta-large , MURIL , REMBERT , BERT trained on XQUAD. All models here have been trained for 5 folds with extra data. MLQA, Hindi xQuad, Tamil translated SQUAD.\n| Model Name | H & T Split | Public LB Score |\n| ---- | ---- | ---- |\n| XLM Roberta Large | No | 0.771 |\n| XLM Roberta Large | Yes | 0.754 |\n| MURIL | Yes | 0.738 |\n| Rembert | No | 0.788 |\n| BERT | No | 0.642 |\n| InfoXLM  | No  | 0.714 |\n\n*My final model was a single five-fold ensemble of from Rembert trained on the extra data available from MLQA and XQUAD and Tamil translated SQUAD*\n\nI would have tried ensembling models but averaging scores did note seem to work and as i had started seriously getting into the competition only in the last week I did not have time to figure out a voting pipeline as many other solutions have done.\n\n## Approach 2: Seq2Seq (Incomplete) 😭\nthe second approach I attempted was by building a text to text transformer model using the mT5-base transformer. This approach showed promise as it was the only one in which all the correct letters were being predicted but in this I face the unique and weird problem that none of the spaces between the words were being predicted. hence I could not properly post process the outputs or this model to get a get a proper submission. I would be really grateful if someone from the Kaggle community would be able to help me out with this problem.\n\nThe links to my notebooks are as follows:\n- Training Notebook : [Click here](https://www.kaggle.com/aishikai/chaii-seq2seq-mt5-fit)\n- Seq2Seq Training notebook : [Click Here](https://www.kaggle.com/aishikai/chaii-rembert-fit)\n- Inference Notebook : [Click Here](https://www.kaggle.com/aishikai/chaii-inference)\n- H&T Split Inference Notebook :  [Click Here](https://www.kaggle.com/aishikai/chaii-inference-ht-split)",
      "votes": 13
    },
    {
      "id": 1587806,
      "postDate": "2021-11-19T00:15:13.800Z",
      "content": "<p>Thank you for sharing ^O^!</p>",
      "rawMarkdown": "Thank you for sharing ^O^!",
      "votes": 1
    },
    {
      "id": 1585638,
      "postDate": "2021-11-17T12:01:10.773Z",
      "content": "<p>Thanks for sharing, really helpful!</p>",
      "rawMarkdown": "Thanks for sharing, really helpful!",
      "votes": 1
    },
    {
      "id": 1585581,
      "postDate": "2021-11-17T11:05:27.743Z",
      "content": "<p>Thanks for sharing aishikkk,you rock!🥳</p>",
      "rawMarkdown": "Thanks for sharing aishikkk,you rock!🥳",
      "votes": 1
    }
  ],
  "complete": true,
  "source": "live",
  "fetched_at": "2026-10-19T16:19:29Z",
  "competition_id": 30060,
  "competition_slug": "chaii-hindi-and-tamil-question-answering",
  "team_id": 7310405,
  "solution_url": "/competitions/chaii-hindi-and-tamil-question-answering/writeups/me-so-sad-37th-place-solution-sad-mt5-vibes"
}