{
  "id": 288058,
  "title": "12th Place Solution - top of the silver...",
  "content": "<p>First of all, I'd like to thank Kaggle and the competition hosts. This competition was really interesting because the training data is really small and we were encouraged to use external data.</p>\n<p>My solution is really simple. There are only two key points - Character Level Ensemble &amp; Postprocessing.</p>\n<h2>Machine Resources</h2>\n<p>Colab pro+ &amp; kaggle notebook</p>\n<h2>Experiment tracking</h2>\n<p>W&amp;B(<a href=\"https://wandb.ai/site\" target=\"_blank\">https://wandb.ai/site</a>)</p>\n<h2>Model</h2>\n<p>I trained three models.</p>\n<ul>\n<li>XLM-RoBERTa</li>\n</ul>\n<pre><code>model: deepset/xlm-roberta-large-squad2\ndata: competition dataset + MLQA + XQUAD\nfold: 5-fold\nmax_length: 384\nstride: 128\nepochs: 2\nlr: 2.0e-5\n</code></pre>\n<ul>\n<li>MuRIL</li>\n</ul>\n<pre><code>model: google/muril-large-cased\ndata: competition dataset + MLQA + XQUAD\nfold: 5-fold\nmax_length: 384\nstride: 128\nepochs: 2\nlr: 2.0e-5\n</code></pre>\n<ul>\n<li>RoBERTa  (WECHSEL)<br>\nThank you for sharing! <a href=\"https://www.kaggle.com/bminixhofer\" target=\"_blank\">@bminixhofer</a></li>\n</ul>\n<pre><code>hindi model: roberta-large-wechsel-hindi\ntamil model: roberta-large-wechsel-tamil\nhindi data: competition dataset + MLQA + XQUAD\ntamil data: competition dataset + translated SQuAD2.0\nfold: 5-fold\nmax_length: 400\nstride: 135\nepochs: 3\nlr: 2.0e-5\n</code></pre>\n<h4>result</h4>\n<table>\n<thead>\n<tr>\n<th>model</th>\n<th>CV</th>\n<th>Public LB</th>\n<th>Private LB</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>XLM-RoBERTa</td>\n<td>0.7060</td>\n<td>0.778</td>\n<td>0.738</td>\n</tr>\n<tr>\n<td>MuRIL</td>\n<td>0.7038</td>\n<td>0.760</td>\n<td>0.743</td>\n</tr>\n<tr>\n<td>RoBERTa</td>\n<td>0.7481 &amp; 0.5907</td>\n<td>0.763</td>\n<td>0.735</td>\n</tr>\n</tbody>\n</table>\n<p>MuRIL is strong💪</p>\n<h2>Character Level Ensemble</h2>\n<p>It's impossible to simply average above three models' output because their tokenizers are different. So I expanded token probability to character probability using offsets.<br>\nAt first, I tried ensembling XLM-RoBERTa &amp; MuRIL. It scored 0.782 in public LB (+0.004 boost), but 0.754 in private LB (+0.011 boost)!!</p>\n<h2>Postprocessing</h2>\n<p>On the last weekend, I checked how the models were getting the answers wrong.<br>\nI found some predictions are consist of Devanagari numbers instead of Arabic numerals.<br>\n<strong>ex) answer_text: 1945, prediction: १९४५</strong><br>\nMy postprocessing function is below.</p>\n<pre><code>def postprocess_v3(pred, context, language):\n    if language == \"hindi\":\n        bad_starts = [\".\", \",\", \"(\", \")\", \"-\", \"–\", \";\", \"।\"]\n        bad_endings = [\"...\", \",\", \"(\", \")\", \"-\", \"–\", \";\", \"।\"]\n    elif language == \"tamil\":\n        bad_starts = [\".\", \",\", \"(\", \")\", \"-\", \"–\", \";\"]\n        bad_endings = [\"...\", \",\", \"(\", \")\", \"-\", \"–\", \";\", \".\"]\n\n    tamil_ad = \"கி.பி\"\n    tamil_bc = \"கி.மு\"\n    tamil_km = \"கி.மீ\"\n    hindi_ad = \"ई\"\n    hindi_bc = \"ई.पू\"\n\n    pred = pred.lstrip().rstrip()\n\n    if language == \"hindi\":\n        numerical_table = str.maketrans({\n            '०': '0',\n            '१': '1',\n            '२': '2',\n            '३': '3',\n            '४': '4',\n            '५': '5',\n            '६': '6',\n            '७': '7',\n            '८': '8',\n            '९': '9'\n        })\n        translated_pred = pred.translate(numerical_table)\n        if translated_pred in context and len(pred.split()) == 1:\n            pred = translated_pred\n\n    if pred == \"\":\n        return pred\n    while any([pred.startswith(y) for y in bad_starts]):\n        pred = pred[1:]\n    while any([pred.endswith(y) for y in bad_endings]):\n        if pred.endswith(\"...\"):\n            pred = pred[:-3]\n        else:\n            pred = pred[:-1]\n    if pred.endswith(\"...\"):\n            pred = pred[:-3]\n\n    if any([pred.endswith(tamil_ad), pred.endswith(tamil_bc), pred.endswith(tamil_km), pred.endswith(hindi_ad), pred.endswith(hindi_bc)]) and pred+\".\" in context:\n        pred = pred+\".\"\n\n    if pred.find(\"(\") != -1:\n        _pred = pred[:pred.find(\"(\")]\n        if _pred != \"\":\n            pred = _pred\n\n    return pred\n</code></pre>\n<p>Applying postprocessing, it scored 0.791 in public LB (+0.009 boost) and 0.761 in private LB (+0.007 boost).</p>\n<p>I selected three models ensemble, it scored 0.792 in public LB but 0.760 in private LB…</p>",
  "comments": [
    {
      "id": 1586355,
      "author_name": "Ayush Singh",
      "author_url": "",
      "post_date": "2021-11-18T02:26:45.003000",
      "content": "<p>congratulations <a href=\"https://www.kaggle.com/hiromoon166\" target=\"_blank\">@hiromoon166</a> really nice work. </p>",
      "votes": 1,
      "replies": []
    },
    {
      "id": 1584846,
      "author_name": "Trushant Kalyanpur",
      "author_url": "",
      "post_date": "2021-11-16T20:03:04.943000",
      "content": "<p>Congratulations <a href=\"https://www.kaggle.com/hiromoon166\" target=\"_blank\">@hiromoon166</a> on the significant jump! I tried similar date processing but for some reason it didnt work out too well for me. </p>",
      "votes": 1,
      "replies": []
    },
    {
      "id": 1590701,
      "author_name": "MINGOOK KIM",
      "author_url": "",
      "post_date": "2021-11-21T15:10:12.107000",
      "content": "<p>Congratulations on your achievements.</p>\n<p>Congratulations~</p>",
      "votes": 0,
      "replies": []
    }
  ],
  "comment_count": 3,
  "messages": [
    {
      "id": 1584410,
      "postDate": "2021-11-16T14:08:49.843Z",
      "content": "<p>First of all, I'd like to thank Kaggle and the competition hosts. This competition was really interesting because the training data is really small and we were encouraged to use external data.</p>\n<p>My solution is really simple. There are only two key points - Character Level Ensemble &amp; Postprocessing.</p>\n<h2>Machine Resources</h2>\n<p>Colab pro+ &amp; kaggle notebook</p>\n<h2>Experiment tracking</h2>\n<p>W&amp;B(<a href=\"https://wandb.ai/site\" target=\"_blank\">https://wandb.ai/site</a>)</p>\n<h2>Model</h2>\n<p>I trained three models.</p>\n<ul>\n<li>XLM-RoBERTa</li>\n</ul>\n<pre><code>model: deepset/xlm-roberta-large-squad2\ndata: competition dataset + MLQA + XQUAD\nfold: 5-fold\nmax_length: 384\nstride: 128\nepochs: 2\nlr: 2.0e-5\n</code></pre>\n<ul>\n<li>MuRIL</li>\n</ul>\n<pre><code>model: google/muril-large-cased\ndata: competition dataset + MLQA + XQUAD\nfold: 5-fold\nmax_length: 384\nstride: 128\nepochs: 2\nlr: 2.0e-5\n</code></pre>\n<ul>\n<li>RoBERTa  (WECHSEL)<br>\nThank you for sharing! <a href=\"https://www.kaggle.com/bminixhofer\" target=\"_blank\">@bminixhofer</a></li>\n</ul>\n<pre><code>hindi model: roberta-large-wechsel-hindi\ntamil model: roberta-large-wechsel-tamil\nhindi data: competition dataset + MLQA + XQUAD\ntamil data: competition dataset + translated SQuAD2.0\nfold: 5-fold\nmax_length: 400\nstride: 135\nepochs: 3\nlr: 2.0e-5\n</code></pre>\n<h4>result</h4>\n<table>\n<thead>\n<tr>\n<th>model</th>\n<th>CV</th>\n<th>Public LB</th>\n<th>Private LB</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>XLM-RoBERTa</td>\n<td>0.7060</td>\n<td>0.778</td>\n<td>0.738</td>\n</tr>\n<tr>\n<td>MuRIL</td>\n<td>0.7038</td>\n<td>0.760</td>\n<td>0.743</td>\n</tr>\n<tr>\n<td>RoBERTa</td>\n<td>0.7481 &amp; 0.5907</td>\n<td>0.763</td>\n<td>0.735</td>\n</tr>\n</tbody>\n</table>\n<p>MuRIL is strong💪</p>\n<h2>Character Level Ensemble</h2>\n<p>It's impossible to simply average above three models' output because their tokenizers are different. So I expanded token probability to character probability using offsets.<br>\nAt first, I tried ensembling XLM-RoBERTa &amp; MuRIL. It scored 0.782 in public LB (+0.004 boost), but 0.754 in private LB (+0.011 boost)!!</p>\n<h2>Postprocessing</h2>\n<p>On the last weekend, I checked how the models were getting the answers wrong.<br>\nI found some predictions are consist of Devanagari numbers instead of Arabic numerals.<br>\n<strong>ex) answer_text: 1945, prediction: १९४५</strong><br>\nMy postprocessing function is below.</p>\n<pre><code>def postprocess_v3(pred, context, language):\n    if language == \"hindi\":\n        bad_starts = [\".\", \",\", \"(\", \")\", \"-\", \"–\", \";\", \"।\"]\n        bad_endings = [\"...\", \",\", \"(\", \")\", \"-\", \"–\", \";\", \"।\"]\n    elif language == \"tamil\":\n        bad_starts = [\".\", \",\", \"(\", \")\", \"-\", \"–\", \";\"]\n        bad_endings = [\"...\", \",\", \"(\", \")\", \"-\", \"–\", \";\", \".\"]\n\n    tamil_ad = \"கி.பி\"\n    tamil_bc = \"கி.மு\"\n    tamil_km = \"கி.மீ\"\n    hindi_ad = \"ई\"\n    hindi_bc = \"ई.पू\"\n\n    pred = pred.lstrip().rstrip()\n\n    if language == \"hindi\":\n        numerical_table = str.maketrans({\n            '०': '0',\n            '१': '1',\n            '२': '2',\n            '३': '3',\n            '४': '4',\n            '५': '5',\n            '६': '6',\n            '७': '7',\n            '८': '8',\n            '९': '9'\n        })\n        translated_pred = pred.translate(numerical_table)\n        if translated_pred in context and len(pred.split()) == 1:\n            pred = translated_pred\n\n    if pred == \"\":\n        return pred\n    while any([pred.startswith(y) for y in bad_starts]):\n        pred = pred[1:]\n    while any([pred.endswith(y) for y in bad_endings]):\n        if pred.endswith(\"...\"):\n            pred = pred[:-3]\n        else:\n            pred = pred[:-1]\n    if pred.endswith(\"...\"):\n            pred = pred[:-3]\n\n    if any([pred.endswith(tamil_ad), pred.endswith(tamil_bc), pred.endswith(tamil_km), pred.endswith(hindi_ad), pred.endswith(hindi_bc)]) and pred+\".\" in context:\n        pred = pred+\".\"\n\n    if pred.find(\"(\") != -1:\n        _pred = pred[:pred.find(\"(\")]\n        if _pred != \"\":\n            pred = _pred\n\n    return pred\n</code></pre>\n<p>Applying postprocessing, it scored 0.791 in public LB (+0.009 boost) and 0.761 in private LB (+0.007 boost).</p>\n<p>I selected three models ensemble, it scored 0.792 in public LB but 0.760 in private LB…</p>",
      "rawMarkdown": "First of all, I'd like to thank Kaggle and the competition hosts. This competition was really interesting because the training data is really small and we were encouraged to use external data.\n\nMy solution is really simple. There are only two key points - Character Level Ensemble & Postprocessing.\n\n## Machine Resources\nColab pro+ & kaggle notebook\n\n## Experiment tracking \nW&B(https://wandb.ai/site)\n\n## Model \nI trained three models.\n- XLM-RoBERTa\n```\nmodel: deepset/xlm-roberta-large-squad2\ndata: competition dataset + MLQA + XQUAD\nfold: 5-fold\nmax_length: 384\nstride: 128\nepochs: 2\nlr: 2.0e-5\n```\n- MuRIL\n```\nmodel: google/muril-large-cased\ndata: competition dataset + MLQA + XQUAD\nfold: 5-fold\nmax_length: 384\nstride: 128\nepochs: 2\nlr: 2.0e-5\n```\n- RoBERTa  (WECHSEL)\nThank you for sharing! @bminixhofer\n```\nhindi model: roberta-large-wechsel-hindi\ntamil model: roberta-large-wechsel-tamil\nhindi data: competition dataset + MLQA + XQUAD\ntamil data: competition dataset + translated SQuAD2.0\nfold: 5-fold\nmax_length: 400\nstride: 135\nepochs: 3\nlr: 2.0e-5\n```\n#### result\n| model       | CV              | Public LB | Private LB | \n| ----------- | --------------- | --------- | ---------- | \n| XLM-RoBERTa | 0.7060          | 0.778     | 0.738      | \n| MuRIL       | 0.7038          | 0.760     | 0.743      | \n| RoBERTa     | 0.7481 & 0.5907 | 0.763     | 0.735      |  \n\nMuRIL is strong💪\n\n## Character Level Ensemble\nIt's impossible to simply average above three models' output because their tokenizers are different. So I expanded token probability to character probability using offsets.\nAt first, I tried ensembling XLM-RoBERTa & MuRIL. It scored 0.782 in public LB (+0.004 boost), but 0.754 in private LB (+0.011 boost)!!\n\n## Postprocessing\nOn the last weekend, I checked how the models were getting the answers wrong.\nI found some predictions are consist of Devanagari numbers instead of Arabic numerals.\n**ex) answer_text: 1945, prediction: १९४५**\nMy postprocessing function is below.\n```\ndef postprocess_v3(pred, context, language):\n    if language == \"hindi\":\n        bad_starts = [\".\", \",\", \"(\", \")\", \"-\", \"–\", \";\", \"।\"]\n        bad_endings = [\"...\", \",\", \"(\", \")\", \"-\", \"–\", \";\", \"।\"]\n    elif language == \"tamil\":\n        bad_starts = [\".\", \",\", \"(\", \")\", \"-\", \"–\", \";\"]\n        bad_endings = [\"...\", \",\", \"(\", \")\", \"-\", \"–\", \";\", \".\"]\n\n    tamil_ad = \"கி.பி\"\n    tamil_bc = \"கி.மு\"\n    tamil_km = \"கி.மீ\"\n    hindi_ad = \"ई\"\n    hindi_bc = \"ई.पू\"\n    \n    pred = pred.lstrip().rstrip()\n    \n    if language == \"hindi\":\n        numerical_table = str.maketrans({\n            '०': '0',\n            '१': '1',\n            '२': '2',\n            '३': '3',\n            '४': '4',\n            '५': '5',\n            '६': '6',\n            '७': '7',\n            '८': '8',\n            '९': '9'\n        })\n        translated_pred = pred.translate(numerical_table)\n        if translated_pred in context and len(pred.split()) == 1:\n            pred = translated_pred\n\n    if pred == \"\":\n        return pred\n    while any([pred.startswith(y) for y in bad_starts]):\n        pred = pred[1:]\n    while any([pred.endswith(y) for y in bad_endings]):\n        if pred.endswith(\"...\"):\n            pred = pred[:-3]\n        else:\n            pred = pred[:-1]\n    if pred.endswith(\"...\"):\n            pred = pred[:-3]\n\n    if any([pred.endswith(tamil_ad), pred.endswith(tamil_bc), pred.endswith(tamil_km), pred.endswith(hindi_ad), pred.endswith(hindi_bc)]) and pred+\".\" in context:\n        pred = pred+\".\"\n    \n    if pred.find(\"(\") != -1:\n        _pred = pred[:pred.find(\"(\")]\n        if _pred != \"\":\n            pred = _pred\n\n    return pred\n```\nApplying postprocessing, it scored 0.791 in public LB (+0.009 boost) and 0.761 in private LB (+0.007 boost).\n\nI selected three models ensemble, it scored 0.792 in public LB but 0.760 in private LB...",
      "votes": 21
    },
    {
      "id": 1586355,
      "postDate": "2021-11-18T02:26:45.003Z",
      "content": "<p>congratulations <a href=\"https://www.kaggle.com/hiromoon166\" target=\"_blank\">@hiromoon166</a> really nice work. </p>",
      "rawMarkdown": "congratulations @hiromoon166 really nice work. ",
      "votes": 1
    },
    {
      "id": 1584846,
      "postDate": "2021-11-16T20:03:04.943Z",
      "content": "<p>Congratulations <a href=\"https://www.kaggle.com/hiromoon166\" target=\"_blank\">@hiromoon166</a> on the significant jump! I tried similar date processing but for some reason it didnt work out too well for me. </p>",
      "rawMarkdown": "Congratulations @hiromoon166 on the significant jump! I tried similar date processing but for some reason it didnt work out too well for me. ",
      "votes": 1
    },
    {
      "id": 1590701,
      "postDate": "2021-11-21T15:10:12.107Z",
      "content": "<p>Congratulations on your achievements.</p>\n<p>Congratulations~</p>",
      "rawMarkdown": "Congratulations on your achievements.\n\nCongratulations~"
    }
  ],
  "complete": true,
  "source": "live",
  "fetched_at": "2026-10-19T16:20:44Z",
  "competition_id": 30060,
  "competition_slug": "chaii-hindi-and-tamil-question-answering",
  "team_id": 7252189,
  "solution_url": "/competitions/chaii-hindi-and-tamil-question-answering/writeups/hiromu-12th-place-solution-top-of-the-silver"
}