{
  "id": 666798,
  "title": "Is test set's transliteration formatted the same as the train set",
  "url": "/competitions/deep-past-initiative-machine-translation/discussion/666798",
  "author_name": "",
  "post_date": "2026-01-09T07:12:25.334465700Z",
  "votes": 4,
  "comment_count": 6,
  "views": 0,
  "content": "<p>For the given test set, I saw the transliterations are like</p>\n<pre><code>['um-ma kà-ru-um kà-ni-ia-ma a-na aa-qí-il… da-tim aí-ip-ri-ni kà-ar kà-ar-ma ú wa-bar-ra-tim qí-bi„-ma mup-pu-um aa a-lim(ki) i-li-kam',\n       'i-na mup-pì-im aa a-lim(ki) ia-tù u„-mì-im a-nim ma-ma-an KÙ.AN i-aa-ú-mu-ni i-na né-mì-lim da-aùr ú-lá e-WA ia-ra-tí-au kà-ru-um kà-ni-ia i-lá-qé',\n       'ki-ma mup-pì-ni ta-áa-me-a-ni a-ma-kam lu a-na aí-mì-im a-na É.GAL-lim i-dí-in lu té-ra-at É.GAL-lim ú-kà-lim lu na-aí-ma a-dí-ni lá i-dí-in ma-lá KÙ.AN na-áa-ú ni-bi„-it a-aí-im au-um-au ú au-mì a-bi„-au i-na mup-pì-im lu-up-ta-nim-ma ia-tí aí-ip-ri-ni aé-bi„-lá-nim']\n</code></pre>\n<p>But in the train set it looks like</p>\n<pre><code> 'KIŠIB en-um-a-šur DUMU IŠTAR-lá-ma-sí KIŠIB in-ba-a-šur DUMU pì-lá-aḫ-sú-in KIŠIB PUZUR₄.IŠTAR DUMU DINGIR-na-da 3 GÍN KÙ.BABBAR i-ṣé-er PUZUR₄.IŠTAR DUMU DINGIR-na-da DAM.GÀR i-šu iš-tù ḫa-mu-uš-tim ša šu-ku-bi-im ú ka-zi-a 22.5 ŠE i-na ITU.KAM ṣí-ib-tám ú-ṣa-áb ITU.KAM ma-ḫu-ur-DINGIR li-mu-um ša qá-té (d)IŠKUR-ba-ni',\n</code></pre>\n<p>Some symbols like <code>„</code>, <code>ḫ</code>, <code>R₄</code> does not seem be be shareable, wonder is the hidden test set formatted differently from the train set?</p>",
  "messages": [
    {
      "id": "3388590",
      "postDate": "01/09/2026 07:12:25",
      "content": "<p>For the given test set, I saw the transliterations are like</p>\n<pre><code>['um-ma kà-ru-um kà-ni-ia-ma a-na aa-qí-il… da-tim aí-ip-ri-ni kà-ar kà-ar-ma ú wa-bar-ra-tim qí-bi„-ma mup-pu-um aa a-lim(ki) i-li-kam',\n       'i-na mup-pì-im aa a-lim(ki) ia-tù u„-mì-im a-nim ma-ma-an KÙ.AN i-aa-ú-mu-ni i-na né-mì-lim da-aùr ú-lá e-WA ia-ra-tí-au kà-ru-um kà-ni-ia i-lá-qé',\n       'ki-ma mup-pì-ni ta-áa-me-a-ni a-ma-kam lu a-na aí-mì-im a-na É.GAL-lim i-dí-in lu té-ra-at É.GAL-lim ú-kà-lim lu na-aí-ma a-dí-ni lá i-dí-in ma-lá KÙ.AN na-áa-ú ni-bi„-it a-aí-im au-um-au ú au-mì a-bi„-au i-na mup-pì-im lu-up-ta-nim-ma ia-tí aí-ip-ri-ni aé-bi„-lá-nim']\n</code></pre>\n<p>But in the train set it looks like</p>\n<pre><code> 'KIŠIB en-um-a-šur DUMU IŠTAR-lá-ma-sí KIŠIB in-ba-a-šur DUMU pì-lá-aḫ-sú-in KIŠIB PUZUR₄.IŠTAR DUMU DINGIR-na-da 3 GÍN KÙ.BABBAR i-ṣé-er PUZUR₄.IŠTAR DUMU DINGIR-na-da DAM.GÀR i-šu iš-tù ḫa-mu-uš-tim ša šu-ku-bi-im ú ka-zi-a 22.5 ŠE i-na ITU.KAM ṣí-ib-tám ú-ṣa-áb ITU.KAM ma-ḫu-ur-DINGIR li-mu-um ša qá-té (d)IŠKUR-ba-ni',\n</code></pre>\n<p>Some symbols like <code>„</code>, <code>ḫ</code>, <code>R₄</code> does not seem be be shareable, wonder is the hidden test set formatted differently from the train set?</p>",
      "rawMarkdown": "For the given test set, I saw the transliterations are like\n```\n['um-ma kà-ru-um kà-ni-ia-ma a-na aa-qí-il… da-tim aí-ip-ri-ni kà-ar kà-ar-ma ú wa-bar-ra-tim qí-bi„-ma mup-pu-um aa a-lim(ki) i-li-kam',\n       'i-na mup-pì-im aa a-lim(ki) ia-tù u„-mì-im a-nim ma-ma-an KÙ.AN i-aa-ú-mu-ni i-na né-mì-lim da-aùr ú-lá e-WA ia-ra-tí-au kà-ru-um kà-ni-ia i-lá-qé',\n       'ki-ma mup-pì-ni ta-áa-me-a-ni a-ma-kam lu a-na aí-mì-im a-na É.GAL-lim i-dí-in lu té-ra-at É.GAL-lim ú-kà-lim lu na-aí-ma a-dí-ni lá i-dí-in ma-lá KÙ.AN na-áa-ú ni-bi„-it a-aí-im au-um-au ú au-mì a-bi„-au i-na mup-pì-im lu-up-ta-nim-ma ia-tí aí-ip-ri-ni aé-bi„-lá-nim']\n```\n\nBut in the train set it looks like\n```\n 'KIŠIB en-um-a-šur DUMU IŠTAR-lá-ma-sí KIŠIB in-ba-a-šur DUMU pì-lá-aḫ-sú-in KIŠIB PUZUR₄.IŠTAR DUMU DINGIR-na-da 3 GÍN KÙ.BABBAR i-ṣé-er PUZUR₄.IŠTAR DUMU DINGIR-na-da DAM.GÀR i-šu iš-tù ḫa-mu-uš-tim ša šu-ku-bi-im ú ka-zi-a 22.5 ŠE i-na ITU.KAM ṣí-ib-tám ú-ṣa-áb ITU.KAM ma-ḫu-ur-DINGIR li-mu-um ša qá-té (d)IŠKUR-ba-ni',\n``` \n\nSome symbols like `„`, `ḫ`, `R₄` does not seem be be shareable, wonder is the hidden test set formatted differently from the train set?",
      "votes": null
    },
    {
      "id": "3388596",
      "postDate": "01/09/2026 07:38:29",
      "content": "<p>Your observation is correct. However, there seems to be a frequent conversion of <code>š to a</code>.</p>\n<p>Although this differs from the training data, I found that my model can predict their meanings accurately, even when individual characters are altered.</p>",
      "rawMarkdown": "Your observation is correct. However, there seems to be a frequent conversion of `š to a`.\n\nAlthough this differs from the training data, I found that my model can predict their meanings accurately, even when individual characters are altered.",
      "votes": null
    },
    {
      "id": "3388752",
      "postDate": "01/09/2026 14:32:34",
      "content": "<p>The example test data was actually sampled from the published texts, not the actual test data (which we are unable to share). The example data is more meant to illustrate sentence alignment and line annotations. I wouldn't draw conclusions about the actual Akkadian text in the test set from this example data.</p>",
      "rawMarkdown": "The example test data was actually sampled from the published texts, not the actual test data (which we are unable to share). The example data is more meant to illustrate sentence alignment and line annotations. I wouldn't draw conclusions about the actual Akkadian text in the test set from this example data.",
      "votes": null
    },
    {
      "id": "3388791",
      "postDate": "01/09/2026 15:44:26",
      "content": "<p>Is there a use for the <code>line_start</code> and <code>line_end</code> in values in the test set? My understanding is that we only need to translate the transliteration column. However, I am also wondering if the line numbers are useful when processing the input file for submission.</p>\n<p>And please I would be glad to have your opinion on this <a href=\"https://www.kaggle.com/competitions/deep-past-initiative-machine-translation/discussion/666443\" target=\"_blank\">thread</a> regarding the use of LLMs to process the competition data especially for translating non-English translations (German, etc) of the Akkadian texts in the publications.csv to English.</p>",
      "rawMarkdown": "Is there a use for the `line_start` and `line_end` in values in the test set? My understanding is that we only need to translate the transliteration column. However, I am also wondering if the line numbers are useful when processing the input file for submission.\n\nAnd please I would be glad to have your opinion on this [thread](https://www.kaggle.com/competitions/deep-past-initiative-machine-translation/discussion/666443) regarding the use of LLMs to process the competition data especially for translating non-English translations (German, etc) of the Akkadian texts in the publications.csv to English.",
      "votes": null
    },
    {
      "id": "3388840",
      "postDate": "01/09/2026 17:16:24",
      "content": "<p><a href=\"https://www.kaggle.com/anaphase21\" target=\"_blank\">@anaphase21</a> The line numbers aren't strictly required for generating the input file for submission. They might be useful in other ways though.</p>\n<p>I think you found my comment about the competition rules. The official competition rules are the place to start for questions like these.</p>",
      "rawMarkdown": "anaphase21 The line numbers aren't strictly required for generating the input file for submission. They might be useful in other ways though.\n\nI think you found my comment about the competition rules. The official competition rules are the place to start for questions like these.",
      "votes": null
    },
    {
      "id": "3390689",
      "postDate": "01/13/2026 18:25:51",
      "content": "<p>That must have been a formatting error, but I can assure you this is not the case for the actual test set. For a complete list of characters used in transliteration and translation, see: <a href=\"https://www.kaggle.com/competitions/deep-past-initiative-machine-translation/discussion/665209#3384200\" target=\"_blank\">https://www.kaggle.com/competitions/deep-past-initiative-machine-translation/discussion/665209#3384200</a></p>",
      "rawMarkdown": "That must have been a formatting error, but I can assure you this is not the case for the actual test set. For a complete list of characters used in transliteration and translation, see: https://www.kaggle.com/competitions/deep-past-initiative-machine-translation/discussion/665209#3384200",
      "votes": null
    },
    {
      "id": "3390693",
      "postDate": "01/13/2026 18:33:40",
      "content": "<p>Thank you for your speech. Yes, if that were really the case, the outcome would be terrible.😃</p>",
      "rawMarkdown": "Thank you for your speech. Yes, if that were really the case, the outcome would be terrible.😃",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 3388596,
      "author_name": "qifeihhh666",
      "author_url": "",
      "post_date": "01/09/2026 07:38:29",
      "content": "<p>Your observation is correct. However, there seems to be a frequent conversion of <code>š to a</code>.</p>\n<p>Although this differs from the training data, I found that my model can predict their meanings accurately, even when individual characters are altered.</p>",
      "votes": null,
      "replies": [
        {
          "id": 3390689,
          "author_name": "deeppast",
          "author_url": "",
          "post_date": "01/13/2026 18:25:51",
          "content": "<p>That must have been a formatting error, but I can assure you this is not the case for the actual test set. For a complete list of characters used in transliteration and translation, see: <a href=\"https://www.kaggle.com/competitions/deep-past-initiative-machine-translation/discussion/665209#3384200\" target=\"_blank\">https://www.kaggle.com/competitions/deep-past-initiative-machine-translation/discussion/665209#3384200</a></p>",
          "votes": null,
          "replies": [
            {
              "id": 3390693,
              "author_name": "qifeihhh666",
              "author_url": "",
              "post_date": "01/13/2026 18:33:40",
              "content": "<p>Thank you for your speech. Yes, if that were really the case, the outcome would be terrible.😃</p>",
              "votes": null,
              "replies": []
            }
          ]
        }
      ]
    },
    {
      "id": 3388752,
      "author_name": "ryanholbrook",
      "author_url": "",
      "post_date": "01/09/2026 14:32:34",
      "content": "<p>The example test data was actually sampled from the published texts, not the actual test data (which we are unable to share). The example data is more meant to illustrate sentence alignment and line annotations. I wouldn't draw conclusions about the actual Akkadian text in the test set from this example data.</p>",
      "votes": null,
      "replies": [
        {
          "id": 3388791,
          "author_name": "anaphase21",
          "author_url": "",
          "post_date": "01/09/2026 15:44:26",
          "content": "<p>Is there a use for the <code>line_start</code> and <code>line_end</code> in values in the test set? My understanding is that we only need to translate the transliteration column. However, I am also wondering if the line numbers are useful when processing the input file for submission.</p>\n<p>And please I would be glad to have your opinion on this <a href=\"https://www.kaggle.com/competitions/deep-past-initiative-machine-translation/discussion/666443\" target=\"_blank\">thread</a> regarding the use of LLMs to process the competition data especially for translating non-English translations (German, etc) of the Akkadian texts in the publications.csv to English.</p>",
          "votes": null,
          "replies": [
            {
              "id": 3388840,
              "author_name": "ryanholbrook",
              "author_url": "",
              "post_date": "01/09/2026 17:16:24",
              "content": "<p><a href=\"https://www.kaggle.com/anaphase21\" target=\"_blank\">@anaphase21</a> The line numbers aren't strictly required for generating the input file for submission. They might be useful in other ways though.</p>\n<p>I think you found my comment about the competition rules. The official competition rules are the place to start for questions like these.</p>",
              "votes": null,
              "replies": []
            }
          ]
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "3388590": "For the given test set, I saw the transliterations are like\n```\n['um-ma kà-ru-um kà-ni-ia-ma a-na aa-qí-il… da-tim aí-ip-ri-ni kà-ar kà-ar-ma ú wa-bar-ra-tim qí-bi„-ma mup-pu-um aa a-lim(ki) i-li-kam',\n       'i-na mup-pì-im aa a-lim(ki) ia-tù u„-mì-im a-nim ma-ma-an KÙ.AN i-aa-ú-mu-ni i-na né-mì-lim da-aùr ú-lá e-WA ia-ra-tí-au kà-ru-um kà-ni-ia i-lá-qé',\n       'ki-ma mup-pì-ni ta-áa-me-a-ni a-ma-kam lu a-na aí-mì-im a-na É.GAL-lim i-dí-in lu té-ra-at É.GAL-lim ú-kà-lim lu na-aí-ma a-dí-ni lá i-dí-in ma-lá KÙ.AN na-áa-ú ni-bi„-it a-aí-im au-um-au ú au-mì a-bi„-au i-na mup-pì-im lu-up-ta-nim-ma ia-tí aí-ip-ri-ni aé-bi„-lá-nim']\n```\n\nBut in the train set it looks like\n```\n 'KIŠIB en-um-a-šur DUMU IŠTAR-lá-ma-sí KIŠIB in-ba-a-šur DUMU pì-lá-aḫ-sú-in KIŠIB PUZUR₄.IŠTAR DUMU DINGIR-na-da 3 GÍN KÙ.BABBAR i-ṣé-er PUZUR₄.IŠTAR DUMU DINGIR-na-da DAM.GÀR i-šu iš-tù ḫa-mu-uš-tim ša šu-ku-bi-im ú ka-zi-a 22.5 ŠE i-na ITU.KAM ṣí-ib-tám ú-ṣa-áb ITU.KAM ma-ḫu-ur-DINGIR li-mu-um ša qá-té (d)IŠKUR-ba-ni',\n``` \n\nSome symbols like `„`, `ḫ`, `R₄` does not seem be be shareable, wonder is the hidden test set formatted differently from the train set?",
    "3388596": "Your observation is correct. However, there seems to be a frequent conversion of `š to a`.\n\nAlthough this differs from the training data, I found that my model can predict their meanings accurately, even when individual characters are altered.",
    "3388752": "The example test data was actually sampled from the published texts, not the actual test data (which we are unable to share). The example data is more meant to illustrate sentence alignment and line annotations. I wouldn't draw conclusions about the actual Akkadian text in the test set from this example data.",
    "3388791": "Is there a use for the `line_start` and `line_end` in values in the test set? My understanding is that we only need to translate the transliteration column. However, I am also wondering if the line numbers are useful when processing the input file for submission.\n\nAnd please I would be glad to have your opinion on this [thread](https://www.kaggle.com/competitions/deep-past-initiative-machine-translation/discussion/666443) regarding the use of LLMs to process the competition data especially for translating non-English translations (German, etc) of the Akkadian texts in the publications.csv to English.",
    "3388840": "anaphase21 The line numbers aren't strictly required for generating the input file for submission. They might be useful in other ways though.\n\nI think you found my comment about the competition rules. The official competition rules are the place to start for questions like these.",
    "3390689": "That must have been a formatting error, but I can assure you this is not the case for the actual test set. For a complete list of characters used in transliteration and translation, see: https://www.kaggle.com/competitions/deep-past-initiative-machine-translation/discussion/665209#3384200",
    "3390693": "Thank you for your speech. Yes, if that were really the case, the outcome would be terrible.😃"
  },
  "source": "meta"
}