{ "id": 668766, "title": "Ground truth examples needed", "url": "/competitions/deep-past-initiative-machine-translation/discussion/668766", "author_name": "", "post_date": "2026-01-18T20:37:26.606657600Z", "votes": -1, "comment_count": 8, "views": 0, "content": "

@deeppast I believe we really need a handful of examples (two or three with gaps etc would suffice) to understand what is the formatting of the transliterations and translations in the hidden test set. These could be made up, they don't have to be exactly from the hidden test set since it is meant to be instructive only.\nIt is really confusing at the moment as to what role all this formatting suggestions have to play. The train.csv is clearly not the ground truth (right?). The samples there are not sentence-level aligned and do not have the formatting suggestions (like <gap>, <big_gap> or have the modern scribal notations removed) like how you are asking us to output.

\n

I think it will put a lot of questions to rest if one could see this examples, ideally with words having subscripts / superscripts etc.

", "messages": [ { "id": "3393343", "postDate": "01/18/2026 20:37:26", "content": "

@deeppast I believe we really need a handful of examples (two or three with gaps etc would suffice) to understand what is the formatting of the transliterations and translations in the hidden test set. These could be made up, they don't have to be exactly from the hidden test set since it is meant to be instructive only.\nIt is really confusing at the moment as to what role all this formatting suggestions have to play. The train.csv is clearly not the ground truth (right?). The samples there are not sentence-level aligned and do not have the formatting suggestions (like <gap>, <big_gap> or have the modern scribal notations removed) like how you are asking us to output.

\n

I think it will put a lot of questions to rest if one could see this examples, ideally with words having subscripts / superscripts etc.

", "rawMarkdown": "deeppast I believe we really need a handful of examples (two or three with gaps etc would suffice) to understand what is the formatting of the transliterations and translations in the hidden test set. These could be made up, they don't have to be exactly from the hidden test set since it is meant to be instructive only.\nIt is really confusing at the moment as to what role all this formatting suggestions have to play. The `train.csv` is clearly not the ground truth (right?). The samples there are not sentence-level aligned and do not have the formatting suggestions (like \\, \\ or have the modern scribal notations removed) like how you are asking us to output. \n\nI think it will put a lot of questions to rest if one could see this examples, ideally with words having subscripts / superscripts etc.", "votes": null }, { "id": "3393357", "postDate": "01/18/2026 21:23:48", "content": "

At the very least the test.csv file should be corrected to account for incorrect syllables, e.g., aa

\n
\n

um-ma kà-ru-um kà-ni-ia-ma a-na aa-qí-il… da-tim aí-ip-ri-ni kà-ar

\n
", "rawMarkdown": "At the very least the `test.csv` file should be corrected to account for incorrect syllables, e.g., `aa`\n>um-ma kà-ru-um kà-ni-ia-ma a-na aa-qí-il… da-tim aí-ip-ri-ni kà-ar", "votes": null }, { "id": "3393358", "postDate": "01/18/2026 21:26:49", "content": "

But yes, a ground truth with 4 examples treating <gap>, <big_gap>, numerical subscripts, determinatives, special case of subscripted x would shed light on what should be expected.

", "rawMarkdown": "But yes, a ground truth with 4 examples treating ``, ``, numerical subscripts, determinatives, special case of subscripted `x` would shed light on what should be expected.", "votes": null }, { "id": "3393368", "postDate": "01/18/2026 22:30:08", "content": "

that's not in the actual text.csv, just so you know… it's a formatting error that appears to be from the š character to me

", "rawMarkdown": "that's not in the actual text.csv, just so you know... it's a formatting error that appears to be from the š character to me", "votes": null }, { "id": "3393483", "postDate": "01/19/2026 05:31:15", "content": "

Yes I read that but how about a handful of carefully curated instructive examples so we know what is actually expected?

", "rawMarkdown": "Yes I read that but how about a handful of carefully curated instructive examples so we know what is actually expected?", "votes": null }, { "id": "3393524", "postDate": "01/19/2026 08:29:29", "content": "
\n

that's not in the actual text.csv, just so you know… it's a formatting error that appears to be from the š character to me

\n
\n

It does to me too, but I follow discussion here regularly. The error however persists in the actual test.csv file. When opened through pandas.read_csv it's still there.\n\"\"

\n

The problem is that the test should be a reference and at the moment it cannot be, not without a list of changes, like the one with š.

\n

We have 4+ different standards at the moment:

\n
    \n
  1. train.csv
    2. \n
  2. Sentences_Oare_FirstWord_LinNum.csv
    3. \n
  3. test.csv
    4. \n
  4. possible several standards in publications.csv (I haven't verified it thoroughly)
    5. \n
\n

From the discussions I understand that Sentences_Oare_FirstWord_LinNum.csv is the closest to following the rules specificed for handling special characters, gaps and subscripts. It should be the test.csv, but at the moment without going through discussions it cannot be used as is. If possible, please repair the original file.

", "rawMarkdown": "> that's not in the actual text.csv, just so you know... it's a formatting error that appears to be from the š character to me\n\nIt does to me too, but I follow discussion here regularly. The error however persists in the actual `test.csv` file. When opened through pandas.read_csv it's still there.\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F2259646%2Fb3d146e3f2dc79e995e61c9c29fbf252%2Ftest.csv-erroneous%20formatting.png?generation=1768810770217508&alt=media)\n\nThe problem is that the test should be a reference and at the moment it cannot be, not without a list of changes, like the one with `š`.\n\nWe have 4+ different standards at the moment:\n1. `train.csv`\n2. `Sentences_Oare_FirstWord_LinNum.csv`\n3. `test.csv`\n4. possible several standards in `publications.csv` (I haven't verified it thoroughly)\n\nFrom the discussions I understand that `Sentences_Oare_FirstWord_LinNum.csv` is the closest to following the rules specificed for handling special characters, gaps and subscripts. It should be the `test.csv`, but at the moment without going through discussions it cannot be used as is. If possible, please repair the original file.", "votes": null }, { "id": "3393553", "postDate": "01/19/2026 09:55:49", "content": "

We should also add published_texts.csv to that list of different \"standards\".\npublished_texts.csv contains two columns transliteration_orig and transliteration. Which of those is closer to the real test set?!

", "rawMarkdown": "We should also add `published_texts.csv` to that list of different \"standards\".\n`published_texts.csv` contains two columns `transliteration_orig` and `transliteration`. Which of those is closer to the real test set?!", "votes": null }, { "id": "3393556", "postDate": "01/19/2026 10:02:34", "content": "

My quick response would be transliteration, since it already contains <big_gap>/<gap> markers. It contains subscripted numerals, though, e.g., PUZUR₄-a-šùr-ma.

", "rawMarkdown": "My quick response would be `transliteration`, since it already contains `/` markers. It contains subscripted numerals, though, e.g., `PUZUR₄-a-šùr-ma`.", "votes": null }, { "id": "3393610", "postDate": "01/19/2026 12:01:19", "content": "

Honestly I thought that transliteration in published_texts.csv is the format of the test set's transliterationbut:

\n

1) if that's the format of the input, then this whole discussion about <gap> and <big_gap> is needed only for the translation. And if that's true, it really begs the question why the translation column was not formatted the same way in the first place?

\n

2) then we don't need to talk about this (d) and {d} etc. either, since that's taken care of (in the transliteration_orig and transliteration columns) in published_texts.csv already.

", "rawMarkdown": "Honestly I thought that `transliteration` in `published_texts.csv` is the format of the test set's `transliteration`but: \n\n1) if that's the format of the input, then this whole discussion about \\ and \\ is needed only for the translation. And if that's true, it really begs the question why the translation column was not formatted the same way in the first place? \n\n2) then we don't need to talk about this `(d)` and `{d}` etc. either, since that's taken care of (in the `transliteration_orig` and `transliteration` columns) in `published_texts.csv` already.", "votes": null } ], "comments": [ { "id": 3393357, "author_name": "angantyr", "author_url": "", "post_date": "01/18/2026 21:23:48", "content": "

At the very least the test.csv file should be corrected to account for incorrect syllables, e.g., aa

\n
\n

um-ma kà-ru-um kà-ni-ia-ma a-na aa-qí-il… da-tim aí-ip-ri-ni kà-ar

\n
", "votes": null, "replies": [ { "id": 3393368, "author_name": "deeppast", "author_url": "", "post_date": "01/18/2026 22:30:08", "content": "

that's not in the actual text.csv, just so you know… it's a formatting error that appears to be from the š character to me

", "votes": null, "replies": [ { "id": 3393483, "author_name": "einherjer", "author_url": "", "post_date": "01/19/2026 05:31:15", "content": "

Yes I read that but how about a handful of carefully curated instructive examples so we know what is actually expected?

", "votes": null, "replies": [] }, { "id": 3393524, "author_name": "angantyr", "author_url": "", "post_date": "01/19/2026 08:29:29", "content": "
\n

that's not in the actual text.csv, just so you know… it's a formatting error that appears to be from the š character to me

\n
\n

It does to me too, but I follow discussion here regularly. The error however persists in the actual test.csv file. When opened through pandas.read_csv it's still there.\n\"\"

\n

The problem is that the test should be a reference and at the moment it cannot be, not without a list of changes, like the one with š.

\n

We have 4+ different standards at the moment:

\n
    \n
  1. train.csv
    2. \n
  2. Sentences_Oare_FirstWord_LinNum.csv
    3. \n
  3. test.csv
    4. \n
  4. possible several standards in publications.csv (I haven't verified it thoroughly)
    5. \n
\n

From the discussions I understand that Sentences_Oare_FirstWord_LinNum.csv is the closest to following the rules specificed for handling special characters, gaps and subscripts. It should be the test.csv, but at the moment without going through discussions it cannot be used as is. If possible, please repair the original file.

", "votes": null, "replies": [ { "id": 3393553, "author_name": "einherjer", "author_url": "", "post_date": "01/19/2026 09:55:49", "content": "

We should also add published_texts.csv to that list of different \"standards\".\npublished_texts.csv contains two columns transliteration_orig and transliteration. Which of those is closer to the real test set?!

", "votes": null, "replies": [ { "id": 3393556, "author_name": "angantyr", "author_url": "", "post_date": "01/19/2026 10:02:34", "content": "

My quick response would be transliteration, since it already contains <big_gap>/<gap> markers. It contains subscripted numerals, though, e.g., PUZUR₄-a-šùr-ma.

", "votes": null, "replies": [ { "id": 3393610, "author_name": "einherjer", "author_url": "", "post_date": "01/19/2026 12:01:19", "content": "

Honestly I thought that transliteration in published_texts.csv is the format of the test set's transliterationbut:

\n

1) if that's the format of the input, then this whole discussion about <gap> and <big_gap> is needed only for the translation. And if that's true, it really begs the question why the translation column was not formatted the same way in the first place?

\n

2) then we don't need to talk about this (d) and {d} etc. either, since that's taken care of (in the transliteration_orig and transliteration columns) in published_texts.csv already.

", "votes": null, "replies": [] } ] } ] } ] } ] } ] }, { "id": 3393358, "author_name": "angantyr", "author_url": "", "post_date": "01/18/2026 21:26:49", "content": "

But yes, a ground truth with 4 examples treating <gap>, <big_gap>, numerical subscripts, determinatives, special case of subscripted x would shed light on what should be expected.

", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "3393343": "deeppast I believe we really need a handful of examples (two or three with gaps etc would suffice) to understand what is the formatting of the transliterations and translations in the hidden test set. These could be made up, they don't have to be exactly from the hidden test set since it is meant to be instructive only.\nIt is really confusing at the moment as to what role all this formatting suggestions have to play. The `train.csv` is clearly not the ground truth (right?). The samples there are not sentence-level aligned and do not have the formatting suggestions (like \\, \\ or have the modern scribal notations removed) like how you are asking us to output. \n\nI think it will put a lot of questions to rest if one could see this examples, ideally with words having subscripts / superscripts etc.", "3393357": "At the very least the `test.csv` file should be corrected to account for incorrect syllables, e.g., `aa`\n>um-ma kà-ru-um kà-ni-ia-ma a-na aa-qí-il… da-tim aí-ip-ri-ni kà-ar", "3393358": "But yes, a ground truth with 4 examples treating ``, ``, numerical subscripts, determinatives, special case of subscripted `x` would shed light on what should be expected.", "3393368": "that's not in the actual text.csv, just so you know... it's a formatting error that appears to be from the š character to me", "3393483": "Yes I read that but how about a handful of carefully curated instructive examples so we know what is actually expected?", "3393524": "> that's not in the actual text.csv, just so you know... it's a formatting error that appears to be from the š character to me\n\nIt does to me too, but I follow discussion here regularly. The error however persists in the actual `test.csv` file. When opened through pandas.read_csv it's still there.\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F2259646%2Fb3d146e3f2dc79e995e61c9c29fbf252%2Ftest.csv-erroneous%20formatting.png?generation=1768810770217508&alt=media)\n\nThe problem is that the test should be a reference and at the moment it cannot be, not without a list of changes, like the one with `š`.\n\nWe have 4+ different standards at the moment:\n1. `train.csv`\n2. `Sentences_Oare_FirstWord_LinNum.csv`\n3. `test.csv`\n4. possible several standards in `publications.csv` (I haven't verified it thoroughly)\n\nFrom the discussions I understand that `Sentences_Oare_FirstWord_LinNum.csv` is the closest to following the rules specificed for handling special characters, gaps and subscripts. It should be the `test.csv`, but at the moment without going through discussions it cannot be used as is. If possible, please repair the original file.", "3393553": "We should also add `published_texts.csv` to that list of different \"standards\".\n`published_texts.csv` contains two columns `transliteration_orig` and `transliteration`. Which of those is closer to the real test set?!", "3393556": "My quick response would be `transliteration`, since it already contains `/` markers. It contains subscripted numerals, though, e.g., `PUZUR₄-a-šùr-ma`.", "3393610": "Honestly I thought that `transliteration` in `published_texts.csv` is the format of the test set's `transliteration`but: \n\n1) if that's the format of the input, then this whole discussion about \\ and \\ is needed only for the translation. And if that's true, it really begs the question why the translation column was not formatted the same way in the first place? \n\n2) then we don't need to talk about this `(d)` and `{d}` etc. either, since that's taken care of (in the `transliteration_orig` and `transliteration` columns) in `published_texts.csv` already." }, "source": "meta" }