Hi everyone,
\nI just posted an update to the dataset that regularizes <gap> conventions. The train.csv, published.csv, the (hidden) test.csv, and the test set labels have all been updated. The competition host will follow up soon with more details.
I will also be initiating a rescore of all current submissions tomorrow. I will keep you updated as this progresses. All new submissions will be scored against the updated labels.
\nUPDATE 02/20/2026: We are working on another (hopefully final) update that should address some of the issues raised here. I hope to have it out soon, likely Monday. We will delay the rescore until then.
", "messages": [ { "id": 3407672, "postDate": "2026-02-18T21:59:11.130Z", "content": "Hi everyone,
\nI just posted an update to the dataset that regularizes <gap> conventions. The train.csv, published.csv, the (hidden) test.csv, and the test set labels have all been updated. The competition host will follow up soon with more details.
I will also be initiating a rescore of all current submissions tomorrow. I will keep you updated as this progresses. All new submissions will be scored against the updated labels.
\nUPDATE 02/20/2026: We are working on another (hopefully final) update that should address some of the issues raised here. I hope to have it out soon, likely Monday. We will delay the rescore until then.
", "rawMarkdown": "Hi everyone,\n\nI just posted an update to the dataset that regularizes `First, thank you for providing the updated, cleaner dataset - it’s clear that a lot of effort went into improving data quality, and it will ultimately benefit the challenge.
\nI do have a couple of concerns I’d like to share:
\nMid-competition changes: Updating the data and rules partway through the competition can be challenging for participants who have been working since the start of the Deep Past Challenge. Many of us invested significant time normalizing and adapting to the original data, and these efforts may no longer be applicable. While I understand the intent to improve the dataset, this kind of change can feel discouraging from a fairness standpoint.
Information consistency across platforms: Some important clarifications appear to be shared on Discord but not reflected in the official Kaggle discussion or data description. For participants who rely solely on the Kaggle platform, this creates an uneven playing field. It would be very helpful if all critical information were consolidated in the official documentation.
That said, I’m still enjoying the challenge and plan to continue participating.
\nTo help avoid further rework, especially for normalizing re-OCR data such as Larsen, could you please provide a detailed list of the normalization changes that were applied? A quick diff of train.csv suggests that, beyond replacing with , there were also changes such as quote removal, number fractions, Hh, subscripts, and possibly other text adjustments. Having a clear summary would help participants align their preprocessing steps with the updated data. Could you provide the updated list of characters allowed in transliteration and translation?
\nFinally:
\nThe thing is. This is obviously a good change that eliminates 2-token preprocessing gamble and should have been done in the first week. I'm sure Adam is a professional and knows his domain perfectly. Just sometimes people are unfamiliar with how many GPU/human hours could be burnt by solving a problem within a particular setting and that changing this setting could be a bit painful.
\nNevertheless, I'm sure that 'transitivity' of a translation quality still holds (in general) for the new test. Let's just hope that it is the last change~~
", "rawMarkdown": "The thing is. This is obviously a good change that eliminates 2-token preprocessing gamble and should have been done in the first week. I'm sure Adam is a professional and knows his domain perfectly. Just sometimes people are unfamiliar with how many GPU/human hours could be burnt by solving a problem within a particular setting and that changing this setting could be a bit painful.\n\nNevertheless, I'm sure that 'transitivity' of a translation quality still holds (in general) for the new test. Let's just hope that it is the last change~~", "votes": 4, "replies": [ { "id": 3408391, "postDate": "2026-02-20T14:30:16.307Z", "content": "Yes, it's a good change, but wowee I spent a lot of time on stuff that is now useless 🫠
", "rawMarkdown": "Yes, it's a good change, but wowee I spent a lot of time on stuff that is now useless 🫠 ", "votes": 4 }, { "id": 3418619, "postDate": "2026-03-08T19:17:35.767Z", "content": "relatable
", "rawMarkdown": "relatable " } ] }, { "id": 3419465, "postDate": "2026-03-10T20:27:12.750Z", "content": "Wher I find that instructions
", "rawMarkdown": "Wher I find that instructions " } ] }, { "id": 3408408, "postDate": "2026-02-20T14:47:17.427Z", "content": "Honestly ,am really frustrated. I’ve been working on many patterns (like […-Suffix] and several others). Now with every dataset update, these patterns are gone or changed. My work on them is basically useless ,it feels like all my time was wasted.
\nThis cannot keep happening. If the dataset isn’t stable, the competition should start only when it’s ready. Also, are you planning to update the data one day before the competition ends? We need a stable dataset so our work actually counts.
", "rawMarkdown": "Honestly ,am really frustrated. I’ve been working on many patterns (like [...-Suffix] and several others). Now with every dataset update, these patterns are gone or changed. My work on them is basically useless ,it feels like all my time was wasted.\n\nThis cannot keep happening. If the dataset isn’t stable, the competition should start only when it’s ready. Also, are you planning to update the data one day before the competition ends? We need a stable dataset so our work actually counts.", "votes": 12, "replies": [ { "id": 3408418, "postDate": "2026-02-20T15:09:12.610Z", "content": "Having more complaints than submissions is craaaaaaaazy
", "rawMarkdown": "Having more complaints than submissions is craaaaaaaazy", "votes": 1 }, { "id": 3417189, "postDate": "2026-03-04T19:52:56.520Z", "content": "Then how the things flow
", "rawMarkdown": "Then how the things flow" } ] }, { "id": 3408858, "postDate": "2026-02-21T12:57:08.337Z", "content": "While waiting the update of update, I've reviewed the update and I'm getting lost now:
\n@mpware by fractions he meant 1 / 3 or 1 / 6. stuff like this. So these fractions are substituted with floats now. I have already lost a sub thinking about unicode fractions. Policy for fractions is unchanged
", "rawMarkdown": "@mpware by fractions he meant 1 / 3 or 1 / 6. stuff like this. So these fractions are substituted with floats now. I have already lost a sub thinking about unicode fractions. Policy for fractions is unchanged", "votes": 1, "replies": [ { "id": 3408872, "postDate": "2026-02-21T13:44:04.660Z", "content": "Hi @samson8 , so we should not try to convert 0.3333 to ⅓ and but only for 1 / 3 ? Look at the 2 examples below, sometimes we've to convert, sometimes not. It makes no sense to me. The instructions and the examples are not going in the same direction.
\nLook at this one:
\n\n\n0b84671a-8753-49d9-859d-b42c3d8944ae
\n
Transliteration:\n2 né-pí-šu 15 ma-na.TA ú iš-tí-in né-pí-šu-um 10 ma-na ni-is-ha-sú DIRI ša-du-a-sú ša-bu-ú ŠU.NÍGIN 42.3333 ma-na KÙ.BABBAR ṣa-ru-pá-am ku-nu-ki-a a-na a-lu-wa ù e-ni-ša-ri-im áp-qí-id-ma a-na a-lim{ki} a ma-lá tí-ir-tí-šu a-ṣé-er ša-lim-a-šùr ú-šé-bi-il5-šu-nu a-ha-ma 13.3333 ma-na URUDU SIG5 a-na ga-am-ri-šu-nu ù 5 GÍN KÙ.BABBAR a-na ú-ku-ul-tí-šu-nu a-dí-in IGI ili5-ba-ni DUMU ba-ší-lam IGI a-hu-qar DUMU zu-ur-zu-ur IGI tù-ra-am-ì-lí DUMU e-dí-na-a-šùr a-ha-ma 10.3333 ma-na 3.5 GÍN KÙ.BABBAR ṣa-ru-pá-am tum ni 0.5 GÍN ṣí-ba-tim ša i-na ṣé-ri-a il5-qé-ú-ni a-na hu-bu-li-šu a-na kà-ri-im wa-ah-šu-ša-na áš-qúl ṣí-ba-at KÙ.GI ša ší-ip-kà-at a-šur-bé-el-a-wa-tim i-ší-tù
\nTranslation:\n\"2 packages of 15 minas each plus a single package of 10 minas, its import duty added, its transport tariff paid - in all 42 ⅓ minas of refined silver under my seal, I entrusted to Aluwa and Enišārum, and I sent them to the City to Šalim-Aššur in accordance with his orders. Furthermore, I paid 13 ⅓ minas of good copper for their expenses and 5 shekels of silver for their food. Witnessed by Ilī-bāni son of Baši-ilum, by Ahu-qar son of Zurzur, by Tūram-ilī son of Eddin-Aššur. Furthermore, 10 ⅓ minas 3 ½ shekels of refined silver ½ shekel interest that they took on my account, I paid for his debt to the Wahšušana colony. The interest on the gold that remained of Aššur-bēl-awātims investment.
\nAnd then this one: Some are kept as floating number, some are fraction. 0.3333 ==> 0.3333, 0.16666 ==> ⅙, 0.5 ==> 0.5,
\n\n\n0faa50f7-b86c-466c-a8ab-3a6f48fcb00a:
\n
Transliteration:\n4 GÍN a-na ší-iṭ-ri-im ša pu-ki-im 1.3333 GÍN a-na e-re-qí-im qá-nu-e áš-qúl 1.6666 GÍN a sú-ba-ri-im áš-qúl 0.3333 ma-na KÙ.BABBAR a-na a-bar-ni-im áš-qúl 0.16666 GÍN a-na um-ṣí-im 0.25 GÍN a šu-um-ki na-ru-uq GIG GÍN a-na ha-áš-lá-tim a-wa-ar-nu-a-lim 0.6666 GÍN a-na e-ṣé áš-qúl 0.6666 GÍN a pá-e ú-ša-qí-il5 1 bi-il5-té-en 0.5 GÍN áš-qúl 0.25 GÍN a-na na-pá-hi-im 95 ki-ra-tim a-na 0.25 GÍN.TA ù 7.5 ŠE.TA 0.5 GÍN a-na 0.16666 GÍN a-na a-na
\nTranslation:\n\"I paid 4 shekels for a scarf of -weave, 1.3333 shekel for a wagonload of reed; I paid 1.6666 shekel for ; I paid 0.3333 mina of silver for an Abarnian textile, ⅙ shekel for a piece of dried meat, ¼ shekel for onions, x shekels for bags of wheat ;I paid shekels for ? , for a ; I paid 0.6666 shekel for firewood; 0.6666 shekels I paid for chaff, for a double load I paid 0.5 shekel; ¼ shekel for the blacksmith. I supplied 95 drinks to at 52.5 grains of silver each, 0.5 shekel for ⅙ shekel for for
", "rawMarkdown": "Hi @samson8 , so we should not try to convert 0.3333 to ⅓ and but only for 1 / 3 ? Look at the 2 examples below, sometimes we've to convert, sometimes not. It makes no sense to me. The instructions and the examples are not going in the same direction.\n\nLook at this one: \n\n>0b84671a-8753-49d9-859d-b42c3d8944ae\n\n**Transliteration**:\n2 né-pí-šu 15 ma-na.TA ú iš-tí-in né-pí-šu-um 10 ma-na ni-is-ha-sú DIRI ša-du-a-sú ša-bu-ú ŠU.NÍGIN 42.3333 ma-na KÙ.BABBAR ṣa-ru-pá-am ku-nu-ki-a a-na a-lu-wa ù e-ni-ša-ri-im áp-qí-id-ma a-na a-lim{ki} a ma-lá tí-ir-tí-šu a-ṣé-er ša-lim-a-šùr ú-šé-bi-il5-šu-nu a-ha-ma 13.3333 ma-na URUDU SIG5 a-na ga-am-ri-šu-nu ù 5 GÍN KÙ.BABBAR a-na ú-ku-ul-tí-šu-nu a-dí-in IGI ili5-ba-ni DUMU ba-ší-lam IGI a-hu-qar DUMU zu-ur-zu-ur IGI tù-ra-am-ì-lí DUMU e-dí-na-a-šùr a-ha-ma **10.3333** ma-na **3.5** GÍN KÙ.BABBAR ṣa-ru-pá-amHave the same question: submission must use ½ or 0.5? Reading above: \"returned fractions to decimals, for alignment purposes\" seems to indicate using 0.5, but scoring doesn't seem to be aware :)
", "rawMarkdown": "Have the same question: submission must use ½ or 0.5? Reading above: \"returned fractions to decimals, for alignment purposes\" seems to indicate using 0.5, but scoring doesn't seem to be aware :)", "votes": 1 }, { "id": 3408879, "postDate": "2026-02-21T13:59:57.797Z", "content": "Hidden test translations use ⅓ instead of 0.3333 and 1 / 3. And that holds for all the other numbers
", "rawMarkdown": "Hidden test translations use ⅓ instead of 0.3333 and 1 / 3. And that holds for all the other numbers", "votes": 1 }, { "id": 3408886, "postDate": "2026-02-21T14:21:09.983Z", "content": "Thanks @samson8 , I've just spent a sub too to double check on transliterations, no 0.3333 in hidden test for transliterations. You've checked that's the same for translations so I think I'm good now as it was my initial understanding.
\nI don't understand why the cleaned train data does not reflect that behavior and reflects other important normalization rules.
\nUpdate: All should be fixed on the next data update next week. Wait and see …
", "rawMarkdown": "Thanks @samson8 , I've just spent a sub too to double check on transliterations, no 0.3333 in hidden test for transliterations. You've checked that's the same for translations so I think I'm good now as it was my initial understanding. \n\nI don't understand why the cleaned train data does not reflect that behavior and reflects other important normalization rules.\n\n**Update**: All should be fixed on the next data update next week. Wait and see ...", "votes": 4 }, { "id": 3408959, "postDate": "2026-02-21T18:21:21.007Z", "content": "My advice would be to leave the normalization replacements for current cases. It won't hurt and there is no single guideline on what the fractions, signs, gaps, months, broken sections etc. should look like. There are many but I've lost count and I'm sorry too say, but a silent update of the existing \"stumbling blocks\" thread without a clear \"EDIT/UPDATE\" section makes everything all the more confusing.
", "rawMarkdown": "My advice would be to leave the normalization replacements for current cases. It won't hurt and there is no single guideline on what the fractions, signs, gaps, months, broken sections etc. should look like. There are many but I've lost count and I'm sorry too say, but a silent update of the existing \"stumbling blocks\" thread without a clear \"EDIT/UPDATE\" section makes everything all the more confusing.", "votes": 3 }, { "id": 3417712, "postDate": "2026-03-06T01:21:56.197Z", "content": "What will be ease guid for this
", "rawMarkdown": "What will be ease guid for this" } ] }, { "id": 3410865, "postDate": "2026-02-23T19:52:01.077Z", "content": "Have the same question. Does the private test set now have decimals (0.333 for example) versus fractions ?\n@deeppast
", "rawMarkdown": "Have the same question. Does the private test set now have decimals (0.333 for example) versus fractions ?\n@deeppast " } ] }, { "id": 3408017, "postDate": "2026-02-19T16:34:25.087Z", "content": "It looks like there are still some -x left.
Also, although the translation of c97bb594-a5a1-4674-9496-48496e91c2ee was originally correct, it has now been completely incorrect.
Thanks for pointing this out. I'll fix it and share a new update shortly.
", "rawMarkdown": "Thanks for pointing this out. I'll fix it and share a new update shortly.", "votes": 3 } ] }, { "id": 3407809, "postDate": "2026-02-19T07:08:35.503Z", "content": "I've spent just a few minutes on this:
\n1) Why is that you fem. plur. have written me, saying this one is confusing, same that I acquired for myself(?) -> that I acquired for myself? etc
2) (ki) -> {ki}
\n3) is in Iddin-Aššur's possession -> is in Iddin-Aššurs possession
4) subscripts
\n5) ḫ
\n6) some texts were extended (?)
\n7) quotations dropped
\n8) many digits normalized (?)
", "rawMarkdown": "I've spent just a few minutes on this:\n\n1) `Why is that you fem. plur. have written me, saying` this one is confusing, same `that I acquired for myself(?)` -> `that I acquired for myself?` etc\n\n2) (ki) -> {ki}\n\n3) `is in Iddin-Aššur's possession` -> `is in Iddin-Aššurs possession`\n\n4) subscripts\n\n5) ḫ\n\n6) some texts were extended (?)\n\n7) quotations dropped \n\n8) many digits normalized (?)", "votes": 4, "replies": [ { "id": 3407831, "postDate": "2026-02-19T08:44:18.947Z", "content": "9a208f3b-1cbb-43a6-b870-566abe5ea9a1,\" <gap> talents of wool, <gap> 5 hides, <gap> 22 sacks, 4 black donkeys, 3 saddle-rugs for each, plus their harness - all this I gave to Ewarimuša.\",\"From Ikūn-pīya to Ali-ahum: Earlier I gave a slave-girl to Buziya son of Asaya and he brought her to you. He did not give you the slave-girl, but returned and here I wrote his tablet about the price of the slave-girl, 0.5 mina 5 shekels of silver. My dear brother, there <gap> \"\nboth english
", "rawMarkdown": "```\n9a208f3b-1cbb-43a6-b870-566abe5ea9a1,\"Thanks Oleg, yes your observations are correct. Thanks for catching that, I will make the update for the training data this week and post another update shortly.
", "rawMarkdown": "Thanks Oleg, yes your observations are correct. Thanks for catching that, I will make the update for the training data this week and post another update shortly." }, { "id": 3408006, "postDate": "2026-02-19T16:06:21.953Z", "rawMarkdown": "", "isDeleted": true }, { "id": 3408062, "postDate": "2026-02-19T18:15:08.417Z", "content": "That's correct, these tablets are copies, so they are different objects with almost the same text on them.
", "rawMarkdown": "That's correct, these tablets are copies, so they are different objects with almost the same text on them.", "votes": 1 }, { "id": 3408063, "postDate": "2026-02-19T18:16:09.667Z", "content": "thanks for clarifying
", "rawMarkdown": "thanks for clarifying " } ] } ] }, { "id": 3417480, "postDate": "2026-03-05T13:47:25.173Z", "content": "Dear Ryan Holbrook, can we have BLEU and ChrF scores along with GM? This will greatly help us to see our model's performance in a transparent manner and thereby help us to diagnose our models well. You can keep GM as the final LB scoring mechanism. Thank you.
", "rawMarkdown": "Dear Ryan Holbrook, can we have BLEU and ChrF scores along with GM? This will greatly help us to see our model's performance in a transparent manner and thereby help us to diagnose our models well. You can keep GM as the final LB scoring mechanism. Thank you.", "votes": 1 }, { "id": 3408390, "postDate": "2026-02-20T14:25:58.993Z", "content": "Concerning the removal of the quotes in translations, (many, all ?) possessive english structures as brother's, father's, Šalim-Aššur's …became brothers, fathers, Šalim-Aššurs … Was it intended and will we have the same in the hiddent test set ?
Does this mean <big_gap> will no longer be part of the hidden test.csv? I can see that they are not in the updated train.csv.
Yes, exactly, the test set was updated as well, no longer <big_gap>. You can see an updated post about that here: https://www.kaggle.com/competitions/deep-past-initiative-machine-translation/discussion/665209
@ryanholbrook certain questions regarding new data:
\n<big_gap> eliminated.<gap> into one <big_gap> just in case they appearYes, that's right. <big_gap> has been replaced with <gap> and then deduplicated, so you should merge multiples. All ellipses should be removed as well and replaced with <gap. I updated the guidance for that here: https://www.kaggle.com/competitions/deep-past-initiative-machine-translation/discussion/665209
That remarkable contribution.
", "rawMarkdown": "That remarkable contribution.", "votes": -4 }, { "id": 3408392, "postDate": "2026-02-20T14:35:37.930Z", "content": "@ryanholbrook any update on rescoring? Also, will this wipe the leaderboard as well?
", "rawMarkdown": "@ryanholbrook any update on rescoring? Also, will this wipe the leaderboard as well?", "replies": [ { "id": 3408416, "postDate": "2026-02-20T15:08:35.287Z", "content": "There's another update incoming to address a couple other things pointed out here. I'll start the rescore once it's up. All of the submissions will be rescored, and the new scores will be reflected on the leaderboard. The new scores are posted as they are available, so while the rescore is ongoing the leaderboard will be a mix of old and new -- I don't think it should take more than an hour or two to complete, though.
", "rawMarkdown": "There's another update incoming to address a couple other things pointed out here. I'll start the rescore once it's up. All of the submissions will be rescored, and the new scores will be reflected on the leaderboard. The new scores are posted as they are available, so while the rescore is ongoing the leaderboard will be a mix of old and new -- I don't think it should take more than an hour or two to complete, though.", "votes": 5, "replies": [ { "id": 3408435, "postDate": "2026-02-20T16:03:58.483Z", "content": "@ryanholbrook : is there a possiblity for extension or a possiblity of increasing the submssion per day , given the data is updated after 2 months of running ..
", "rawMarkdown": "@ryanholbrook : is there a possiblity for extension or a possiblity of increasing the submssion per day , given the data is updated after 2 months of running .. " }, { "id": 3408439, "postDate": "2026-02-20T16:11:17.800Z", "content": "Data is basically not updated, It's just a clear 2-token related change~~ And still there is one more month left. It's not worth it
", "rawMarkdown": "Data is basically not updated, It's just a clear 2-token related change~~ And still there is one more month left. It's not worth it", "votes": 2 }, { "id": 3408441, "postDate": "2026-02-20T16:13:58.040Z", "content": "Yes, you are right .. :)
", "rawMarkdown": "Yes, you are right .. :)" }, { "id": 3408831, "postDate": "2026-02-21T12:00:22.707Z", "content": "@ryanholbrook I'm pretty sure that another update only adresses train issues and test will be unchanged. May be delay is not a necessity
", "rawMarkdown": "@ryanholbrook I'm pretty sure that another update only adresses train issues and test will be unchanged. May be delay is not a necessity", "votes": 1 }, { "id": 3409126, "postDate": "2026-02-22T07:33:31.283Z", "content": "I'm not so sure test will be unchanged 😀
\nif so why @ryanholbrook said
\n\n\nI'll start the rescore once it's up. \n ?
\n
I think that at least 's issue must be address on test test.
Hopefully @deeppast will be clear in the changes that were made to the final test set compared to the initial test set.
", "rawMarkdown": "I'm not so sure test will be unchanged 😀\n \nif so why @ryanholbrook said \n> I'll start the rescore once it's up. \n?\n\nI think that at least `'s` issue must be address on test test.\n\nHopefully @deeppast will be clear in the changes that were made to the final test set compared to the initial test set.\n", "votes": 1 }, { "id": 3413233, "postDate": "2026-02-24T14:16:25.897Z", "content": "Yeah still looking forward to this to ensure that all of the work I am doing actually translates.
", "rawMarkdown": "Yeah still looking forward to this to ensure that all of the work I am doing actually translates. ", "votes": 1 }, { "id": 3416762, "postDate": "2026-03-03T17:15:07.747Z", "content": "It depends
", "rawMarkdown": "It depends " } ] } ] }, { "id": 3408692, "postDate": "2026-02-21T05:57:57.553Z", "rawMarkdown": "", "isDeleted": true }, { "id": 3408209, "postDate": "2026-02-20T03:26:47.103Z", "rawMarkdown": "", "votes": 7, "isDeleted": true, "isPinned": true, "replies": [ { "id": 3408248, "postDate": "2026-02-20T06:30:45.503Z", "content": "@deeppast Thanks for clarifying the updates to the train set translations. I have a few questions:
\nPN = <gap> mean? Was the literal string “PN” replaced with <gap>, or were all personal names replaced with <gap>?Just checking, is the data section updated? I can still see fem. sing. pl. in translations
Edit - I noticed the apostrophe was removed, but I think it’s important for translation quality. ref- The current train.csv has \"Šāt-Annas representative\", but the previous version had \"Šāt-Anna's representative\".
", "rawMarkdown": "Just checking, is the data section updated? I can still see `fem.` `sing.` `pl.` in translations\n\nEdit - I noticed the apostrophe was removed, but I think it’s important for translation quality. ref- The current train.csv has \"Šāt-Annas representative\", but the previous version had \"Šāt-Anna's representative\".", "votes": 1 } ] } ], "comments": [ { "id": 3408114, "author_name": "MPWARE", "author_url": "", "post_date": "2026-02-19T20:34:56.230000", "content": "First, thank you for providing the updated, cleaner dataset - it’s clear that a lot of effort went into improving data quality, and it will ultimately benefit the challenge.
\nI do have a couple of concerns I’d like to share:
\nMid-competition changes: Updating the data and rules partway through the competition can be challenging for participants who have been working since the start of the Deep Past Challenge. Many of us invested significant time normalizing and adapting to the original data, and these efforts may no longer be applicable. While I understand the intent to improve the dataset, this kind of change can feel discouraging from a fairness standpoint.
Information consistency across platforms: Some important clarifications appear to be shared on Discord but not reflected in the official Kaggle discussion or data description. For participants who rely solely on the Kaggle platform, this creates an uneven playing field. It would be very helpful if all critical information were consolidated in the official documentation.
That said, I’m still enjoying the challenge and plan to continue participating.
\nTo help avoid further rework, especially for normalizing re-OCR data such as Larsen, could you please provide a detailed list of the normalization changes that were applied? A quick diff of train.csv suggests that, beyond replacing with , there were also changes such as quote removal, number fractions, Hh, subscripts, and possibly other text adjustments. Having a clear summary would help participants align their preprocessing steps with the updated data. Could you provide the updated list of characters allowed in transliteration and translation?
\nFinally:
\nThe thing is. This is obviously a good change that eliminates 2-token preprocessing gamble and should have been done in the first week. I'm sure Adam is a professional and knows his domain perfectly. Just sometimes people are unfamiliar with how many GPU/human hours could be burnt by solving a problem within a particular setting and that changing this setting could be a bit painful.
\nNevertheless, I'm sure that 'transitivity' of a translation quality still holds (in general) for the new test. Let's just hope that it is the last change~~
", "votes": 4, "replies": [ { "id": 3408391, "author_name": "Jack", "author_url": "", "post_date": "2026-02-20T14:30:16.307000", "content": "Yes, it's a good change, but wowee I spent a lot of time on stuff that is now useless 🫠
", "votes": 4, "replies": [] }, { "id": 3418619, "author_name": "Hafiz Atif Ali", "author_url": "", "post_date": "2026-03-08T19:17:35.767000", "content": "relatable
", "votes": 0, "replies": [] } ] }, { "id": 3419465, "author_name": "Hafiz Atif Ali", "author_url": "", "post_date": "2026-03-10T20:27:12.750000", "content": "Wher I find that instructions
", "votes": 0, "replies": [] } ] }, { "id": 3408408, "author_name": "Tarek Ziad", "author_url": "", "post_date": "2026-02-20T14:47:17.427000", "content": "Honestly ,am really frustrated. I’ve been working on many patterns (like […-Suffix] and several others). Now with every dataset update, these patterns are gone or changed. My work on them is basically useless ,it feels like all my time was wasted.
\nThis cannot keep happening. If the dataset isn’t stable, the competition should start only when it’s ready. Also, are you planning to update the data one day before the competition ends? We need a stable dataset so our work actually counts.
", "votes": 12, "replies": [ { "id": 3408418, "author_name": "Jack", "author_url": "", "post_date": "2026-02-20T15:09:12.610000", "content": "Having more complaints than submissions is craaaaaaaazy
", "votes": 1, "replies": [] }, { "id": 3417189, "author_name": "Hafiz Atif Ali", "author_url": "", "post_date": "2026-03-04T19:52:56.520000", "content": "Then how the things flow
", "votes": 0, "replies": [] } ] }, { "id": 3408858, "author_name": "MPWARE", "author_url": "", "post_date": "2026-02-21T12:57:08.337000", "content": "While waiting the update of update, I've reviewed the update and I'm getting lost now:
\n@mpware by fractions he meant 1 / 3 or 1 / 6. stuff like this. So these fractions are substituted with floats now. I have already lost a sub thinking about unicode fractions. Policy for fractions is unchanged
", "votes": 1, "replies": [ { "id": 3408872, "author_name": "MPWARE", "author_url": "", "post_date": "2026-02-21T13:44:04.660000", "content": "Hi @samson8 , so we should not try to convert 0.3333 to ⅓ and but only for 1 / 3 ? Look at the 2 examples below, sometimes we've to convert, sometimes not. It makes no sense to me. The instructions and the examples are not going in the same direction.
\nLook at this one:
\n\n\n0b84671a-8753-49d9-859d-b42c3d8944ae
\n
Transliteration:\n2 né-pí-šu 15 ma-na.TA ú iš-tí-in né-pí-šu-um 10 ma-na ni-is-ha-sú DIRI ša-du-a-sú ša-bu-ú ŠU.NÍGIN 42.3333 ma-na KÙ.BABBAR ṣa-ru-pá-am ku-nu-ki-a a-na a-lu-wa ù e-ni-ša-ri-im áp-qí-id-ma a-na a-lim{ki} a ma-lá tí-ir-tí-šu a-ṣé-er ša-lim-a-šùr ú-šé-bi-il5-šu-nu a-ha-ma 13.3333 ma-na URUDU SIG5 a-na ga-am-ri-šu-nu ù 5 GÍN KÙ.BABBAR a-na ú-ku-ul-tí-šu-nu a-dí-in IGI ili5-ba-ni DUMU ba-ší-lam IGI a-hu-qar DUMU zu-ur-zu-ur IGI tù-ra-am-ì-lí DUMU e-dí-na-a-šùr a-ha-ma 10.3333 ma-na 3.5 GÍN KÙ.BABBAR ṣa-ru-pá-am tum ni 0.5 GÍN ṣí-ba-tim ša i-na ṣé-ri-a il5-qé-ú-ni a-na hu-bu-li-šu a-na kà-ri-im wa-ah-šu-ša-na áš-qúl ṣí-ba-at KÙ.GI ša ší-ip-kà-at a-šur-bé-el-a-wa-tim i-ší-tù
\nTranslation:\n\"2 packages of 15 minas each plus a single package of 10 minas, its import duty added, its transport tariff paid - in all 42 ⅓ minas of refined silver under my seal, I entrusted to Aluwa and Enišārum, and I sent them to the City to Šalim-Aššur in accordance with his orders. Furthermore, I paid 13 ⅓ minas of good copper for their expenses and 5 shekels of silver for their food. Witnessed by Ilī-bāni son of Baši-ilum, by Ahu-qar son of Zurzur, by Tūram-ilī son of Eddin-Aššur. Furthermore, 10 ⅓ minas 3 ½ shekels of refined silver ½ shekel interest that they took on my account, I paid for his debt to the Wahšušana colony. The interest on the gold that remained of Aššur-bēl-awātims investment.
\nAnd then this one: Some are kept as floating number, some are fraction. 0.3333 ==> 0.3333, 0.16666 ==> ⅙, 0.5 ==> 0.5,
\n\n\n0faa50f7-b86c-466c-a8ab-3a6f48fcb00a:
\n
Transliteration:\n4 GÍN a-na ší-iṭ-ri-im ša pu-ki-im 1.3333 GÍN a-na e-re-qí-im qá-nu-e áš-qúl 1.6666 GÍN a sú-ba-ri-im áš-qúl 0.3333 ma-na KÙ.BABBAR a-na a-bar-ni-im áš-qúl 0.16666 GÍN a-na um-ṣí-im 0.25 GÍN a šu-um-ki na-ru-uq GIG GÍN a-na ha-áš-lá-tim a-wa-ar-nu-a-lim 0.6666 GÍN a-na e-ṣé áš-qúl 0.6666 GÍN a pá-e ú-ša-qí-il5 1 bi-il5-té-en 0.5 GÍN áš-qúl 0.25 GÍN a-na na-pá-hi-im 95 ki-ra-tim a-na 0.25 GÍN.TA ù 7.5 ŠE.TA 0.5 GÍN a-na 0.16666 GÍN a-na a-na
\nTranslation:\n\"I paid 4 shekels for a scarf of -weave, 1.3333 shekel for a wagonload of reed; I paid 1.6666 shekel for ; I paid 0.3333 mina of silver for an Abarnian textile, ⅙ shekel for a piece of dried meat, ¼ shekel for onions, x shekels for bags of wheat ;I paid shekels for ? , for a ; I paid 0.6666 shekel for firewood; 0.6666 shekels I paid for chaff, for a double load I paid 0.5 shekel; ¼ shekel for the blacksmith. I supplied 95 drinks to at 52.5 grains of silver each, 0.5 shekel for ⅙ shekel for for
", "votes": 2, "replies": [] }, { "id": 3408878, "author_name": "vinpro", "author_url": "", "post_date": "2026-02-21T13:58:24.260000", "content": "Have the same question: submission must use ½ or 0.5? Reading above: \"returned fractions to decimals, for alignment purposes\" seems to indicate using 0.5, but scoring doesn't seem to be aware :)
", "votes": 1, "replies": [] }, { "id": 3408879, "author_name": "Yurnero", "author_url": "", "post_date": "2026-02-21T13:59:57.797000", "content": "Hidden test translations use ⅓ instead of 0.3333 and 1 / 3. And that holds for all the other numbers
", "votes": 1, "replies": [] }, { "id": 3408886, "author_name": "MPWARE", "author_url": "", "post_date": "2026-02-21T14:21:09.983000", "content": "Thanks @samson8 , I've just spent a sub too to double check on transliterations, no 0.3333 in hidden test for transliterations. You've checked that's the same for translations so I think I'm good now as it was my initial understanding.
\nI don't understand why the cleaned train data does not reflect that behavior and reflects other important normalization rules.
\nUpdate: All should be fixed on the next data update next week. Wait and see …
", "votes": 4, "replies": [] }, { "id": 3408959, "author_name": "Angantyr", "author_url": "", "post_date": "2026-02-21T18:21:21.007000", "content": "My advice would be to leave the normalization replacements for current cases. It won't hurt and there is no single guideline on what the fractions, signs, gaps, months, broken sections etc. should look like. There are many but I've lost count and I'm sorry too say, but a silent update of the existing \"stumbling blocks\" thread without a clear \"EDIT/UPDATE\" section makes everything all the more confusing.
", "votes": 3, "replies": [] }, { "id": 3417712, "author_name": "Hafiz Atif Ali", "author_url": "", "post_date": "2026-03-06T01:21:56.197000", "content": "What will be ease guid for this
", "votes": 0, "replies": [] } ] }, { "id": 3410865, "author_name": "tucking_fired", "author_url": "", "post_date": "2026-02-23T19:52:01.077000", "content": "Have the same question. Does the private test set now have decimals (0.333 for example) versus fractions ?\n@deeppast
", "votes": 0, "replies": [] } ] }, { "id": 3408017, "author_name": "esprit", "author_url": "", "post_date": "2026-02-19T16:34:25.087000", "content": "It looks like there are still some -x left.
Also, although the translation of c97bb594-a5a1-4674-9496-48496e91c2ee was originally correct, it has now been completely incorrect.
Thanks for pointing this out. I'll fix it and share a new update shortly.
", "votes": 3, "replies": [] } ] }, { "id": 3407809, "author_name": "Oleg Yaroshevskiy", "author_url": "", "post_date": "2026-02-19T07:08:35.503000", "content": "I've spent just a few minutes on this:
\n1) Why is that you fem. plur. have written me, saying this one is confusing, same that I acquired for myself(?) -> that I acquired for myself? etc
2) (ki) -> {ki}
\n3) is in Iddin-Aššur's possession -> is in Iddin-Aššurs possession
4) subscripts
\n5) ḫ
\n6) some texts were extended (?)
\n7) quotations dropped
\n8) many digits normalized (?)
", "votes": 4, "replies": [ { "id": 3407831, "author_name": "Oleg Yaroshevskiy", "author_url": "", "post_date": "2026-02-19T08:44:18.947000", "content": "9a208f3b-1cbb-43a6-b870-566abe5ea9a1,\" <gap> talents of wool, <gap> 5 hides, <gap> 22 sacks, 4 black donkeys, 3 saddle-rugs for each, plus their harness - all this I gave to Ewarimuša.\",\"From Ikūn-pīya to Ali-ahum: Earlier I gave a slave-girl to Buziya son of Asaya and he brought her to you. He did not give you the slave-girl, but returned and here I wrote his tablet about the price of the slave-girl, 0.5 mina 5 shekels of silver. My dear brother, there <gap> \"\nboth english
", "votes": 3, "replies": [ { "id": 3407967, "author_name": "Adam Anderson", "author_url": "", "post_date": "2026-02-19T14:51:47.120000", "content": "Thanks Oleg, yes your observations are correct. Thanks for catching that, I will make the update for the training data this week and post another update shortly.
", "votes": 0, "replies": [] }, { "id": 3408006, "author_name": "", "author_url": "", "post_date": "2026-02-19T16:06:21.953000", "content": "", "votes": 0, "replies": [] }, { "id": 3408062, "author_name": "Adam Anderson", "author_url": "", "post_date": "2026-02-19T18:15:08.417000", "content": "That's correct, these tablets are copies, so they are different objects with almost the same text on them.
", "votes": 1, "replies": [] }, { "id": 3408063, "author_name": "AK", "author_url": "", "post_date": "2026-02-19T18:16:09.667000", "content": "thanks for clarifying
", "votes": 0, "replies": [] } ] } ] }, { "id": 3417480, "author_name": "KishoreKashyap", "author_url": "", "post_date": "2026-03-05T13:47:25.173000", "content": "Dear Ryan Holbrook, can we have BLEU and ChrF scores along with GM? This will greatly help us to see our model's performance in a transparent manner and thereby help us to diagnose our models well. You can keep GM as the final LB scoring mechanism. Thank you.
", "votes": 1, "replies": [] }, { "id": 3408390, "author_name": "FabienDaniel", "author_url": "", "post_date": "2026-02-20T14:25:58.993000", "content": "Concerning the removal of the quotes in translations, (many, all ?) possessive english structures as brother's, father's, Šalim-Aššur's …became brothers, fathers, Šalim-Aššurs … Was it intended and will we have the same in the hiddent test set ?
Does this mean <big_gap> will no longer be part of the hidden test.csv? I can see that they are not in the updated train.csv.
Yes, exactly, the test set was updated as well, no longer <big_gap>. You can see an updated post about that here: https://www.kaggle.com/competitions/deep-past-initiative-machine-translation/discussion/665209
@ryanholbrook certain questions regarding new data:
\n<big_gap> eliminated.<gap> into one <big_gap> just in case they appearYes, that's right. <big_gap> has been replaced with <gap> and then deduplicated, so you should merge multiples. All ellipses should be removed as well and replaced with <gap. I updated the guidance for that here: https://www.kaggle.com/competitions/deep-past-initiative-machine-translation/discussion/665209
@deeppast Perhaps the new guidance would be clearer if presented in a new post rather than updating a month-old thread whose replies are no longer valid.
", "votes": 6, "replies": [] } ] } ] }, { "id": 3410860, "author_name": "Hafiz Atif Ali", "author_url": "", "post_date": "2026-02-23T19:30:17.520000", "content": "That remarkable contribution.
", "votes": -4, "replies": [] }, { "id": 3408392, "author_name": "Jack", "author_url": "", "post_date": "2026-02-20T14:35:37.930000", "content": "@ryanholbrook any update on rescoring? Also, will this wipe the leaderboard as well?
", "votes": 0, "replies": [ { "id": 3408416, "author_name": "Ryan Holbrook", "author_url": "", "post_date": "2026-02-20T15:08:35.287000", "content": "There's another update incoming to address a couple other things pointed out here. I'll start the rescore once it's up. All of the submissions will be rescored, and the new scores will be reflected on the leaderboard. The new scores are posted as they are available, so while the rescore is ongoing the leaderboard will be a mix of old and new -- I don't think it should take more than an hour or two to complete, though.
", "votes": 5, "replies": [ { "id": 3408435, "author_name": "AK", "author_url": "", "post_date": "2026-02-20T16:03:58.483000", "content": "@ryanholbrook : is there a possiblity for extension or a possiblity of increasing the submssion per day , given the data is updated after 2 months of running ..
", "votes": 0, "replies": [] }, { "id": 3408439, "author_name": "Yurnero", "author_url": "", "post_date": "2026-02-20T16:11:17.800000", "content": "Data is basically not updated, It's just a clear 2-token related change~~ And still there is one more month left. It's not worth it
", "votes": 2, "replies": [] }, { "id": 3408441, "author_name": "AK", "author_url": "", "post_date": "2026-02-20T16:13:58.040000", "content": "Yes, you are right .. :)
", "votes": 0, "replies": [] }, { "id": 3408831, "author_name": "Yurnero", "author_url": "", "post_date": "2026-02-21T12:00:22.707000", "content": "@ryanholbrook I'm pretty sure that another update only adresses train issues and test will be unchanged. May be delay is not a necessity
", "votes": 1, "replies": [] }, { "id": 3409126, "author_name": "steubk", "author_url": "", "post_date": "2026-02-22T07:33:31.283000", "content": "I'm not so sure test will be unchanged 😀
\nif so why @ryanholbrook said
\n\n\nI'll start the rescore once it's up. \n ?
\n
I think that at least 's issue must be address on test test.
Hopefully @deeppast will be clear in the changes that were made to the final test set compared to the initial test set.
", "votes": 1, "replies": [] }, { "id": 3413233, "author_name": "Cody_Null", "author_url": "", "post_date": "2026-02-24T14:16:25.897000", "content": "Yeah still looking forward to this to ensure that all of the work I am doing actually translates.
", "votes": 1, "replies": [] }, { "id": 3416762, "author_name": "Hafiz Atif Ali", "author_url": "", "post_date": "2026-03-03T17:15:07.747000", "content": "It depends
", "votes": 0, "replies": [] } ] } ] }, { "id": 3408692, "author_name": "", "author_url": "", "post_date": "2026-02-21T05:57:57.553000", "content": "", "votes": 0, "replies": [] }, { "id": 3408209, "author_name": "", "author_url": "", "post_date": "2026-02-20T03:26:47.103000", "content": "", "votes": 7, "replies": [ { "id": 3408248, "author_name": "steubk", "author_url": "", "post_date": "2026-02-20T06:30:45.503000", "content": "@deeppast Thanks for clarifying the updates to the train set translations. I have a few questions:
\nPN = <gap> mean? Was the literal string “PN” replaced with <gap>, or were all personal names replaced with <gap>?Just checking, is the data section updated? I can still see fem. sing. pl. in translations
Edit - I noticed the apostrophe was removed, but I think it’s important for translation quality. ref- The current train.csv has \"Šāt-Annas representative\", but the previous version had \"Šāt-Anna's representative\".
", "votes": 1, "replies": [] } ] } ], "raw_markdown_by_id": { "3407672": "Hi everyone,\n\nI just posted an update to the dataset that regularizes `