![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F2259646%2F6172f1dd39fc4adf91d4f645676cb3cc%2FNumber%20alignment%20between%20transliterations%20and%20translations%20in%20train.csv.png?generation=1768251784155423&alt=media\"){kind=spacer}
Some time ago I posted a short analysis on how some translations were too short for their corresponding transliterations and more recently how insight gained from \"Sentences_Oare_FirstWord_LinNum.csv\" confirmed that the problem.
\nThe problem became elusive for a lack of good rule to pinpoint the faulty texts, until I followed a conversation with Aaron about number allignment.
\nWhen implemented, this yielded around 400 texts, which gives about 25% of total texts from train.csv. That's significant and could impact the performance of the models learning to predict something that is not there.
The number is not fixed and is strongly dependent on the pattern I used for capturing the consecutive number sequences in transliterations and in translations, then aligning the matches by ID and identifying mismatches.
\n'((?:\\d+\\.*\\d*)*\\s*(?:(?:ma-na)|GÍN|GÚ|(?:me-at))(?:(?:\\s|\\sù\\s)(?:\\d+\\.*\\d*)\\s(?:(?:ma-na)|GÍN|GÚ))*)'\nThis captures all sequencess that:
\nù-separated (Old Assyrian for and)((?:(?:\\d+\\.*\\d*)|(?:(?:\\d+\\s?\\d+)\\s?/\\s?\\d+)|one|two|three|four|five|six|seven|eight|nine|ten)*\\s*(?:(?:mina[s]?)|s[hk]{1}e[hk]{1}el[s]?|talent[s]?))(?:(?:\\s|\\sand\\s|\\sor\\s)(?:(?:\\d+\\.*\\d*)|one|two|three|four|five|six|seven|eight|nine|ten)\\s(?:(?:mina[s]?)|s[hk]{1}e[hk]{1}el[s]?|talent[s]?))*\nThis works similarly to the transliteration pattern but accounts for more numerals (one up to ten) and typos in the word shekel (sometimes spelled skekel).
\nUsing the above captures patterns results in matches, which alligned based on the text index give the alignment:\n
If removed, the correlation between the word lengths (split on whitespace) of transliterations and translations becomes more apparent. The texts have not been cleaned prior to the splitting, which is the source of additional noise. The gaps should be collapsed to <gap> and <big_gap> but I wanted to share some information on raw data first.\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F2259646%2F05b431e89a56186025427773e469d4a8%2FTranslation%20len%20vs%20transliteration%20len.png?generation=1768251530227407&alt=media\"){kind=spacer}
That and more information is shared in the analysis notebook.
\nPersonally I would not remove the texts altogether, since there are already signs and words poorly represented. A clean way to crop the texts at the last matching point could be of potential benefit, though.
", "messages": [ { "id": "3390256", "postDate": "01/12/2026 21:11:17", "content": "Some time ago I posted a short analysis on how some translations were too short for their corresponding transliterations and more recently how insight gained from \"Sentences_Oare_FirstWord_LinNum.csv\" confirmed that the problem.
\nThe problem became elusive for a lack of good rule to pinpoint the faulty texts, until I followed a conversation with Aaron about number allignment.
\nWhen implemented, this yielded around 400 texts, which gives about 25% of total texts from train.csv. That's significant and could impact the performance of the models learning to predict something that is not there.
The number is not fixed and is strongly dependent on the pattern I used for capturing the consecutive number sequences in transliterations and in translations, then aligning the matches by ID and identifying mismatches.
\n'((?:\\d+\\.*\\d*)*\\s*(?:(?:ma-na)|GÍN|GÚ|(?:me-at))(?:(?:\\s|\\sù\\s)(?:\\d+\\.*\\d*)\\s(?:(?:ma-na)|GÍN|GÚ))*)'\nThis captures all sequencess that:
\nù-separated (Old Assyrian for and)((?:(?:\\d+\\.*\\d*)|(?:(?:\\d+\\s?\\d+)\\s?/\\s?\\d+)|one|two|three|four|five|six|seven|eight|nine|ten)*\\s*(?:(?:mina[s]?)|s[hk]{1}e[hk]{1}el[s]?|talent[s]?))(?:(?:\\s|\\sand\\s|\\sor\\s)(?:(?:\\d+\\.*\\d*)|one|two|three|four|five|six|seven|eight|nine|ten)\\s(?:(?:mina[s]?)|s[hk]{1}e[hk]{1}el[s]?|talent[s]?))*\nThis works similarly to the transliteration pattern but accounts for more numerals (one up to ten) and typos in the word shekel (sometimes spelled skekel).
\nUsing the above captures patterns results in matches, which alligned based on the text index give the alignment:\n
If removed, the correlation between the word lengths (split on whitespace) of transliterations and translations becomes more apparent. The texts have not been cleaned prior to the splitting, which is the source of additional noise. The gaps should be collapsed to <gap> and <big_gap> but I wanted to share some information on raw data first.\n
That and more information is shared in the analysis notebook.
\nPersonally I would not remove the texts altogether, since there are already signs and words poorly represented. A clean way to crop the texts at the last matching point could be of potential benefit, though.
", "rawMarkdown": "Some time ago I posted [a short analysis on how some translations were too short for their corresponding transliterations](https://www.kaggle.com/competitions/deep-past-initiative-machine-translation/discussion/663849) and more recently [how insight gained from \"Sentences_Oare_FirstWord_LinNum.csv\" confirmed that the problem](https://www.kaggle.com/competitions/deep-past-initiative-machine-translation/discussion/664177#3384304).\n\nThe problem became elusive for a lack of good rule to pinpoint the faulty texts, until [I followed a conversation with Aaron about number allignment](https://www.kaggle.com/competitions/deep-past-initiative-machine-translation/discussion/666745#3388631).\n\nWhen implemented, this yielded around 400 texts, which gives about 25% of total texts from `train.csv`. That's significant and could impact the performance of the models learning to predict something that is not there.\n\nThe number is not fixed and is strongly dependent on the pattern I used for capturing the consecutive number sequences in transliterations and in translations, then aligning the matches by ID and identifying mismatches.\n\nI'm noticing that the main reason i don't think this is impacting the leaderboard is that everyone seems to be truncating after 512 chars when training on the documents if we want to take advantage of chunking the full text I think approaches like this are going to be important.
", "rawMarkdown": "I'm noticing that the main reason i don't think this is impacting the leaderboard is that everyone seems to be truncating after 512 chars when training on the documents if we want to take advantage of chunking the full text I think approaches like this are going to be important.", "votes": null }, { "id": "3391863", "postDate": "01/15/2026 20:31:15", "content": "100% agree on the chunking. I think the \"necessary\" transliterated texts can be way shorter than 512 chars if one uses things like PN, GN, quantities, etc.
", "rawMarkdown": "100% agree on the chunking. I think the \"necessary\" transliterated texts can be way shorter than 512 chars if one uses things like PN, GN, quantities, etc.", "votes": null }, { "id": "3392050", "postDate": "01/16/2026 08:20:02", "content": "It’s an interesting idea using masking and slot filling to make the model generalize I saw some one try it in another thread though and the model just learned to generate the text PN son of PN. I think the model attention can lose crucial token context if you do that since part of how byT5 learns is by learning latent feature relationships between the input and output characters. If you mask all these terms at once you end up with less information to learn on. If you’re going to do this approach what I would suggest is applying augmented masking probabilistically on different samples in your batch that way you can try to learn these relationships and let your model generalize better.
", "rawMarkdown": "It’s an interesting idea using masking and slot filling to make the model generalize I saw some one try it in another thread though and the model just learned to generate the text PN son of PN. I think the model attention can lose crucial token context if you do that since part of how byT5 learns is by learning latent feature relationships between the input and output characters. If you mask all these terms at once you end up with less information to learn on. If you’re going to do this approach what I would suggest is applying augmented masking probabilistically on different samples in your batch that way you can try to learn these relationships and let your model generalize better.", "votes": null }, { "id": "3392055", "postDate": "01/16/2026 08:35:29", "content": "Oh, no, I meant fragmenting the text and sacrificing the bigger context, i.e.,
\n\n\n\n
Um-ma PN DUMU PN a-na PN <some words> 5 GÍN <some words> 0.3333 ma-na <some words> PN
would be separated into:
\n\n\n[
\nUm-ma PN DUMU PN a-na PN <some words> 5 GÍN,5 GÍN <some words> 0.3333 ma-na, 0.3333 ma-na PN`]
Every chunk is then translated and stitched together on common and known separators. The more such \"static\" separators are found the smaller the text to translate becomes.
", "rawMarkdown": "Oh, no, I meant fragmenting the text and sacrificing the bigger context, i.e.,\n> `Um-ma PN DUMU PN a-na PNI think I understand better, but I’m still a little confused whether this kind of segmentation will work cleanly because the subject verb object order is different between the two languages. What you’re trying to do is use the core units as segmentation and alignment anchors. You should check out my alignment notebook it has a lot of these ideas.
", "rawMarkdown": "I think I understand better, but I’m still a little confused whether this kind of segmentation will work cleanly because the subject verb object order is different between the two languages. What you’re trying to do is use the core units as segmentation and alignment anchors. You should check out my alignment notebook it has a lot of these ideas.", "votes": null } ], "comments": [ { "id": 3391850, "author_name": "aaronbornstein", "author_url": "", "post_date": "01/15/2026 19:54:14", "content": "I'm noticing that the main reason i don't think this is impacting the leaderboard is that everyone seems to be truncating after 512 chars when training on the documents if we want to take advantage of chunking the full text I think approaches like this are going to be important.
", "votes": null, "replies": [ { "id": 3391863, "author_name": "angantyr", "author_url": "", "post_date": "01/15/2026 20:31:15", "content": "100% agree on the chunking. I think the \"necessary\" transliterated texts can be way shorter than 512 chars if one uses things like PN, GN, quantities, etc.
", "votes": null, "replies": [ { "id": 3392050, "author_name": "aaronbornstein", "author_url": "", "post_date": "01/16/2026 08:20:02", "content": "It’s an interesting idea using masking and slot filling to make the model generalize I saw some one try it in another thread though and the model just learned to generate the text PN son of PN. I think the model attention can lose crucial token context if you do that since part of how byT5 learns is by learning latent feature relationships between the input and output characters. If you mask all these terms at once you end up with less information to learn on. If you’re going to do this approach what I would suggest is applying augmented masking probabilistically on different samples in your batch that way you can try to learn these relationships and let your model generalize better.
", "votes": null, "replies": [ { "id": 3392055, "author_name": "angantyr", "author_url": "", "post_date": "01/16/2026 08:35:29", "content": "Oh, no, I meant fragmenting the text and sacrificing the bigger context, i.e.,
\n\n\n\n
Um-ma PN DUMU PN a-na PN <some words> 5 GÍN <some words> 0.3333 ma-na <some words> PN
would be separated into:
\n\n\n[
\nUm-ma PN DUMU PN a-na PN <some words> 5 GÍN,5 GÍN <some words> 0.3333 ma-na, 0.3333 ma-na PN`]
Every chunk is then translated and stitched together on common and known separators. The more such \"static\" separators are found the smaller the text to translate becomes.
", "votes": null, "replies": [ { "id": 3392145, "author_name": "aaronbornstein", "author_url": "", "post_date": "01/16/2026 10:56:39", "content": "I think I understand better, but I’m still a little confused whether this kind of segmentation will work cleanly because the subject verb object order is different between the two languages. What you’re trying to do is use the core units as segmentation and alignment anchors. You should check out my alignment notebook it has a lot of these ideas.
", "votes": null, "replies": [] } ] } ] } ] } ] } ], "raw_markdown_by_id": { "3390256": "Some time ago I posted [a short analysis on how some translations were too short for their corresponding transliterations](https://www.kaggle.com/competitions/deep-past-initiative-machine-translation/discussion/663849) and more recently [how insight gained from \"Sentences_Oare_FirstWord_LinNum.csv\" confirmed that the problem](https://www.kaggle.com/competitions/deep-past-initiative-machine-translation/discussion/664177#3384304).\n\nThe problem became elusive for a lack of good rule to pinpoint the faulty texts, until [I followed a conversation with Aaron about number allignment](https://www.kaggle.com/competitions/deep-past-initiative-machine-translation/discussion/666745#3388631).\n\nWhen implemented, this yielded around 400 texts, which gives about 25% of total texts from `train.csv`. That's significant and could impact the performance of the models learning to predict something that is not there.\n\nThe number is not fixed and is strongly dependent on the pattern I used for capturing the consecutive number sequences in transliterations and in translations, then aligning the matches by ID and identifying mismatches.\n\n