It seems unclear to me how the training data is supposed to be formatted to achieve the high result. I have read the instructions, however a lot of it is very unclear. On my cross validation I am getting very high GM scores like 42-46, however i'm getting garbage scores on the LB of 21-26. I suspect heavily that it's got to do with the gap handling, because that seems to make a significant difference from my probings. Is there any reference implementation that shows how gaps are handled during training for both translation and transliteration to achieve that baseline score?
\nI have already done all of the fraction normalization, letters normalization, etc… that were suggested to do across all posts, so this is quite confusing to me why I might be getting such a low score when i'm getting a high score locally.
", "messages": [ { "id": 3396833, "postDate": "2026-01-26T01:50:34.243Z", "content": "It seems unclear to me how the training data is supposed to be formatted to achieve the high result. I have read the instructions, however a lot of it is very unclear. On my cross validation I am getting very high GM scores like 42-46, however i'm getting garbage scores on the LB of 21-26. I suspect heavily that it's got to do with the gap handling, because that seems to make a significant difference from my probings. Is there any reference implementation that shows how gaps are handled during training for both translation and transliteration to achieve that baseline score?
\nI have already done all of the fraction normalization, letters normalization, etc… that were suggested to do across all posts, so this is quite confusing to me why I might be getting such a low score when i'm getting a high score locally.
", "rawMarkdown": "It seems unclear to me how the training data is supposed to be formatted to achieve the high result. I have read the instructions, however a lot of it is very unclear. On my cross validation I am getting very high GM scores like 42-46, however i'm getting garbage scores on the LB of 21-26. I suspect heavily that it's got to do with the gap handling, because that seems to make a significant difference from my probings. Is there any reference implementation that shows how gaps are handled during training for both translation and transliteration to achieve that baseline score?\n\nI have already done all of the fraction normalization, letters normalization, etc... that were suggested to do across all posts, so this is quite confusing to me why I might be getting such a low score when i'm getting a high score locally.", "votes": 15 }, { "id": 3396893, "postDate": "2026-01-26T05:22:12.240Z", "content": "speaking for myself: i have a single model reaching 34.5 what i did is simply formatting (preprocessing). I'm guessing formatting will get you up to roughly the range of ~36-37. However, I am beginning to lose interest in this competition since much of the gain is from formatting which the host can prepare before this competition and let the kaggle community working on actually translating rather than book-keeping. The host can probably do a better job at formatting and their ByT5 is likely to outperform ours if all we do is formatting… (Correct me if I am wrong here!) With that said, I believe top teams are doing something more than preprocessing, perhaps they are trying different training methods and data aug.
", "rawMarkdown": "speaking for myself: i have a single model reaching 34.5 what i did is simply formatting (preprocessing). I'm guessing formatting will get you up to roughly the range of ~36-37. However, I am beginning to lose interest in this competition since much of the gain is from formatting which the host can prepare before this competition and let the kaggle community working on actually translating rather than book-keeping. The host can probably do a better job at formatting and their ByT5 is likely to outperform ours if all we do is formatting... (Correct me if I am wrong here!) With that said, I believe top teams are doing something more than preprocessing, perhaps they are trying different training methods and data aug. ", "votes": 10, "replies": [ { "id": 3397390, "postDate": "2026-01-27T07:00:55.520Z", "rawMarkdown": "", "isDeleted": true } ] }, { "id": 3396847, "postDate": "2026-01-26T03:21:35.327Z", "content": "I could share my experience although I haven't achieved 34.5 myself, so it might not be accurate, just for your reference.
\nWeeks ago I trained on raw train.csv with heavy normalization, I only got a LB score of 28.4.\nI later aligned half of the instances in train.csv and mixed them with the rest of the rows, and the score went up to 31.4.\nAfter that I tried bidirectional training and got a model of score 32.4.\nEnsembling the two models with a random ratio gave me 33.0.
\nSo to me, the sentence alignment problem could be the major cause of a 21-26 LB score. If you check the train.csv file closely, you might find out that about a half of them are misaligned to some extent, i.e. the transliteration or the translation contains more info than the other. Also, for long instances, if you set max token to 512, the model would only use the first 512 bytes for training and validation, which is also a source of misalignment.
\nAs for gap handling, one possible reference would be the transliteration_orig and transliteration columns in published_texts.csv. You could try applying the gap fixing function to the transliteration_orig column and see if it matches the transliteration column. However there is no guarantee that the hidden test is handled in the exactly same way (well, the Data section says it is the \"Clean version of the text transliteration based on these formatting suggestions.\").
", "rawMarkdown": "I could share my experience although I haven't achieved 34.5 myself, so it might not be accurate, just for your reference.\n\nWeeks ago I trained on raw train.csv with heavy normalization, I only got a LB score of 28.4.\nI later aligned half of the instances in train.csv and mixed them with the rest of the rows, and the score went up to 31.4.\nAfter that I tried bidirectional training and got a model of score 32.4.\nEnsembling the two models with a random ratio gave me 33.0.\n\nSo to me, the sentence alignment problem could be the major cause of a 21-26 LB score. If you check the train.csv file closely, you might find out that about a half of them are misaligned to some extent, i.e. the transliteration or the translation contains more info than the other. Also, for long instances, if you set max token to 512, the model would only use the first 512 bytes for training and validation, which is also a source of misalignment.\n\nAs for gap handling, one possible reference would be the transliteration_orig and transliteration columns in published_texts.csv. You could try applying the gap fixing function to the transliteration_orig column and see if it matches the transliteration column. However there is no guarantee that the hidden test is handled in the exactly same way (well, the Data section says it is the \"Clean version of the text transliteration based on these formatting suggestions.\").", "votes": 3, "replies": [ { "id": 3396850, "postDate": "2026-01-26T03:25:17.007Z", "content": "The local gm is not necessarily indicative since it depends on how you build the validation set. My local gm could reach 60 as currently I only validate on shorter sentence-level instances.
", "rawMarkdown": "The local gm is not necessarily indicative since it depends on how you build the validation set. My local gm could reach 60 as currently I only validate on shorter sentence-level instances.", "votes": 1 }, { "id": 3396966, "postDate": "2026-01-26T08:00:11.777Z", "content": "Do you have a heuristic for identifying misaligned transliteration-translation pairs? I was only able to precisely identify around 25% of the train.csv as misaligned.
I checked them one by one with a LLM. Hope someone might come up with a better approach😂.
\nThe estimate of \"50%\" could be imprecise and misleading, as I included cases where transliteration or translation is significantly longer, many instances generated from Sentences_Oare_FirstWord_LinNum.csv that fail to locate the correct first word, and word-level cases where the translation omit an Akkadian word or translated wrongly (e.g. -textiles -> kutānu-textiles; number mismatches…). I haven’t finished this task yet, so the estimate is still a rough guess. Apologies for any confusion my words may have caused.
", "rawMarkdown": "I checked them one by one with a LLM. Hope someone might come up with a better approach😂. \n\nThe estimate of \"50%\" could be imprecise and misleading, as I included cases where transliteration or translation is significantly longer, many instances generated from Sentences_Oare_FirstWord_LinNum.csv that fail to locate the correct first word, and word-level cases where the translation omit an Akkadian word or translated wrongly (e.g. -textiles -> kutānu-textiles; number mismatches...). I haven’t finished this task yet, so the estimate is still a rough guess. Apologies for any confusion my words may have caused.", "votes": 2 } ] } ] } ], "comments": [ { "id": 3396893, "author_name": "hongan", "author_url": "", "post_date": "2026-01-26T05:22:12.240000", "content": "speaking for myself: i have a single model reaching 34.5 what i did is simply formatting (preprocessing). I'm guessing formatting will get you up to roughly the range of ~36-37. However, I am beginning to lose interest in this competition since much of the gain is from formatting which the host can prepare before this competition and let the kaggle community working on actually translating rather than book-keeping. The host can probably do a better job at formatting and their ByT5 is likely to outperform ours if all we do is formatting… (Correct me if I am wrong here!) With that said, I believe top teams are doing something more than preprocessing, perhaps they are trying different training methods and data aug.
", "votes": 10, "replies": [ { "id": 3397390, "author_name": "", "author_url": "", "post_date": "2026-01-27T07:00:55.520000", "content": "", "votes": 0, "replies": [] } ] }, { "id": 3396847, "author_name": "Tomorin", "author_url": "", "post_date": "2026-01-26T03:21:35.327000", "content": "I could share my experience although I haven't achieved 34.5 myself, so it might not be accurate, just for your reference.
\nWeeks ago I trained on raw train.csv with heavy normalization, I only got a LB score of 28.4.\nI later aligned half of the instances in train.csv and mixed them with the rest of the rows, and the score went up to 31.4.\nAfter that I tried bidirectional training and got a model of score 32.4.\nEnsembling the two models with a random ratio gave me 33.0.
\nSo to me, the sentence alignment problem could be the major cause of a 21-26 LB score. If you check the train.csv file closely, you might find out that about a half of them are misaligned to some extent, i.e. the transliteration or the translation contains more info than the other. Also, for long instances, if you set max token to 512, the model would only use the first 512 bytes for training and validation, which is also a source of misalignment.
\nAs for gap handling, one possible reference would be the transliteration_orig and transliteration columns in published_texts.csv. You could try applying the gap fixing function to the transliteration_orig column and see if it matches the transliteration column. However there is no guarantee that the hidden test is handled in the exactly same way (well, the Data section says it is the \"Clean version of the text transliteration based on these formatting suggestions.\").
", "votes": 3, "replies": [ { "id": 3396850, "author_name": "Tomorin", "author_url": "", "post_date": "2026-01-26T03:25:17.007000", "content": "The local gm is not necessarily indicative since it depends on how you build the validation set. My local gm could reach 60 as currently I only validate on shorter sentence-level instances.
", "votes": 1, "replies": [] }, { "id": 3396966, "author_name": "Angantyr", "author_url": "", "post_date": "2026-01-26T08:00:11.777000", "content": "Do you have a heuristic for identifying misaligned transliteration-translation pairs? I was only able to precisely identify around 25% of the train.csv as misaligned.
I checked them one by one with a LLM. Hope someone might come up with a better approach😂.
\nThe estimate of \"50%\" could be imprecise and misleading, as I included cases where transliteration or translation is significantly longer, many instances generated from Sentences_Oare_FirstWord_LinNum.csv that fail to locate the correct first word, and word-level cases where the translation omit an Akkadian word or translated wrongly (e.g. -textiles -> kutānu-textiles; number mismatches…). I haven’t finished this task yet, so the estimate is still a rough guess. Apologies for any confusion my words may have caused.
", "votes": 2, "replies": [] } ] } ] } ], "raw_markdown_by_id": { "3396833": "It seems unclear to me how the training data is supposed to be formatted to achieve the high result. I have read the instructions, however a lot of it is very unclear. On my cross validation I am getting very high GM scores like 42-46, however i'm getting garbage scores on the LB of 21-26. I suspect heavily that it's got to do with the gap handling, because that seems to make a significant difference from my probings. Is there any reference implementation that shows how gaps are handled during training for both translation and transliteration to achieve that baseline score?\n\nI have already done all of the fraction normalization, letters normalization, etc... that were suggested to do across all posts, so this is quite confusing to me why I might be getting such a low score when i'm getting a high score locally.", "3396893": "speaking for myself: i have a single model reaching 34.5 what i did is simply formatting (preprocessing). I'm guessing formatting will get you up to roughly the range of ~36-37. However, I am beginning to lose interest in this competition since much of the gain is from formatting which the host can prepare before this competition and let the kaggle community working on actually translating rather than book-keeping. The host can probably do a better job at formatting and their ByT5 is likely to outperform ours if all we do is formatting... (Correct me if I am wrong here!) With that said, I believe top teams are doing something more than preprocessing, perhaps they are trying different training methods and data aug. ", "3396847": "I could share my experience although I haven't achieved 34.5 myself, so it might not be accurate, just for your reference.\n\nWeeks ago I trained on raw train.csv with heavy normalization, I only got a LB score of 28.4.\nI later aligned half of the instances in train.csv and mixed them with the rest of the rows, and the score went up to 31.4.\nAfter that I tried bidirectional training and got a model of score 32.4.\nEnsembling the two models with a random ratio gave me 33.0.\n\nSo to me, the sentence alignment problem could be the major cause of a 21-26 LB score. If you check the train.csv file closely, you might find out that about a half of them are misaligned to some extent, i.e. the transliteration or the translation contains more info than the other. Also, for long instances, if you set max token to 512, the model would only use the first 512 bytes for training and validation, which is also a source of misalignment.\n\nAs for gap handling, one possible reference would be the transliteration_orig and transliteration columns in published_texts.csv. You could try applying the gap fixing function to the transliteration_orig column and see if it matches the transliteration column. However there is no guarantee that the hidden test is handled in the exactly same way (well, the Data section says it is the \"Clean version of the text transliteration based on these formatting suggestions.\")." } }