I noticed that the English text after translation sometimes sounded unnatural or awkward. To address this issue, I have shared code that applies LLM-based post-processing to improve the quality of the translated text.
\nThe current implementation is based on the publicly available Best Score code, with additional post-processing using Gemma-3.
\nhttps://www.kaggle.com/code/takamichitoda/dpc-infer-with-llm-pp
\nAlthough the LLM post-processing improved readability, it did not improve the leaderboard score and often made it worse.
\nFor example, the LLM tends to “correct” formulaic expressions:
\n\n\n“… he will add interest at the rate 1.5 shekel per mina per month.”
\n
\n → “… he will add interest at a rate of 1.5 shekels per mina per month.”
While linguistically more natural, such changes reduce surface-form overlap with the reference translation.\nSimilar issues occur with numbers, units, and proper names with diacritics (š, ṣ, ī), where even minor normalization or pluralization can hurt the score.
\nSince the evaluation metric appears to rely heavily on surface-level matching, even small paraphrasing introduced by LLM post-processing can be detrimental.
\nIn this competition, preserving literal phrasing and consistency seems more important than improving fluency via LLM-based rewriting.
", "messages": [ { "id": "3380446", "postDate": "12/22/2025 12:35:11", "content": "I noticed that the English text after translation sometimes sounded unnatural or awkward. To address this issue, I have shared code that applies LLM-based post-processing to improve the quality of the translated text.
\nThe current implementation is based on the publicly available Best Score code, with additional post-processing using Gemma-3.
\nhttps://www.kaggle.com/code/takamichitoda/dpc-infer-with-llm-pp
\nAlthough the LLM post-processing improved readability, it did not improve the leaderboard score and often made it worse.
\nFor example, the LLM tends to “correct” formulaic expressions:
\n\n\n“… he will add interest at the rate 1.5 shekel per mina per month.”
\n
\n → “… he will add interest at a rate of 1.5 shekels per mina per month.”
While linguistically more natural, such changes reduce surface-form overlap with the reference translation.\nSimilar issues occur with numbers, units, and proper names with diacritics (š, ṣ, ī), where even minor normalization or pluralization can hurt the score.
\nSince the evaluation metric appears to rely heavily on surface-level matching, even small paraphrasing introduced by LLM post-processing can be detrimental.
\nIn this competition, preserving literal phrasing and consistency seems more important than improving fluency via LLM-based rewriting.
", "rawMarkdown": "I noticed that the English text after translation sometimes sounded unnatural or awkward. To address this issue, I have shared code that applies LLM-based post-processing to improve the quality of the translated text.\n\nThe current implementation is based on [the publicly available Best Score code](https://www.kaggle.com/code/xbar19/deep-past-challenge-byt5-base-inference), with additional post-processing using Gemma-3.\n\nhttps://www.kaggle.com/code/takamichitoda/dpc-infer-with-llm-pp\n\nAlthough the LLM post-processing improved readability, it did not improve the leaderboard score and often made it worse.\n\nFor example, the LLM tends to “correct” formulaic expressions:\n\n>“… he will add interest at the rate 1.5 shekel per mina per month.” \n→ “… he will add interest at a rate of 1.5 shekels per mina per month.”\n\nWhile linguistically more natural, such changes reduce surface-form overlap with the reference translation.\nSimilar issues occur with numbers, units, and proper names with diacritics (š, ṣ, ī), where even minor normalization or pluralization can hurt the score.\n\nSince the evaluation metric appears to rely heavily on surface-level matching, even small paraphrasing introduced by LLM post-processing can be detrimental.\n\nIn this competition, preserving literal phrasing and consistency seems more important than improving fluency via LLM-based rewriting.", "votes": null }, { "id": "3380592", "postDate": "12/22/2025 17:38:13", "content": "I feel a better way for using the LLMs is for sentence alignment since heuristic based alignments aren't changing the input structure or clean the input transliterations. Any post-processing on this task will make the overall scores more prone to a shakeup during the complete evaluation of the test dataset due to the nature of the evaluation metrics.
", "rawMarkdown": "I feel a better way for using the LLMs is for sentence alignment since heuristic based alignments aren't changing the input structure or clean the input transliterations. Any post-processing on this task will make the overall scores more prone to a shakeup during the complete evaluation of the test dataset due to the nature of the evaluation metrics.", "votes": null }, { "id": "3380759", "postDate": "12/23/2025 01:32:43", "content": "I thought your intentions were good, but sometimes repeating some words can improve the BLEU score.
", "rawMarkdown": "I thought your intentions were good, but sometimes repeating some words can improve the BLEU score.", "votes": null }, { "id": "3380788", "postDate": "12/23/2025 04:10:57", "content": "Thank you, much needed @takamichitoda
", "rawMarkdown": "Thank you, much needed @takamichitoda", "votes": null }, { "id": "3381009", "postDate": "12/23/2025 15:12:36", "content": "Yes, that's a good observation. Most specialists prefer a more literal translation, since there is no such thing as fluency for these ancient languages. The translations we gathered were from the specialists in formal publication, rather than translations made for the general public.
", "rawMarkdown": "Yes, that's a good observation. Most specialists prefer a more literal translation, since there is no such thing as fluency for these ancient languages. The translations we gathered were from the specialists in formal publication, rather than translations made for the general public.", "votes": null } ], "comments": [ { "id": 3380592, "author_name": "shrenzypanda", "author_url": "", "post_date": "12/22/2025 17:38:13", "content": "I feel a better way for using the LLMs is for sentence alignment since heuristic based alignments aren't changing the input structure or clean the input transliterations. Any post-processing on this task will make the overall scores more prone to a shakeup during the complete evaluation of the test dataset due to the nature of the evaluation metrics.
", "votes": null, "replies": [] }, { "id": 3380759, "author_name": "hengzheng", "author_url": "", "post_date": "12/23/2025 01:32:43", "content": "I thought your intentions were good, but sometimes repeating some words can improve the BLEU score.
", "votes": null, "replies": [] }, { "id": 3380788, "author_name": "navneetbende", "author_url": "", "post_date": "12/23/2025 04:10:57", "content": "Thank you, much needed @takamichitoda
", "votes": null, "replies": [] }, { "id": 3381009, "author_name": "deeppast", "author_url": "", "post_date": "12/23/2025 15:12:36", "content": "Yes, that's a good observation. Most specialists prefer a more literal translation, since there is no such thing as fluency for these ancient languages. The translations we gathered were from the specialists in formal publication, rather than translations made for the general public.
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "3380446": "I noticed that the English text after translation sometimes sounded unnatural or awkward. To address this issue, I have shared code that applies LLM-based post-processing to improve the quality of the translated text.\n\nThe current implementation is based on [the publicly available Best Score code](https://www.kaggle.com/code/xbar19/deep-past-challenge-byt5-base-inference), with additional post-processing using Gemma-3.\n\nhttps://www.kaggle.com/code/takamichitoda/dpc-infer-with-llm-pp\n\nAlthough the LLM post-processing improved readability, it did not improve the leaderboard score and often made it worse.\n\nFor example, the LLM tends to “correct” formulaic expressions:\n\n>“… he will add interest at the rate 1.5 shekel per mina per month.” \n→ “… he will add interest at a rate of 1.5 shekels per mina per month.”\n\nWhile linguistically more natural, such changes reduce surface-form overlap with the reference translation.\nSimilar issues occur with numbers, units, and proper names with diacritics (š, ṣ, ī), where even minor normalization or pluralization can hurt the score.\n\nSince the evaluation metric appears to rely heavily on surface-level matching, even small paraphrasing introduced by LLM post-processing can be detrimental.\n\nIn this competition, preserving literal phrasing and consistency seems more important than improving fluency via LLM-based rewriting.", "3380592": "I feel a better way for using the LLMs is for sentence alignment since heuristic based alignments aren't changing the input structure or clean the input transliterations. Any post-processing on this task will make the overall scores more prone to a shakeup during the complete evaluation of the test dataset due to the nature of the evaluation metrics.", "3380759": "I thought your intentions were good, but sometimes repeating some words can improve the BLEU score.", "3380788": "Thank you, much needed @takamichitoda", "3381009": "Yes, that's a good observation. Most specialists prefer a more literal translation, since there is no such thing as fluency for these ancient languages. The translations we gathered were from the specialists in formal publication, rather than translations made for the general public." }, "source": "meta" }