Many thanks to the hosts and Kaggle. This challenge was a unique chance to delve into Akkadian (Old Assyrian) and gain insights into Mesopotamian history, ancient maps, and the region's modern geopolitical landscape. Below is a summary of our solution.
\n1. Data Preprocessing & Cleaning\nWe aligned the training data at the sentence level and focused on cleaning the eBL_Dictionary. Our pipeline included:\nExtracting high-value translation pairs.\nRemoving nulls and duplicates to ensure data quality.
\n2. Model Fine-tuning\nWe fine-tuned both ByT5-base and ByT5-large models on the cleaned dataset.\nIterative Improvement: We filtered samples from train.csv that achieved high scores based on the Geometric Mean of BLEU and chrF++. We then performed additional fine-tuning on this high-quality subset, which yielded an improvement.\nChallenge: Despite these improvements, our single model's performance still lagged behind some of the top public notebooks.
\n3. Ensembling & Inference Strategy\nTo bridge the gap, we leveraged a bit of luck and extensive searching within the Kaggle \"Models\" and \"Datasets\" sections. We identified three high-performing public models and integrated them into our solution.
\n \"/kaggle/input/datasets/assiaben/final-byt5/byt5-akkadian-optimized-34x\",\n \"/kaggle/input/datasets/heriqis777/byt5-akkadian-mbr-3804\",\n \"/kaggle/input/models/mattiaangeli/byt5-akkadian-mbr-v2/pytorch/default/1\",\nMBR Decoding: Crucially, we incorporated the official competition metric directly into our MBR decoding process to maximize our score.
\nKey Hyperparameters
\nnum_beam_cands: int = 8\nnum_beams: int = 8\nlength_penalty: float = 1.0\nrepetition_penalty: float = 1.1\nnum_sample_per_temp: int = 8\ntemperatures: [0.70, 0.90]\nmbr_pool_cap: int = 64 \nWhat NOT to try\nLarge language models are inherently stochastic. Do not rely on multiple runs to chase a lucky high score. While this might temporarily boost your public leaderboard ranking, it often results in a significant drop on the private leaderboard once final results are revealed.
\nThanks again for this inspiring competition. We learned a great deal and hope this write-up helps others in their future!
", "messages": [ { "id": 3427490, "postDate": "2026-03-24T06:29:49.447Z", "content": "Many thanks to the hosts and Kaggle. This challenge was a unique chance to delve into Akkadian (Old Assyrian) and gain insights into Mesopotamian history, ancient maps, and the region's modern geopolitical landscape. Below is a summary of our solution.
\n1. Data Preprocessing & Cleaning\nWe aligned the training data at the sentence level and focused on cleaning the eBL_Dictionary. Our pipeline included:\nExtracting high-value translation pairs.\nRemoving nulls and duplicates to ensure data quality.
\n2. Model Fine-tuning\nWe fine-tuned both ByT5-base and ByT5-large models on the cleaned dataset.\nIterative Improvement: We filtered samples from train.csv that achieved high scores based on the Geometric Mean of BLEU and chrF++. We then performed additional fine-tuning on this high-quality subset, which yielded an improvement.\nChallenge: Despite these improvements, our single model's performance still lagged behind some of the top public notebooks.
\n3. Ensembling & Inference Strategy\nTo bridge the gap, we leveraged a bit of luck and extensive searching within the Kaggle \"Models\" and \"Datasets\" sections. We identified three high-performing public models and integrated them into our solution.
\n \"/kaggle/input/datasets/assiaben/final-byt5/byt5-akkadian-optimized-34x\",\n \"/kaggle/input/datasets/heriqis777/byt5-akkadian-mbr-3804\",\n \"/kaggle/input/models/mattiaangeli/byt5-akkadian-mbr-v2/pytorch/default/1\",\nMBR Decoding: Crucially, we incorporated the official competition metric directly into our MBR decoding process to maximize our score.
\nKey Hyperparameters
\nnum_beam_cands: int = 8\nnum_beams: int = 8\nlength_penalty: float = 1.0\nrepetition_penalty: float = 1.1\nnum_sample_per_temp: int = 8\ntemperatures: [0.70, 0.90]\nmbr_pool_cap: int = 64 \nWhat NOT to try\nLarge language models are inherently stochastic. Do not rely on multiple runs to chase a lucky high score. While this might temporarily boost your public leaderboard ranking, it often results in a significant drop on the private leaderboard once final results are revealed.
\nThanks again for this inspiring competition. We learned a great deal and hope this write-up helps others in their future!
", "rawMarkdown": "Many thanks to the hosts and Kaggle. This challenge was a unique chance to delve into Akkadian (Old Assyrian) and gain insights into Mesopotamian history, ancient maps, and the region's modern geopolitical landscape. Below is a summary of our solution.\n\n**1. Data Preprocessing & Cleaning**\nWe aligned the training data at the sentence level and focused on cleaning the eBL_Dictionary. Our pipeline included:\nExtracting high-value translation pairs.\nRemoving nulls and duplicates to ensure data quality.\n\n**2. Model Fine-tuning**\nWe fine-tuned both ByT5-base and ByT5-large models on the cleaned dataset.\nIterative Improvement: We filtered samples from train.csv that achieved high scores based on the Geometric Mean of BLEU and chrF++. We then performed additional fine-tuning on this high-quality subset, which yielded an improvement.\nChallenge: Despite these improvements, our single model's performance still lagged behind some of the top public notebooks.\n\n**3. Ensembling & Inference Strategy**\nTo bridge the gap, we leveraged a bit of luck and extensive searching within the Kaggle \"Models\" and \"Datasets\" sections. We identified three high-performing public models and integrated them into our solution.\n```python\n \"/kaggle/input/datasets/assiaben/final-byt5/byt5-akkadian-optimized-34x\",\n \"/kaggle/input/datasets/heriqis777/byt5-akkadian-mbr-3804\",\n \"/kaggle/input/models/mattiaangeli/byt5-akkadian-mbr-v2/pytorch/default/1\",\n```\nMBR Decoding: Crucially, we incorporated the official competition metric directly into our MBR decoding process to maximize our score.\n\n**Key Hyperparameters**\n```python\nnum_beam_cands: int = 8\nnum_beams: int = 8\nlength_penalty: float = 1.0\nrepetition_penalty: float = 1.1\nnum_sample_per_temp: int = 8\ntemperatures: [0.70, 0.90]\nmbr_pool_cap: int = 64 \n```\n**What NOT to try**\nLarge language models are inherently stochastic. Do not rely on multiple runs to chase a lucky high score. While this might temporarily boost your public leaderboard ranking, it often results in a significant drop on the private leaderboard once final results are revealed.\n\nThanks again for this inspiring competition. We learned a great deal and hope this write-up helps others in their future!", "votes": 2 } ], "comments": [], "raw_markdown_by_id": { "3427490": "Many thanks to the hosts and Kaggle. This challenge was a unique chance to delve into Akkadian (Old Assyrian) and gain insights into Mesopotamian history, ancient maps, and the region's modern geopolitical landscape. Below is a summary of our solution.\n\n**1. Data Preprocessing & Cleaning**\nWe aligned the training data at the sentence level and focused on cleaning the eBL_Dictionary. Our pipeline included:\nExtracting high-value translation pairs.\nRemoving nulls and duplicates to ensure data quality.\n\n**2. Model Fine-tuning**\nWe fine-tuned both ByT5-base and ByT5-large models on the cleaned dataset.\nIterative Improvement: We filtered samples from train.csv that achieved high scores based on the Geometric Mean of BLEU and chrF++. We then performed additional fine-tuning on this high-quality subset, which yielded an improvement.\nChallenge: Despite these improvements, our single model's performance still lagged behind some of the top public notebooks.\n\n**3. Ensembling & Inference Strategy**\nTo bridge the gap, we leveraged a bit of luck and extensive searching within the Kaggle \"Models\" and \"Datasets\" sections. We identified three high-performing public models and integrated them into our solution.\n```python\n \"/kaggle/input/datasets/assiaben/final-byt5/byt5-akkadian-optimized-34x\",\n \"/kaggle/input/datasets/heriqis777/byt5-akkadian-mbr-3804\",\n \"/kaggle/input/models/mattiaangeli/byt5-akkadian-mbr-v2/pytorch/default/1\",\n```\nMBR Decoding: Crucially, we incorporated the official competition metric directly into our MBR decoding process to maximize our score.\n\n**Key Hyperparameters**\n```python\nnum_beam_cands: int = 8\nnum_beams: int = 8\nlength_penalty: float = 1.0\nrepetition_penalty: float = 1.1\nnum_sample_per_temp: int = 8\ntemperatures: [0.70, 0.90]\nmbr_pool_cap: int = 64 \n```\n**What NOT to try**\nLarge language models are inherently stochastic. Do not rely on multiple runs to chase a lucky high score. While this might temporarily boost your public leaderboard ranking, it often results in a significant drop on the private leaderboard once final results are revealed.\n\nThanks again for this inspiring competition. We learned a great deal and hope this write-up helps others in their future!" } }