Our final solution was based on an ensemble of two ByT5 translation models combined with a Minimum Bayes Risk (MBR) decoding framework.
\nInstead of selecting the highest-probability output from a single model, we generated a diverse pool of translation candidates from multiple decoding strategies and selected the candidate with the strongest consensus across the pool.
\nThe key idea was that candidate diversity and consensus-based selection produced more robust translations than relying solely on beam search.
\nThe Deep Past Initiative Machine Translation task involves translating Akkadian transliterations into English.
\nSeveral characteristics made the problem challenging:
\nBecause of these issues, preprocessing and postprocessing played an important role in overall performance.
\nBefore inference, transliterations were normalized to reduce notation variability.
\nThe preprocessing pipeline included:
\nExamples:
\nsz → š\ns, → ṣ\nt, → ṭ\nGap indicators from multiple formats were unified into a single <gap> token.
This reduced fragmentation in the input space and improved consistency between similar texts.
\nThe final system used two independently trained ByT5 models:
\nModel A: akkadian-optimized\nModel B: akkadian-mbr\nFor every test sample, both models generated candidate translations.
\nUsing multiple models reduced model-specific biases and increased candidate diversity.
\nInstead of generating a single translation, each model produced multiple candidates.
\nFor each model:
\nBeam search provided high-probability translations.
\nAdditional candidates were generated using stochastic decoding:
\nTemperature = 0.60\nTemperature = 0.80\nTemperature = 1.05\nTwo samples were generated for each temperature.
\nThis produced:
\n4 beam candidates\n6 sampled candidates\n------------------\n10 candidates/model\nWith two models:
\n20 candidates/sample\n(before deduplication)\nThe purpose was to create a heterogeneous candidate pool from which MBR could select the most reliable translation.
\nAfter generation, translations were normalized using a rule-based postprocessing pipeline.
\nImportant corrections included:
\nSpecial care was taken to preserve meaningful content while removing formatting noise.
\nThe core of the solution was a weighted MBR selection procedure.
\nInstead of selecting the candidate with the highest model probability, we selected the candidate with the highest agreement with all other candidates.
\nEach candidate pair was evaluated using:
\n| Metric | \nWeight | \n
|---|---|
| chrF++ | \n0.55 | \n
| BLEU | \n0.25 | \n
| Jaccard similarity | \n0.20 | \n
Additionally, a length-consistency bonus was applied.
\nThe final score was:
\nMBR Score =\n0.55 × chrF++\n+ 0.25 × BLEU\n+ 0.20 × Jaccard\n+ Length Bonus\nThe candidate with the highest consensus score was selected as the final translation.
\nBeam search often produced several translations that were very similar but differed in wording.
\nSampling generated more diverse alternatives.
\nMBR acted as a consensus mechanism:
\nThis generally improved robustness compared with selecting the top beam candidate directly.
\nTo keep inference efficient:
\nThese optimizations allowed larger candidate pools without exceeding resource constraints.
\nInput Transliteration\n ↓\nPreprocessing\n ↓\nModel A (ByT5)\n ↓\nBeam + Sampling Candidates\n ↓\n\nModel B (ByT5)\n ↓\nBeam + Sampling Candidates\n ↓\n\nCandidate Pool Merge\n ↓\nPostprocessing\n ↓\nWeighted MBR Selection\n ↓\nFinal Translation\nThe most important lesson from this competition was that candidate diversity can be as valuable as model quality.
\nThe largest gains came from:
\nRather than relying on a single best prediction, aggregating evidence from multiple models and decoding strategies produced more stable translations and better overall performance.
\nOur final solution was based on an ensemble of two ByT5 translation models combined with a Minimum Bayes Risk (MBR) decoding framework.
\nInstead of selecting the highest-probability output from a single model, we generated a diverse pool of translation candidates from multiple decoding strategies and selected the candidate with the strongest consensus across the pool.
\nThe key idea was that candidate diversity and consensus-based selection produced more robust translations than relying solely on beam search.
\nThe Deep Past Initiative Machine Translation task involves translating Akkadian transliterations into English.
\nSeveral characteristics made the problem challenging:
\nBecause of these issues, preprocessing and postprocessing played an important role in overall performance.
\nBefore inference, transliterations were normalized to reduce notation variability.
\nThe preprocessing pipeline included:
\nExamples:
\nsz → š\ns, → ṣ\nt, → ṭ\nGap indicators from multiple formats were unified into a single <gap> token.
This reduced fragmentation in the input space and improved consistency between similar texts.
\nThe final system used two independently trained ByT5 models:
\nModel A: akkadian-optimized\nModel B: akkadian-mbr\nFor every test sample, both models generated candidate translations.
\nUsing multiple models reduced model-specific biases and increased candidate diversity.
\nInstead of generating a single translation, each model produced multiple candidates.
\nFor each model:
\nBeam search provided high-probability translations.
\nAdditional candidates were generated using stochastic decoding:
\nTemperature = 0.60\nTemperature = 0.80\nTemperature = 1.05\nTwo samples were generated for each temperature.
\nThis produced:
\n4 beam candidates\n6 sampled candidates\n------------------\n10 candidates/model\nWith two models:
\n20 candidates/sample\n(before deduplication)\nThe purpose was to create a heterogeneous candidate pool from which MBR could select the most reliable translation.
\nAfter generation, translations were normalized using a rule-based postprocessing pipeline.
\nImportant corrections included:
\nSpecial care was taken to preserve meaningful content while removing formatting noise.
\nThe core of the solution was a weighted MBR selection procedure.
\nInstead of selecting the candidate with the highest model probability, we selected the candidate with the highest agreement with all other candidates.
\nEach candidate pair was evaluated using:
\n| Metric | \nWeight | \n
|---|---|
| chrF++ | \n0.55 | \n
| BLEU | \n0.25 | \n
| Jaccard similarity | \n0.20 | \n
Additionally, a length-consistency bonus was applied.
\nThe final score was:
\nMBR Score =\n0.55 × chrF++\n+ 0.25 × BLEU\n+ 0.20 × Jaccard\n+ Length Bonus\nThe candidate with the highest consensus score was selected as the final translation.
\nBeam search often produced several translations that were very similar but differed in wording.
\nSampling generated more diverse alternatives.
\nMBR acted as a consensus mechanism:
\nThis generally improved robustness compared with selecting the top beam candidate directly.
\nTo keep inference efficient:
\nThese optimizations allowed larger candidate pools without exceeding resource constraints.
\nInput Transliteration\n ↓\nPreprocessing\n ↓\nModel A (ByT5)\n ↓\nBeam + Sampling Candidates\n ↓\n\nModel B (ByT5)\n ↓\nBeam + Sampling Candidates\n ↓\n\nCandidate Pool Merge\n ↓\nPostprocessing\n ↓\nWeighted MBR Selection\n ↓\nFinal Translation\nThe most important lesson from this competition was that candidate diversity can be as valuable as model quality.
\nThe largest gains came from:
\nRather than relying on a single best prediction, aggregating evidence from multiple models and decoding strategies produced more stable translations and better overall performance.
\n