![\"url\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F4004698%2Fdcaf9313ee33874e6ee4b32d6398a8dd%2FSCR-20260327-tjze.png?generation=1774618029414321&alt=media\")
Thank you to the competition hosts for this fascinating challenge and to all participants for the insightful discussions throughout.
\nPrivate LB: 36 | +222 shake-up (382nd public -> 160th private)
\nThe core of this solution was data engineering -- extracting parallel corpora from scholarly PDFs using Gemini Vision, which expanded the training set from ~1,500 to ~36,750 rows. On the modeling side: mT5-large 4-fold ensemble with MBR decoding.
\nWith only ~1,500 document-level pairs, the competition was a data problem. The AKT publication PDFs contained thousands of transliteration-translation pairs buried in scholarly text.
\nSentence-first extraction: Gemini extracted each PDF page as a list of sentence-level pairs, not a single document blob. From these, I produced two views:
\nBoth views were concatenated into a single training set. <eos> separators were replaced with periods so the model could leverage its English sentence boundary understanding.
| Source | \nRows | \n
|---|---|
| AKT volumes (12 vols, sentence+doc split) | \n23,963 | \n
| CAD dictionary (13 volumes) | \n4,059 | \n
| Secondary scholarly sources | \n3,630 | \n
| ICK IV 1998 | \n1,857 | \n
| Larsen 2002 | \n1,681 | \n
| Subtotal (publications_csv) | \n35,190 | \n
| Competition train.csv | \n1,561 | \n
| Total training pool | \n~36,750 | \n
Started with ByT5-base (byte-level, no OOV issues) but switched to mT5 for speed and vocabulary coverage. mT5's subword tokenizer produces ~4x shorter sequences than ByT5's byte-level encoding, enabling ensemble + MBR within Kaggle's 9-hour time limit.
\nScale-up: mT5-base (LB 33.1) -> mT5-large (LB 34.6 single, 36.0 ensemble). BF16 on vast.ai GPUs.
\n| Parameter | \nValue | \n
|---|---|
| Model | \nmT5-large | \n
| Optimizer | \nAdamW, cosine LR 5e-5, 5% warmup | \n
| Batch | \n8 | \n
| Epochs | \n20-40, early stopping patience 5 | \n
| Label smoothing | \n0.1 | \n
| Precision | \nBF16 | \n
| Validation | \nseries_group_kfold (5 folds, leave-series-out) | \n
Validation strategy: series_group_kfold (5 folds, leave-series-out) -- a GroupKFold where each fold holds out an entire publication series (e.g., AKT 6a). This prevents series-level data leakage, since documents within the same series share vocabulary, formulaic expressions, and proper nouns.
For each source, generated 10 candidates per model:
\nGeoMeanMBRSelector picks the candidate with highest average pairwise sqrt(BLEU * chrF++) against all others.
5-fold CV but only 4 folds used for ensemble -- 5 models × 10 candidates × ~4,000 sentences exceeded Kaggle's 9-hour time limit. Merged pools from 4 mT5-large folds (40 candidates total). CharConstraintProcessor masks disallowed characters during beam search (prevents CJK/Cyrillic generation from mT5).
Source (Akkadian):
\nsz->š, s,->ṣ, t,->ṭ, vowel+index to accents (a2->á, a3->à)ḫ/Ḫ -> h/H (confirmed by host)[...], [x x x], ... -> <gap>, with deduplication(d) -> {d}!, ?) and brackets (keep content)0.5 -> 1/2Target (English):
\n-gold -> pasallum gold), grammar marker removalPN -> <gap>-gold -> pasallum gold)Thank you to the competition hosts for this fascinating challenge and to all participants for the insightful discussions throughout.
\nPrivate LB: 36 | +222 shake-up (382nd public -> 160th private)
\nThe core of this solution was data engineering -- extracting parallel corpora from scholarly PDFs using Gemini Vision, which expanded the training set from ~1,500 to ~36,750 rows. On the modeling side: mT5-large 4-fold ensemble with MBR decoding.
\nWith only ~1,500 document-level pairs, the competition was a data problem. The AKT publication PDFs contained thousands of transliteration-translation pairs buried in scholarly text.
\nSentence-first extraction: Gemini extracted each PDF page as a list of sentence-level pairs, not a single document blob. From these, I produced two views:
\nBoth views were concatenated into a single training set. <eos> separators were replaced with periods so the model could leverage its English sentence boundary understanding.
| Source | \nRows | \n
|---|---|
| AKT volumes (12 vols, sentence+doc split) | \n23,963 | \n
| CAD dictionary (13 volumes) | \n4,059 | \n
| Secondary scholarly sources | \n3,630 | \n
| ICK IV 1998 | \n1,857 | \n
| Larsen 2002 | \n1,681 | \n
| Subtotal (publications_csv) | \n35,190 | \n
| Competition train.csv | \n1,561 | \n
| Total training pool | \n~36,750 | \n
Started with ByT5-base (byte-level, no OOV issues) but switched to mT5 for speed and vocabulary coverage. mT5's subword tokenizer produces ~4x shorter sequences than ByT5's byte-level encoding, enabling ensemble + MBR within Kaggle's 9-hour time limit.
\nScale-up: mT5-base (LB 33.1) -> mT5-large (LB 34.6 single, 36.0 ensemble). BF16 on vast.ai GPUs.
\n| Parameter | \nValue | \n
|---|---|
| Model | \nmT5-large | \n
| Optimizer | \nAdamW, cosine LR 5e-5, 5% warmup | \n
| Batch | \n8 | \n
| Epochs | \n20-40, early stopping patience 5 | \n
| Label smoothing | \n0.1 | \n
| Precision | \nBF16 | \n
| Validation | \nseries_group_kfold (5 folds, leave-series-out) | \n
Validation strategy: series_group_kfold (5 folds, leave-series-out) -- a GroupKFold where each fold holds out an entire publication series (e.g., AKT 6a). This prevents series-level data leakage, since documents within the same series share vocabulary, formulaic expressions, and proper nouns.
For each source, generated 10 candidates per model:
\nGeoMeanMBRSelector picks the candidate with highest average pairwise sqrt(BLEU * chrF++) against all others.
5-fold CV but only 4 folds used for ensemble -- 5 models × 10 candidates × ~4,000 sentences exceeded Kaggle's 9-hour time limit. Merged pools from 4 mT5-large folds (40 candidates total). CharConstraintProcessor masks disallowed characters during beam search (prevents CJK/Cyrillic generation from mT5).
Source (Akkadian):
\nsz->š, s,->ṣ, t,->ṭ, vowel+index to accents (a2->á, a3->à)ḫ/Ḫ -> h/H (confirmed by host)[...], [x x x], ... -> <gap>, with deduplication(d) -> {d}!, ?) and brackets (keep content)0.5 -> 1/2Target (English):
\n-gold -> pasallum gold), grammar marker removalPN -> <gap>-gold -> pasallum gold)