First, I would like to thank the Kaggle hosts for organizing this competition and for providing such an interesting problem.
\nMy final solution was a contextual ByT5-large system with chunked ensemble inference, but the biggest gains came from data work rather than model changes.
\nOverall, the system was built around three ideas: better corpus construction, stronger style adaptation, and more careful chunk-level inference.
\nApproximate contribution to the final score (from ablations)\n\nPrivate LB:\nData adaptation [########################......] +0.93\nChunking [#########.....................] +0.15 to +0.35\nByT5-Large [#########.....................] +0.35\nEnsemble + MBR [###...........................] +0.11\nPost-edit / GR [..............................] ~0.00\n\nPublic LB:\nData adaptation [##########################....] +1.00\nChunking [##############################] +0.82 to +1.15\nByT5-Large [###################...........] +0.73\nEnsemble + MBR [..............................] no consistent gain\nPost-edit / GR [###...........................] +0.10\nI prepared two corpora for training:
\nI found several useful APIs.
\nUUID feeder:\nhttps://oare.byu.edu/api/v2/search?characters=&textTitle=AKT&page=1&rows=100&mode=respectAllBoundaries
Tablet feeder:\nhttps://oare.byu.edu/api/v2/text_epigraphies/text/${uuid}?forceAllowAdminView=false
CDLI provides an easy way to download all Old Assyrian tablets:
\ncurl -L 'https://cdli.earth/search?period=old+ass&limit=10000&aspect=inscriptions&format=atf' -o old_assyrian.atf\nAfter downloading the data from CDLI, I converted it into the OARE format.
\nI merged the two datasets above. After that, I grouped the rows of each tablet into 4–6 chunks; if fewer than two lines remained, I merged them into the previous chunk.
\nAn example after merging:
\n{\"text_id\": \"055b4918-e8aa-4442-99b6-cc4d6f33adf4\", \"designation\": \"AKT 1 30\", \"header_raw\": \"&055b4918-e8aa-4442-99b6-cc4d6f33adf4 = AKT 1 30\", \"atf\": {\"lang\": \"akk\"}, \"sections\": [\"tablet\", \"obverse\", \"reverse\"], \"lines\": [{\"label\": [\"1\", \"2\", \"3\", \"4\"], \"section\": \"obverse\", \"text\": \"4 i-lá-tim ku nu ki ša a-mur-IŠTAR ma-aḫ-ri-ni a-du-ú ù i-ku-pì-a\", \"translation\": null}, {\"label\": [\"5\", \"6\", \"7\", \"8\"], \"section\": \"obverse\", \"text\": \"iš ru mu ma i-na 3 i-lá-tim 3 GÚ 7+0.5 ma-na URUDU is ni qú\", \"translation\": null}, {\"label\": [\"9\", \"10\", \"11\", \"12\"], \"section\": \"reverse\", \"text\": \"a-ḫa-ma i-na 1 i-li-im 20+6 ma-na ta <gap> mu <gap> ù ša a-ba-ri-im\", \"translation\": null}, {\"label\": [\"13\", \"14\", \"15\"], \"section\": \"reverse\", \"text\": \"is ni qú ú IGI a-šur-iš-tí-kál IGI en-nam-a-a\", \"translation\": null}], \"annotations\": []}\nI used Gemini 3 Flash Preview to fill in missing translations.
\nPrompt example:
\nUse the following reference notes while translating Old Assyrian transliteration lines:\n\n{reference_text}\n\nTask:\n- Translate each `text` into concise scholarly English.\n- Keep placeholders and damage markers such as `<gap>`, `[ ... ]`, `/` intact when possible.\n- Keep proper names transliterated.\n- Translate in the same sequence as the input items.\n- Return ONLY JSON with this exact schema:\n {\n \"translations\": [\n {\"line_index\": 0, \"translation\": \"...\"}\n ]\n }\n- Include all line_index values exactly once.\n- Keep output order identical to input order.\n\nInput JSON:\n{payload_json}\nreference_text:
## Old Assyrian Formulaic Phrases (Patterns 1–12) — Prompt-ready English Descriptions\n\n> Placeholders: `{PN}` (personal name), `{Place}`, `{Amount}`, `{Goods}`, `{MonthName}`.\n\n### 1.1) Recipient-first address (letter opening)\n- **Transliteration template:** `a-na {RecipientPN} qí-bi-ma um-ma {SenderPN(s)}(-ma)`\n- **English meaning:** “To {RecipientPN} from {SenderPN(s)}:”\n\n### 1.2) Recipient-first address (letter opening)\n- **Transliteration template:** `a-na {RecipientPN} ù {RecipientPN} qí-bi-ma um-ma {SenderPN(s)}(-ma)`\n- **English meaning:** “To {RecipientPN} and {RecipientPN} from {SenderPN(s)}:”\n\n### 2.1) Sender-first address (variant letter opening)\n- **Transliteration template:** `um-ma {SenderPN}(-ma) a-na {RecipientPN} qí-bi-ma`\n- **English meaning:** “From {SenderPN} to {RecipientPN}:”\n\n### 2.2) Sender-first address (variant letter opening)\n- **Transliteration template:** `um-ma {SenderPN}-ma a-na {RecipientPN} {RecipientPN} ú {RecipientPN} qí-bi₄-ma a-na {RecipientPN} qí-bi₄-ma`\n- **English meaning:** \"From {SenderPN} to {RecipientPN}, {RecipientPN} and {RecipientPN}; specifically to {RecipientPN}:\"\n\n### 3) Well-being / peace formula\n- **Transliteration template:** `a-na-kam šu-ul-mu-um i-ba-ší`\n- **English meaning:** “All is well here / Peace reigns here.”\n\n### 4) Seal notation (legal label)\n- **Transliteration template:** `KIŠIB {PN} DUMU {FatherPN}`\n- **English meaning:** “Seal of {PN}, son of {FatherPN}.”\n\n### 5) Witness list / “before (the eyes of)”\n- **Transliteration template:** `IGI {Witness1PN} … IGI {WitnessNPN}`\n- **English meaning:** “In the presence of {Witness...} / Witnessed by {Witness...}.”\n\n### 6) kārum (trade colony) procedural/legal clause\n- **Transliteration template:** `a-na a-wa-tim a-ni-a-tim kà-ru-um {Place} i-dí-ni-a-tí(-ma)`\n- **English meaning:** “Regarding these proceedings/matters, the kārum of {Place} granted/issued (to us) ...”\n\n### 7) Oath/testimony “before Aššur’s dagger”\n- **Transliteration template:** `IGI GÍR ša a-šùr ší-bu-tí-ni ni-dí-in`\n- **English meaning:** “We gave our testimony (swore) before Aššur’s dagger.”\n\n### 8) Debt obligation formula (“X owes Y”)\n- **Transliteration template:** `{Amount} KÙ.BABBAR i-ṣé-er {CreditorPN} ... {DebtorPN} i-šu-ú`\n- **English meaning:** “{DebtorPN} owes {Amount} of silver to {CreditorPN}.”\n\n### 9) Receipt / acknowledgement (“has received”)\n- **Transliteration template:** `{Goods/Amount} ... {ReceiverPN} il₅-qé`\n- **English meaning:** “{ReceiverPN} has received {Goods/Amount}.”\n\n### 10) Return/repayment clause\n- **Transliteration template:** `i-tù-ra-ma {Amount/Goods} ...`\n- **English meaning:** “And he returned (it) / and he repaid (...).”\n\n### 11) Dating clause: week + month + eponym (limmu)\n- **Transliteration template:** `iš-tù ḫa-mu-uš-tim ... ITU.KAM {MonthName} li-mu-um {EponymPN} DUMU {FatherPN}`\n- **English meaning:** “Reckoned from the week of ...; month {MonthName}; eponymy (limmu-year) of {EponymPN} son of {FatherPN}.”\n\n### 12) Interest rate clause (per mina per month)\n- **Transliteration template:** `a-na 1 ma-na-im {X} GÍN.TA i-na ITU.KAM ṣí-ib-tám ú-ṣú-bu`\n- **English meaning:** “They will add interest at {X} shekels per 1 mina per month.”\nI matched IDs between the main corpus and published_texts.csv and obtained unmatched transliteration records.
Before translating the unmatched published corpus, I split each long transliteration into shorter chunk-sized units with a chi-square-based chunker.
\nMy working hypothesis was that physical row and edge breaks on tablets often behave like weak punctuation. They do not correspond perfectly to modern sentence boundaries, but they frequently coincide with discourse shifts such as a new legal formula, witness clause, dating clause, or procedural statement. I therefore used the already row-segmented training corpus as weak supervision for where a long unsegmented span might also be safely split.
\nConcretely, for each adjacent token pair X || Y, I counted how often the pair stayed within one training row (no_split) and how often a row boundary fell between them (split). Pairs whose boundary rate was unusually high relative to the corpus-wide average received a positive boundary score; pairs that usually stayed together received a negative one. When chunking the target corpus, dynamic programming then searched for cuts that balanced these learned boundary preferences against a target length (min_chunk=16, max_chunk=32, hard_max=47).
A concrete example is the recurrent legal phrase a-na a-wa-tim a-ni-a-tim .... The pair a-wa-tim || a-ni-a-tim was learned as a relatively strong candidate boundary:
| boundary candidate | \nsplit_count | \nno_split_count | \npair_boundary_rate | \nglobal_boundary_rate | \nsigned_sqrt_chi_square | \nchunk_boundary_score | \n
|---|---|---|---|---|---|---|
a-wa-tim || a-ni-a-tim | \n33 | \n226 | \n0.1274 | \n0.0410 | \n7.0189 | \n2.0424 | \n
Even though this pair also occurred many times inside a row, its boundary rate was still about three times higher than the corpus-wide average. In other words, the chunker learned that a break here was substantially more plausible than for an average token transition, so when the surrounding length constraints also fit, it preferred that cut.
\nExample:
\nscope total length: 46 tokens\n\nchunk 1 (tokens 1-25):\nILLAT šu-nu-nu i-na za-al-pá ni-kà-sí i-sí-ú-ma 7 ma-na 3 GÍN KÙ.BABBAR ni-pí-il-tí <gap> a-šu-mì <gap> ša <gap> ku-<gap> ILLAT <gap> DUMU <gap> ú-ša-<gap> a-na a-wa-tim\n\nchunk 2 (tokens 26-46):\na-ni-a-tim kà-ru-um kà-ni-iš i-dí-ni-a-tí-ma IGI GÍR ša a-šur ṭup-pá-am ni-dí-in IGI dan-a-šur DUMU ša-lim-a-ḫi-im IGI pí-lá-aḫ-a-šur bi-lu-um IGI šu-IŠTAR DUMU da-da-nim\n\nchosen boundary:\na-wa-tim || a-ni-a-tim\nsigned_sqrt_chi_square = 7.01886377\nchunk_boundary_score = 2.04236877\nI used Gemini 3 Flash Preview to translate the unmatched published transliteration records. The prompt here drifted slightly from the one used in the earlier stage, but not for any especially principled reason. In hindsight these two stages should have shared a single cleaned-up prompt template; instead, under time pressure, I ended up with duplicated prompt variants and lower-quality code than I would have liked.
\nPrompt (formulaic_phrase_section is the same as the previously mentioned Old Assyrian Formulaic Phrases (Patterns 1–12) markdown):
return (\n \"# Task\\n\"\n \"Translate the current Old Assyrian / Akkadian transliteration line into English.\\n\"\n \"Use the surrounding lines only as context. Do not translate the previous or following lines.\\n\\n\"\n \"## Output Format\\n\"\n 'Return JSON only in the form `{\"translation\": \"...\"}`.\\n\\n'\n \"## Rules\\n\"\n \"- No commentary, notes, markdown, or transliteration in the answer.\\n\"\n \"- Preserve uncertainty and damage tersely when the source is fragmentary.\\n\"\n \"- Keep names transliterated if they are uncertain.\\n\"\n \"- Translate measures and numbers naturally when possible.\\n\"\n \"- If the line is too fragmentary, still give the best partial English translation.\\n\\n\"\n f\"{formulaic_phrase_section}\\n\\n\"\n \"## Record\\n\"\n f\"- **Text ID:** {text_id}\\n\"\n f\"- **Designation:** {designation}\\n\\n\"\n \"## Context From The Same Text\\n\"\n \"### Previous Lines\\n\"\n f\"{previous_context or '- none'}\\n\\n\"\n \"### Current Line\\n\"\n f\"- **Label:** {label_text}\\n\"\n f\" **Section:** {section}\\n\"\n f\" **Transliteration:** {transliteration}\\n\\n\"\n \"### Following Lines\\n\"\n f\"{following_context or '- none'}\\n\"\n)\ntrain.csvI generated retranslated outputs with a model in the score-33.0 class, but used them only for length-based filtering: I excluded cases whose retranslated output length fell outside the +/-10% range. For the actual auxiliary data, I kept the transliteration and translation sentences from train.csv.
I used DPC - Larsen by MPWARE as an auxiliary corpus.
\nThe raw Gemini translations were only a starting point. Relative to train.csv, they differed systematically in the following respects:
<gap> and other damage markers were often handled inconsistently, either left as spurious English fragments or removed too aggressivelyI therefore treated data adaptation as an iterative patch-and-audit pipeline over two corpora: the published row-wise corpus and the longer auxiliary corpus.
\nI kept the normalization of recurrent seal headings, witness clauses, debt clauses, payment clauses, and kinship formulas so that the English matched the compact, formulaic style of train.csv.
Example:
\n10 shekels of refined silver are with Ikupi-Aššur son of Puzur-Ištar.\n-> 10 shekels of refined silver are owed by Ikupi-Aššur son of Puzur-Ištar.\n\nthey shall pay within one month.\n-> they will pay within one month.\n\nfine copper -> good copper\n\nIf he does not pay -> If he has not paid\nFor distinctive Old Assyrian terms, I generally did not rewrite the corpus text itself. Instead, I registered preferred equivalents in a glossary and applied them later in post-processing. My reasoning was that keeping these marked terms unchanged during translation reduced the risk of introducing odd lexical variation, while their distinctiveness also made them easy to replace deterministically afterward.
\nExample:
\nkārum -> colony\n\nmonth Bēltī-ekallim -> month 1\nI kept the punctuation alignment that split clause-like semicolon chains into sentence-like units, as well as the later conservative speech-framing cleanup used in the final branch.
\nExample:
\nAššur-rē'ī owes it; from the week of Akūtum\n-> Aššur-rē'ī owes it. From the week of Akūtum\nI removed <gap> from the translation when the transliteration had no corresponding <gap>. When the transliteration contained fewer <gap> markers than the translation, I also removed a final translation-side <gap> when it was attached to an adjacent alphanumeric character.
Example:
\n<gap>seize (him) and the silver and its interest<gap>\n-> seize (him) and the silver and its interest\n\nWith his deposits<gap>\n-> With his deposits\nI first tried broader normalization of personal and place names, including (1) simple replacements based on occurrence counts in the train.csv and (2) preferences for more scholarly spellings in the onomasticon.csv, especially long-vowel forms. In practice, however, I could not find a consistently effective methodology. I therefore retained only a small set of changes that seemed to improve the LB, added them to a glossary, and used that glossary in post-processing, even if the changes did not appear to follow a fully consistent principle.
Example:
\nAbua -> Abuwa\nAdudua -> Aduduwa\nAgua -> Aguwa\nAkua -> Akuwa\nSamāya -> Samaya\nAs a lottery-ticket heuristic, I compared train.csv with published_texts.csv, manually inspected the top 100 cases whose frequency in train.csv looked implausibly low, and added selected entries to a normalization dictionary. The top five personal-name examples were as follows:
i-ku-pì-a -> Ikupia\ni-dí-a-šur -> Idi-Aššur\nšál-ma-a-šur -> Šalma-Aššur\ni-dí-sú-en -> Idi-Suen\nšu-be-lim -> Šu-Belim\nAs another lottery-ticket heuristic, I replaced Thus you (said):, Thus (said) PN:, and related Thus-based expressions outside letter openings. The score began to drop partway through this line of experimentation, but I still kept this direction because the translation style of train.csv used Thus far less often.
Example:
\nThus you (said):\n-> You said:\n\nThus (said) Ennam-Suen:\n-> Ennam-Suen said:\nIn other words, after the manual auxiliary cleanup stage, the auxiliary corpus was stable, and the remaining adaptation work focused on bringing the published row-wise corpus closer to the auxiliary / train.csv punctuation and discourse style.
For the final byt5-large runs, I optimized two data streams in parallel: a main row-wise objective on the adapted published corpus (46,502 transliteration/translation pairs) and a secondary full-sequence objective on a smaller auxiliary corpus (1,358 examples used in training). The main corpus taught row-level alignment in the competition format, while the auxiliary corpus helped preserve longer-range fluency and formulaic phrasing.
For offline model selection, I used the script's default random approximate 80/10/10 split on the final 46,502-example main row-wise corpus, which produced 37,202 training rows, 4,650 validation rows, and 4,650 test rows in the seed-42 run.
\nBefore training, each row in the main corpus was converted into a context-augmented input of the form:
\n[LOOKBACK_ONLY] <previous tokens> [SOURCE] <current row> [LOOKAHEAD_ONLY] <next tokens>\nIn the final setting, each row received up to four tokens of left context and up to four tokens of right context, drawn from earlier and later rows within the same tablet/document. The target remained only the English translation of the current row, so the surrounding text served only as additional source-side evidence. Thus, this was row-wise contextual training, not the chi-square chunking used later at inference time.
\ncdli_id:\nbce88917-e85f-02a8-609f-3f2d37c0d85d.tablet.4\n\ntransliteration:\nša 2 ma-na KÙ.BABBAR ša i-ṣé-er bu-za-zu-ma be-lá-num-ma i-šu-ú 4 ṭup-pè-e a-ni-ú-tim a ma-lá na-áš-pár-tim ša be-lá-nim DUMU šu-a-šur\n\nmodel_input:\n[LOOKBACK_ONLY] šu-a-šur i-šu-ú 1 ṭup-pu-um [SOURCE] ša 2 ma-na KÙ.BABBAR ša i-ṣé-er bu-za-zu-ma be-lá-num-ma i-šu-ú 4 ṭup-pè-e a-ni-ú-tim a ma-lá na-áš-pár-tim ša be-lá-nim DUMU šu-a-šur [LOOKAHEAD_ONLY] ba-lá-ṭí-šu-nu-ma a-na pu-šu-ke-en6 DUMU\n\ntarget:\nof 2 minas of silver which Buzazu owes to Belanum; these 4 tablets according to the proxy of Belanum, son of Šu-Aššur.\nThe auxiliary examples were longer, style-cleaned translations derived from cleaned train-side material and Larsen-related texts. They were not row-wise context windows; instead, they exposed the model to longer document-scale source/target mappings.
\nImplementation-wise, the main loss was computed on the row-wise contextual batches above, and during the same optimizer step I also sampled one auxiliary batch and added 0.3 * aux_loss before the parameter update. For the final setting, the auxiliary loader used batch_size=1, max_source_length=1536, and max_target_length=1536. One overlength auxiliary example was filtered out before training, leaving 1,358 auxiliary examples in the final training set.
The main training hyperparameters for the final byt5-large models were:
google/byt5-large4322e-40.014484484651.0I trained the same configuration with different random seeds (42, 60, 28, and 52) and selected the best checkpoint from each run based on validation loss.
At inference time, I first normalized the source-side division-number notation, then split each test transliteration with the same chi-square chunker described above. I used min_chunk=16 and max_chunk=32, so the model saw chunk-sized units rather than entire long test rows at once.
For each chunk, I then constructed a context-augmented input in the same format used during training:
\n[LOOKBACK_ONLY] <previous tokens> [SOURCE] <current chunk> [LOOKAHEAD_ONLY] <next tokens>\nIn the final setting, I used lookback_tokens=4 and lookahead_tokens=4. If this context was too short in characters, I allowed the neighboring window to expand slightly until it reached a small minimum textual mass. This kept the prediction target local to one chunk while still exposing nearby formulae, names, and discourse cues.
I ensembled four byt5-large checkpoints at inference time: two runs trained on the final row-wise adapted corpus produced by the last data-adaptation pass, and two runs trained on the immediately preceding row-wise adapted version from one step earlier in that same pipeline. Their mixture weights were not uniform; the two checkpoints from the final adapted corpus contributed 0.2 each, and the two checkpoints from the preceding adapted corpus contributed 0.3 each. Generation used max_source_length=448, and the decoding budget for the target side was scaled adaptively between 192 and 448 new tokens depending on the encoded source length.
For each chunk and each checkpoint, I generated a heterogeneous candidate pool:
\n4 standard beam candidates4 diverse-beam candidates6 nucleus-sampling candidates (2 samples each at temperatures 0.55, 0.75, and 0.95)This yielded up to 56 raw candidates per chunk before deduplication across the four checkpoints.
I then selected the final chunk translation with minimum Bayes risk decoding over the pooled candidate set. The MBR gain combined sentence-level chrF++, sentence BLEU, token-level Jaccard overlap, and a mild length bonus centered on the median candidate length. I weighted chrF++ most heavily (6.0), followed by BLEU (2.0), while Jaccard and the length bonus each received weight 1.0. Within each checkpoint, the evidence mass was distributed uniformly across its candidates, and the total contribution of each checkpoint was then scaled by the mixture weights above. When several hypotheses remained close, ranking was further stabilized by support count and generation score.
After MBR selection, I concatenated the winning chunk translations in their original order to recover one English translation per test row. I then applied a small deterministic cleanup and post-editing pipeline: normalization of transliteration-style characters and quotation marks, removal of residual bracketed uncertainty markers, source-aware handling of <gap>, restoration of competition-specific number and glossary conventions, and two lightweight logistic-regression post-editors to nudge the output toward the training-set style.
| Method | \nCorpus | \nChunker | \nPrivate LB | \nPublic LB | \n
|---|---|---|---|---|
| Single ByT5-Large(seed 52) | \nFinal adapted corpus | \nchi-square | \n38.4389 | \n37.7799 | \n
| Single ByT5-Large (seed 28) | \nFinal adapted corpus | \nchi-square | \n37.7196 | \n36.9437 | \n
| Single ByT5-Large (seed 42) | \nThe preceding adapted corpus | \nchi-square | \n38.1309 | \n37.6288 | \n
| Single ByT5-Large (seed 60) | \nThe preceding adapted corpus | \nchi-square | \n38.3819 | \n37.1679 | \n
| Ensemble (MBR) | \nN/A | \nchi-square | \n38.5514 | \n37.4818 | \n
| Method | \nCorpus | \nChunker | \nPrivate LB | \nPublic LB | \n
|---|---|---|---|---|
| Single ByT5-Large(seed 52) | \nFinal adapted corpus | \nchi-square | \n38.4389 | \n37.7799 | \n
| Single ByT5-Large (seed 52) -PP | \nFinal adapted corpus | \nchi-square | \n38.4362 | \n37.6650 | \n
| Single ByT5-Large (seed 52) -PP-GR | \nFinal adapted corpus | \nchi-square | \n38.4443 | \n37.6819 | \n
| Method | \nCorpus | \nChunker | \nPrivate LB | \nPublic LB | \n
|---|---|---|---|---|
| Ensemble (MBR) | \nN/A | \nchi-square | \n38.5514 | \n37.4818 | \n
| Ensemble (MBR) -PP | \nN/A | \nchi-square | \n38.5614 | \n37.4821 | \n
| Ensemble (MBR) -PP-GR | \nN/A | \nchi-square | \n38.5686 | \n37.3774 | \n
| Method | \nCorpus | \nChunker | \nPrivate LB | \nPublic LB | \n
|---|---|---|---|---|
| Single ByT5-Large (seed 52) | \nFinal adapted corpus | \nchi-square | \n38.4389 | \n37.7799 | \n
| Single ByT5-Base (seed 52, epochs 5, learning rate 3e-4) | \nFinal adapted corpus | \nchi-square | \n38.0925 | \n37.0492 | \n
| Single ByT5-Base (seed 52, epochs 5, learning rate 2e-4) | \nFinal adapted corpus | \nchi-square | \n37.9296 | \n37.0533 | \n
| Method | \nCorpus | \nChunker | \nPrivate LB | \nPublic LB | \n
|---|---|---|---|---|
| Single ByT5-Large (seed 52) | \nPre-adaptation corpus | \nchi-square | \n37.5068 | \n36.7824 | \n
| Single ByT5-Base (seed 52, epochs 5, learning rate 3e-4) | \nPre-adaptation corpus | \nchi-square | \n37.0518 | \n36.5225 | \n
| Single ByT5-Base (seed 52, epochs 5, learning rate 2e-4) | \nPre-adaptation corpus | \nchi-square | \n37.1781 | \n36.9766 | \n
| Method | \nCorpus | \nChunker | \nPrivate LB | \nPublic LB | \n
|---|---|---|---|---|
| Single ByT5-Large (seed 52) | \nPre-adaptation corpus | \nchi-square | \n37.5068 | \n36.7824 | \n
| Single ByT5-Large (seed 52) | \nFinal adapted corpus | \nchi-square | \n38.4389 | \n37.7799 | \n
| Method | \nCorpus | \nChunker | \nPrivate LB | \nPublic LB | \n
|---|---|---|---|---|
| Single ByT5-Large (seed 52) | \nFinal adapted corpus | \nchi-square | \n38.4389 | \n37.7799 | \n
| Single ByT5-Large (seed 52) | \nFinal adapted corpus | \n[SOURCE] only (max_source_length/max_new_tokens 768 tokens) | \n38.2861 | \n36.6346 | \n
| Single ByT5-Large (seed 52) | \nFinal adapted corpus | \nfixed 16-token chunks | \n38.0926 | \n36.9625 | \n
Thanks again to the hosts for this competition. I learned a lot from this problem, especially about corpus construction and style alignment. I am also grateful to the community for sharing many interesting ideas throughout the competition.
", "messages": [ { "id": 3427861, "postDate": "2026-03-24T16:21:20.233Z", "content": "First, I would like to thank the Kaggle hosts for organizing this competition and for providing such an interesting problem.
\nMy final solution was a contextual ByT5-large system with chunked ensemble inference, but the biggest gains came from data work rather than model changes.
\nOverall, the system was built around three ideas: better corpus construction, stronger style adaptation, and more careful chunk-level inference.
\nApproximate contribution to the final score (from ablations)\n\nPrivate LB:\nData adaptation [########################......] +0.93\nChunking [#########.....................] +0.15 to +0.35\nByT5-Large [#########.....................] +0.35\nEnsemble + MBR [###...........................] +0.11\nPost-edit / GR [..............................] ~0.00\n\nPublic LB:\nData adaptation [##########################....] +1.00\nChunking [##############################] +0.82 to +1.15\nByT5-Large [###################...........] +0.73\nEnsemble + MBR [..............................] no consistent gain\nPost-edit / GR [###...........................] +0.10\nI prepared two corpora for training:
\nI found several useful APIs.
\nUUID feeder:\nhttps://oare.byu.edu/api/v2/search?characters=&textTitle=AKT&page=1&rows=100&mode=respectAllBoundaries
Tablet feeder:\nhttps://oare.byu.edu/api/v2/text_epigraphies/text/${uuid}?forceAllowAdminView=false
CDLI provides an easy way to download all Old Assyrian tablets:
\ncurl -L 'https://cdli.earth/search?period=old+ass&limit=10000&aspect=inscriptions&format=atf' -o old_assyrian.atf\nAfter downloading the data from CDLI, I converted it into the OARE format.
\nI merged the two datasets above. After that, I grouped the rows of each tablet into 4–6 chunks; if fewer than two lines remained, I merged them into the previous chunk.
\nAn example after merging:
\n{\"text_id\": \"055b4918-e8aa-4442-99b6-cc4d6f33adf4\", \"designation\": \"AKT 1 30\", \"header_raw\": \"&055b4918-e8aa-4442-99b6-cc4d6f33adf4 = AKT 1 30\", \"atf\": {\"lang\": \"akk\"}, \"sections\": [\"tablet\", \"obverse\", \"reverse\"], \"lines\": [{\"label\": [\"1\", \"2\", \"3\", \"4\"], \"section\": \"obverse\", \"text\": \"4 i-lá-tim ku nu ki ša a-mur-IŠTAR ma-aḫ-ri-ni a-du-ú ù i-ku-pì-a\", \"translation\": null}, {\"label\": [\"5\", \"6\", \"7\", \"8\"], \"section\": \"obverse\", \"text\": \"iš ru mu ma i-na 3 i-lá-tim 3 GÚ 7+0.5 ma-na URUDU is ni qú\", \"translation\": null}, {\"label\": [\"9\", \"10\", \"11\", \"12\"], \"section\": \"reverse\", \"text\": \"a-ḫa-ma i-na 1 i-li-im 20+6 ma-na ta <gap> mu <gap> ù ša a-ba-ri-im\", \"translation\": null}, {\"label\": [\"13\", \"14\", \"15\"], \"section\": \"reverse\", \"text\": \"is ni qú ú IGI a-šur-iš-tí-kál IGI en-nam-a-a\", \"translation\": null}], \"annotations\": []}\nI used Gemini 3 Flash Preview to fill in missing translations.
\nPrompt example:
\nUse the following reference notes while translating Old Assyrian transliteration lines:\n\n{reference_text}\n\nTask:\n- Translate each `text` into concise scholarly English.\n- Keep placeholders and damage markers such as `<gap>`, `[ ... ]`, `/` intact when possible.\n- Keep proper names transliterated.\n- Translate in the same sequence as the input items.\n- Return ONLY JSON with this exact schema:\n {\n \"translations\": [\n {\"line_index\": 0, \"translation\": \"...\"}\n ]\n }\n- Include all line_index values exactly once.\n- Keep output order identical to input order.\n\nInput JSON:\n{payload_json}\nreference_text:
## Old Assyrian Formulaic Phrases (Patterns 1–12) — Prompt-ready English Descriptions\n\n> Placeholders: `{PN}` (personal name), `{Place}`, `{Amount}`, `{Goods}`, `{MonthName}`.\n\n### 1.1) Recipient-first address (letter opening)\n- **Transliteration template:** `a-na {RecipientPN} qí-bi-ma um-ma {SenderPN(s)}(-ma)`\n- **English meaning:** “To {RecipientPN} from {SenderPN(s)}:”\n\n### 1.2) Recipient-first address (letter opening)\n- **Transliteration template:** `a-na {RecipientPN} ù {RecipientPN} qí-bi-ma um-ma {SenderPN(s)}(-ma)`\n- **English meaning:** “To {RecipientPN} and {RecipientPN} from {SenderPN(s)}:”\n\n### 2.1) Sender-first address (variant letter opening)\n- **Transliteration template:** `um-ma {SenderPN}(-ma) a-na {RecipientPN} qí-bi-ma`\n- **English meaning:** “From {SenderPN} to {RecipientPN}:”\n\n### 2.2) Sender-first address (variant letter opening)\n- **Transliteration template:** `um-ma {SenderPN}-ma a-na {RecipientPN} {RecipientPN} ú {RecipientPN} qí-bi₄-ma a-na {RecipientPN} qí-bi₄-ma`\n- **English meaning:** \"From {SenderPN} to {RecipientPN}, {RecipientPN} and {RecipientPN}; specifically to {RecipientPN}:\"\n\n### 3) Well-being / peace formula\n- **Transliteration template:** `a-na-kam šu-ul-mu-um i-ba-ší`\n- **English meaning:** “All is well here / Peace reigns here.”\n\n### 4) Seal notation (legal label)\n- **Transliteration template:** `KIŠIB {PN} DUMU {FatherPN}`\n- **English meaning:** “Seal of {PN}, son of {FatherPN}.”\n\n### 5) Witness list / “before (the eyes of)”\n- **Transliteration template:** `IGI {Witness1PN} … IGI {WitnessNPN}`\n- **English meaning:** “In the presence of {Witness...} / Witnessed by {Witness...}.”\n\n### 6) kārum (trade colony) procedural/legal clause\n- **Transliteration template:** `a-na a-wa-tim a-ni-a-tim kà-ru-um {Place} i-dí-ni-a-tí(-ma)`\n- **English meaning:** “Regarding these proceedings/matters, the kārum of {Place} granted/issued (to us) ...”\n\n### 7) Oath/testimony “before Aššur’s dagger”\n- **Transliteration template:** `IGI GÍR ša a-šùr ší-bu-tí-ni ni-dí-in`\n- **English meaning:** “We gave our testimony (swore) before Aššur’s dagger.”\n\n### 8) Debt obligation formula (“X owes Y”)\n- **Transliteration template:** `{Amount} KÙ.BABBAR i-ṣé-er {CreditorPN} ... {DebtorPN} i-šu-ú`\n- **English meaning:** “{DebtorPN} owes {Amount} of silver to {CreditorPN}.”\n\n### 9) Receipt / acknowledgement (“has received”)\n- **Transliteration template:** `{Goods/Amount} ... {ReceiverPN} il₅-qé`\n- **English meaning:** “{ReceiverPN} has received {Goods/Amount}.”\n\n### 10) Return/repayment clause\n- **Transliteration template:** `i-tù-ra-ma {Amount/Goods} ...`\n- **English meaning:** “And he returned (it) / and he repaid (...).”\n\n### 11) Dating clause: week + month + eponym (limmu)\n- **Transliteration template:** `iš-tù ḫa-mu-uš-tim ... ITU.KAM {MonthName} li-mu-um {EponymPN} DUMU {FatherPN}`\n- **English meaning:** “Reckoned from the week of ...; month {MonthName}; eponymy (limmu-year) of {EponymPN} son of {FatherPN}.”\n\n### 12) Interest rate clause (per mina per month)\n- **Transliteration template:** `a-na 1 ma-na-im {X} GÍN.TA i-na ITU.KAM ṣí-ib-tám ú-ṣú-bu`\n- **English meaning:** “They will add interest at {X} shekels per 1 mina per month.”\nI matched IDs between the main corpus and published_texts.csv and obtained unmatched transliteration records.
Before translating the unmatched published corpus, I split each long transliteration into shorter chunk-sized units with a chi-square-based chunker.
\nMy working hypothesis was that physical row and edge breaks on tablets often behave like weak punctuation. They do not correspond perfectly to modern sentence boundaries, but they frequently coincide with discourse shifts such as a new legal formula, witness clause, dating clause, or procedural statement. I therefore used the already row-segmented training corpus as weak supervision for where a long unsegmented span might also be safely split.
\nConcretely, for each adjacent token pair X || Y, I counted how often the pair stayed within one training row (no_split) and how often a row boundary fell between them (split). Pairs whose boundary rate was unusually high relative to the corpus-wide average received a positive boundary score; pairs that usually stayed together received a negative one. When chunking the target corpus, dynamic programming then searched for cuts that balanced these learned boundary preferences against a target length (min_chunk=16, max_chunk=32, hard_max=47).
A concrete example is the recurrent legal phrase a-na a-wa-tim a-ni-a-tim .... The pair a-wa-tim || a-ni-a-tim was learned as a relatively strong candidate boundary:
| boundary candidate | \nsplit_count | \nno_split_count | \npair_boundary_rate | \nglobal_boundary_rate | \nsigned_sqrt_chi_square | \nchunk_boundary_score | \n
|---|---|---|---|---|---|---|
a-wa-tim || a-ni-a-tim | \n33 | \n226 | \n0.1274 | \n0.0410 | \n7.0189 | \n2.0424 | \n
Even though this pair also occurred many times inside a row, its boundary rate was still about three times higher than the corpus-wide average. In other words, the chunker learned that a break here was substantially more plausible than for an average token transition, so when the surrounding length constraints also fit, it preferred that cut.
\nExample:
\nscope total length: 46 tokens\n\nchunk 1 (tokens 1-25):\nILLAT šu-nu-nu i-na za-al-pá ni-kà-sí i-sí-ú-ma 7 ma-na 3 GÍN KÙ.BABBAR ni-pí-il-tí <gap> a-šu-mì <gap> ša <gap> ku-<gap> ILLAT <gap> DUMU <gap> ú-ša-<gap> a-na a-wa-tim\n\nchunk 2 (tokens 26-46):\na-ni-a-tim kà-ru-um kà-ni-iš i-dí-ni-a-tí-ma IGI GÍR ša a-šur ṭup-pá-am ni-dí-in IGI dan-a-šur DUMU ša-lim-a-ḫi-im IGI pí-lá-aḫ-a-šur bi-lu-um IGI šu-IŠTAR DUMU da-da-nim\n\nchosen boundary:\na-wa-tim || a-ni-a-tim\nsigned_sqrt_chi_square = 7.01886377\nchunk_boundary_score = 2.04236877\nI used Gemini 3 Flash Preview to translate the unmatched published transliteration records. The prompt here drifted slightly from the one used in the earlier stage, but not for any especially principled reason. In hindsight these two stages should have shared a single cleaned-up prompt template; instead, under time pressure, I ended up with duplicated prompt variants and lower-quality code than I would have liked.
\nPrompt (formulaic_phrase_section is the same as the previously mentioned Old Assyrian Formulaic Phrases (Patterns 1–12) markdown):
return (\n \"# Task\\n\"\n \"Translate the current Old Assyrian / Akkadian transliteration line into English.\\n\"\n \"Use the surrounding lines only as context. Do not translate the previous or following lines.\\n\\n\"\n \"## Output Format\\n\"\n 'Return JSON only in the form `{\"translation\": \"...\"}`.\\n\\n'\n \"## Rules\\n\"\n \"- No commentary, notes, markdown, or transliteration in the answer.\\n\"\n \"- Preserve uncertainty and damage tersely when the source is fragmentary.\\n\"\n \"- Keep names transliterated if they are uncertain.\\n\"\n \"- Translate measures and numbers naturally when possible.\\n\"\n \"- If the line is too fragmentary, still give the best partial English translation.\\n\\n\"\n f\"{formulaic_phrase_section}\\n\\n\"\n \"## Record\\n\"\n f\"- **Text ID:** {text_id}\\n\"\n f\"- **Designation:** {designation}\\n\\n\"\n \"## Context From The Same Text\\n\"\n \"### Previous Lines\\n\"\n f\"{previous_context or '- none'}\\n\\n\"\n \"### Current Line\\n\"\n f\"- **Label:** {label_text}\\n\"\n f\" **Section:** {section}\\n\"\n f\" **Transliteration:** {transliteration}\\n\\n\"\n \"### Following Lines\\n\"\n f\"{following_context or '- none'}\\n\"\n)\ntrain.csvI generated retranslated outputs with a model in the score-33.0 class, but used them only for length-based filtering: I excluded cases whose retranslated output length fell outside the +/-10% range. For the actual auxiliary data, I kept the transliteration and translation sentences from train.csv.
I used DPC - Larsen by MPWARE as an auxiliary corpus.
\nThe raw Gemini translations were only a starting point. Relative to train.csv, they differed systematically in the following respects:
<gap> and other damage markers were often handled inconsistently, either left as spurious English fragments or removed too aggressivelyI therefore treated data adaptation as an iterative patch-and-audit pipeline over two corpora: the published row-wise corpus and the longer auxiliary corpus.
\nI kept the normalization of recurrent seal headings, witness clauses, debt clauses, payment clauses, and kinship formulas so that the English matched the compact, formulaic style of train.csv.
Example:
\n10 shekels of refined silver are with Ikupi-Aššur son of Puzur-Ištar.\n-> 10 shekels of refined silver are owed by Ikupi-Aššur son of Puzur-Ištar.\n\nthey shall pay within one month.\n-> they will pay within one month.\n\nfine copper -> good copper\n\nIf he does not pay -> If he has not paid\nFor distinctive Old Assyrian terms, I generally did not rewrite the corpus text itself. Instead, I registered preferred equivalents in a glossary and applied them later in post-processing. My reasoning was that keeping these marked terms unchanged during translation reduced the risk of introducing odd lexical variation, while their distinctiveness also made them easy to replace deterministically afterward.
\nExample:
\nkārum -> colony\n\nmonth Bēltī-ekallim -> month 1\nI kept the punctuation alignment that split clause-like semicolon chains into sentence-like units, as well as the later conservative speech-framing cleanup used in the final branch.
\nExample:
\nAššur-rē'ī owes it; from the week of Akūtum\n-> Aššur-rē'ī owes it. From the week of Akūtum\nI removed <gap> from the translation when the transliteration had no corresponding <gap>. When the transliteration contained fewer <gap> markers than the translation, I also removed a final translation-side <gap> when it was attached to an adjacent alphanumeric character.
Example:
\n<gap>seize (him) and the silver and its interest<gap>\n-> seize (him) and the silver and its interest\n\nWith his deposits<gap>\n-> With his deposits\nI first tried broader normalization of personal and place names, including (1) simple replacements based on occurrence counts in the train.csv and (2) preferences for more scholarly spellings in the onomasticon.csv, especially long-vowel forms. In practice, however, I could not find a consistently effective methodology. I therefore retained only a small set of changes that seemed to improve the LB, added them to a glossary, and used that glossary in post-processing, even if the changes did not appear to follow a fully consistent principle.
Example:
\nAbua -> Abuwa\nAdudua -> Aduduwa\nAgua -> Aguwa\nAkua -> Akuwa\nSamāya -> Samaya\nAs a lottery-ticket heuristic, I compared train.csv with published_texts.csv, manually inspected the top 100 cases whose frequency in train.csv looked implausibly low, and added selected entries to a normalization dictionary. The top five personal-name examples were as follows:
i-ku-pì-a -> Ikupia\ni-dí-a-šur -> Idi-Aššur\nšál-ma-a-šur -> Šalma-Aššur\ni-dí-sú-en -> Idi-Suen\nšu-be-lim -> Šu-Belim\nAs another lottery-ticket heuristic, I replaced Thus you (said):, Thus (said) PN:, and related Thus-based expressions outside letter openings. The score began to drop partway through this line of experimentation, but I still kept this direction because the translation style of train.csv used Thus far less often.
Example:
\nThus you (said):\n-> You said:\n\nThus (said) Ennam-Suen:\n-> Ennam-Suen said:\nIn other words, after the manual auxiliary cleanup stage, the auxiliary corpus was stable, and the remaining adaptation work focused on bringing the published row-wise corpus closer to the auxiliary / train.csv punctuation and discourse style.
For the final byt5-large runs, I optimized two data streams in parallel: a main row-wise objective on the adapted published corpus (46,502 transliteration/translation pairs) and a secondary full-sequence objective on a smaller auxiliary corpus (1,358 examples used in training). The main corpus taught row-level alignment in the competition format, while the auxiliary corpus helped preserve longer-range fluency and formulaic phrasing.
For offline model selection, I used the script's default random approximate 80/10/10 split on the final 46,502-example main row-wise corpus, which produced 37,202 training rows, 4,650 validation rows, and 4,650 test rows in the seed-42 run.
\nBefore training, each row in the main corpus was converted into a context-augmented input of the form:
\n[LOOKBACK_ONLY] <previous tokens> [SOURCE] <current row> [LOOKAHEAD_ONLY] <next tokens>\nIn the final setting, each row received up to four tokens of left context and up to four tokens of right context, drawn from earlier and later rows within the same tablet/document. The target remained only the English translation of the current row, so the surrounding text served only as additional source-side evidence. Thus, this was row-wise contextual training, not the chi-square chunking used later at inference time.
\ncdli_id:\nbce88917-e85f-02a8-609f-3f2d37c0d85d.tablet.4\n\ntransliteration:\nša 2 ma-na KÙ.BABBAR ša i-ṣé-er bu-za-zu-ma be-lá-num-ma i-šu-ú 4 ṭup-pè-e a-ni-ú-tim a ma-lá na-áš-pár-tim ša be-lá-nim DUMU šu-a-šur\n\nmodel_input:\n[LOOKBACK_ONLY] šu-a-šur i-šu-ú 1 ṭup-pu-um [SOURCE] ša 2 ma-na KÙ.BABBAR ša i-ṣé-er bu-za-zu-ma be-lá-num-ma i-šu-ú 4 ṭup-pè-e a-ni-ú-tim a ma-lá na-áš-pár-tim ša be-lá-nim DUMU šu-a-šur [LOOKAHEAD_ONLY] ba-lá-ṭí-šu-nu-ma a-na pu-šu-ke-en6 DUMU\n\ntarget:\nof 2 minas of silver which Buzazu owes to Belanum; these 4 tablets according to the proxy of Belanum, son of Šu-Aššur.\nThe auxiliary examples were longer, style-cleaned translations derived from cleaned train-side material and Larsen-related texts. They were not row-wise context windows; instead, they exposed the model to longer document-scale source/target mappings.
\nImplementation-wise, the main loss was computed on the row-wise contextual batches above, and during the same optimizer step I also sampled one auxiliary batch and added 0.3 * aux_loss before the parameter update. For the final setting, the auxiliary loader used batch_size=1, max_source_length=1536, and max_target_length=1536. One overlength auxiliary example was filtered out before training, leaving 1,358 auxiliary examples in the final training set.
The main training hyperparameters for the final byt5-large models were:
google/byt5-large4322e-40.014484484651.0I trained the same configuration with different random seeds (42, 60, 28, and 52) and selected the best checkpoint from each run based on validation loss.
At inference time, I first normalized the source-side division-number notation, then split each test transliteration with the same chi-square chunker described above. I used min_chunk=16 and max_chunk=32, so the model saw chunk-sized units rather than entire long test rows at once.
For each chunk, I then constructed a context-augmented input in the same format used during training:
\n[LOOKBACK_ONLY] <previous tokens> [SOURCE] <current chunk> [LOOKAHEAD_ONLY] <next tokens>\nIn the final setting, I used lookback_tokens=4 and lookahead_tokens=4. If this context was too short in characters, I allowed the neighboring window to expand slightly until it reached a small minimum textual mass. This kept the prediction target local to one chunk while still exposing nearby formulae, names, and discourse cues.
I ensembled four byt5-large checkpoints at inference time: two runs trained on the final row-wise adapted corpus produced by the last data-adaptation pass, and two runs trained on the immediately preceding row-wise adapted version from one step earlier in that same pipeline. Their mixture weights were not uniform; the two checkpoints from the final adapted corpus contributed 0.2 each, and the two checkpoints from the preceding adapted corpus contributed 0.3 each. Generation used max_source_length=448, and the decoding budget for the target side was scaled adaptively between 192 and 448 new tokens depending on the encoded source length.
For each chunk and each checkpoint, I generated a heterogeneous candidate pool:
\n4 standard beam candidates4 diverse-beam candidates6 nucleus-sampling candidates (2 samples each at temperatures 0.55, 0.75, and 0.95)This yielded up to 56 raw candidates per chunk before deduplication across the four checkpoints.
I then selected the final chunk translation with minimum Bayes risk decoding over the pooled candidate set. The MBR gain combined sentence-level chrF++, sentence BLEU, token-level Jaccard overlap, and a mild length bonus centered on the median candidate length. I weighted chrF++ most heavily (6.0), followed by BLEU (2.0), while Jaccard and the length bonus each received weight 1.0. Within each checkpoint, the evidence mass was distributed uniformly across its candidates, and the total contribution of each checkpoint was then scaled by the mixture weights above. When several hypotheses remained close, ranking was further stabilized by support count and generation score.
After MBR selection, I concatenated the winning chunk translations in their original order to recover one English translation per test row. I then applied a small deterministic cleanup and post-editing pipeline: normalization of transliteration-style characters and quotation marks, removal of residual bracketed uncertainty markers, source-aware handling of <gap>, restoration of competition-specific number and glossary conventions, and two lightweight logistic-regression post-editors to nudge the output toward the training-set style.
| Method | \nCorpus | \nChunker | \nPrivate LB | \nPublic LB | \n
|---|---|---|---|---|
| Single ByT5-Large(seed 52) | \nFinal adapted corpus | \nchi-square | \n38.4389 | \n37.7799 | \n
| Single ByT5-Large (seed 28) | \nFinal adapted corpus | \nchi-square | \n37.7196 | \n36.9437 | \n
| Single ByT5-Large (seed 42) | \nThe preceding adapted corpus | \nchi-square | \n38.1309 | \n37.6288 | \n
| Single ByT5-Large (seed 60) | \nThe preceding adapted corpus | \nchi-square | \n38.3819 | \n37.1679 | \n
| Ensemble (MBR) | \nN/A | \nchi-square | \n38.5514 | \n37.4818 | \n
| Method | \nCorpus | \nChunker | \nPrivate LB | \nPublic LB | \n
|---|---|---|---|---|
| Single ByT5-Large(seed 52) | \nFinal adapted corpus | \nchi-square | \n38.4389 | \n37.7799 | \n
| Single ByT5-Large (seed 52) -PP | \nFinal adapted corpus | \nchi-square | \n38.4362 | \n37.6650 | \n
| Single ByT5-Large (seed 52) -PP-GR | \nFinal adapted corpus | \nchi-square | \n38.4443 | \n37.6819 | \n
| Method | \nCorpus | \nChunker | \nPrivate LB | \nPublic LB | \n
|---|---|---|---|---|
| Ensemble (MBR) | \nN/A | \nchi-square | \n38.5514 | \n37.4818 | \n
| Ensemble (MBR) -PP | \nN/A | \nchi-square | \n38.5614 | \n37.4821 | \n
| Ensemble (MBR) -PP-GR | \nN/A | \nchi-square | \n38.5686 | \n37.3774 | \n
| Method | \nCorpus | \nChunker | \nPrivate LB | \nPublic LB | \n
|---|---|---|---|---|
| Single ByT5-Large (seed 52) | \nFinal adapted corpus | \nchi-square | \n38.4389 | \n37.7799 | \n
| Single ByT5-Base (seed 52, epochs 5, learning rate 3e-4) | \nFinal adapted corpus | \nchi-square | \n38.0925 | \n37.0492 | \n
| Single ByT5-Base (seed 52, epochs 5, learning rate 2e-4) | \nFinal adapted corpus | \nchi-square | \n37.9296 | \n37.0533 | \n
| Method | \nCorpus | \nChunker | \nPrivate LB | \nPublic LB | \n
|---|---|---|---|---|
| Single ByT5-Large (seed 52) | \nPre-adaptation corpus | \nchi-square | \n37.5068 | \n36.7824 | \n
| Single ByT5-Base (seed 52, epochs 5, learning rate 3e-4) | \nPre-adaptation corpus | \nchi-square | \n37.0518 | \n36.5225 | \n
| Single ByT5-Base (seed 52, epochs 5, learning rate 2e-4) | \nPre-adaptation corpus | \nchi-square | \n37.1781 | \n36.9766 | \n
| Method | \nCorpus | \nChunker | \nPrivate LB | \nPublic LB | \n
|---|---|---|---|---|
| Single ByT5-Large (seed 52) | \nPre-adaptation corpus | \nchi-square | \n37.5068 | \n36.7824 | \n
| Single ByT5-Large (seed 52) | \nFinal adapted corpus | \nchi-square | \n38.4389 | \n37.7799 | \n
| Method | \nCorpus | \nChunker | \nPrivate LB | \nPublic LB | \n
|---|---|---|---|---|
| Single ByT5-Large (seed 52) | \nFinal adapted corpus | \nchi-square | \n38.4389 | \n37.7799 | \n
| Single ByT5-Large (seed 52) | \nFinal adapted corpus | \n[SOURCE] only (max_source_length/max_new_tokens 768 tokens) | \n38.2861 | \n36.6346 | \n
| Single ByT5-Large (seed 52) | \nFinal adapted corpus | \nfixed 16-token chunks | \n38.0926 | \n36.9625 | \n
Thanks again to the hosts for this competition. I learned a lot from this problem, especially about corpus construction and style alignment. I am also grateful to the community for sharing many interesting ideas throughout the competition.
", "rawMarkdown": "First, I would like to thank the Kaggle hosts for organizing this competition and for providing such an interesting problem.\n\n# Summary\n\nMy final solution was a contextual ByT5-large system with chunked ensemble inference, but the biggest gains came from data work rather than model changes.\n\n+ **Data**: built a larger published row-wise corpus from OARE/CDLI, added a smaller auxiliary full-sequence corpus, and filled missing translations with formula-aware Gemini prompting\n+ **Black-box Distillation**: used a stronger closed model to generate pseudo-labels, then trained a smaller open model on the resulting parallel data\n+ **Style adaptation**: repeatedly cleaned names, formulaic expressions, punctuation, and