I’m working on an English–Akkadian sentence alignment notebook aimed at creating higher-quality parallel training data for NMT in the context of the Deep Past Initiative challenge.
\nhttps://www.kaggle.com/code/aaronbornstein/eng-akk-low-resource-sent-alignment
\nThe goal is practical: get usable sentence-level alignments for a very low-resource, historically messy setting, where many standard tools don’t really apply.
\nWhy this is hard
\nAligning Akkadian cuneiform with English translations breaks a lot of assumptions that common alignment methods rely on:
Damaged source text: Tablets are often fragmentary, with lacunae ([…]) and missing context.
Morphological mismatch: Akkadian is a morphologically rich Semitic language. A single Akkadian word can correspond to an entire English phrase, and word order differs a lot from English.
Data scarcity: We’re nowhere near the scale of modern parallel corpora used for contemporary MT.
No pretrained embeddings: There’s no BERT/RoBERTa-style model for Akkadian, which rules out embedding-based aligners like SimAlign.
Script complexity: The mix of syllabic writing and logograms (Sumerograms) makes straightforward lexical matching unreliable.
In practice, this means that off-the-shelf methods (e.g. Gale–Church) tend to fail pretty badly.
\nWhat I’m trying instead
\nThis notebook implements a custom unsupervised alignment pipeline designed around those constraints.
\nA few key ideas:
\nNormalized regret scoring\nInstead of scoring alignments purely by raw fit, each sentence is judged relative to its best possible alignment. The idea is to prevent “easy” sentences from soaking up all the alignment mass while damaged or complex sentences get dropped as NULL.
Entity-based anchoring\nProper names, divine names, geographic names, numbers, and some Sumerograms are detected and used as high-confidence anchors. I’m also using simple phonetic matching (Soundex-style) to connect Akkadian transliterations with English spellings (e.g. Aššur ↔ Ashur).
Monotonic global alignment\nThese anchors constrain a dynamic programming / Viterbi pass so that the final alignment path stays monotonic and globally consistent, rather than making lots of locally optimal but globally incoherent matches.
Current pipeline
\nAt a high level, the workflow is:
\nA. Preprocessing: normalize Sumerograms, determinatives, and gaps
\nB. Lexical induction: unsupervised word translation learning via IBM Model 1
\nC. Scout phase: estimate an upper bound (“alignment ceiling”) for each sentence
\nD. Global optimization: dynamic programming that maximizes total satisfaction under monotonic constraints
\nWhat I’m looking for
\nI’d really appreciate feedback from anyone who has worked on:
\nsentence alignment in low-resource or historical settings
alternatives to embedding-based alignment
better ways to handle one-to-many / many-to-one sentence mappings
If you see conceptual flaws, simpler alternatives, or places where this could be tightened up, I’d love to hear it. The notebook is open-source and very much a work in progress.
\nThanks!
", "messages": [ { "id": "3388441", "postDate": "01/08/2026 21:05:06", "content": "I’m working on an English–Akkadian sentence alignment notebook aimed at creating higher-quality parallel training data for NMT in the context of the Deep Past Initiative challenge.
\nhttps://www.kaggle.com/code/aaronbornstein/eng-akk-low-resource-sent-alignment
\nThe goal is practical: get usable sentence-level alignments for a very low-resource, historically messy setting, where many standard tools don’t really apply.
\nWhy this is hard
\nAligning Akkadian cuneiform with English translations breaks a lot of assumptions that common alignment methods rely on:
Damaged source text: Tablets are often fragmentary, with lacunae ([…]) and missing context.
Morphological mismatch: Akkadian is a morphologically rich Semitic language. A single Akkadian word can correspond to an entire English phrase, and word order differs a lot from English.
Data scarcity: We’re nowhere near the scale of modern parallel corpora used for contemporary MT.
No pretrained embeddings: There’s no BERT/RoBERTa-style model for Akkadian, which rules out embedding-based aligners like SimAlign.
Script complexity: The mix of syllabic writing and logograms (Sumerograms) makes straightforward lexical matching unreliable.
In practice, this means that off-the-shelf methods (e.g. Gale–Church) tend to fail pretty badly.
\nWhat I’m trying instead
\nThis notebook implements a custom unsupervised alignment pipeline designed around those constraints.
\nA few key ideas:
\nNormalized regret scoring\nInstead of scoring alignments purely by raw fit, each sentence is judged relative to its best possible alignment. The idea is to prevent “easy” sentences from soaking up all the alignment mass while damaged or complex sentences get dropped as NULL.
Entity-based anchoring\nProper names, divine names, geographic names, numbers, and some Sumerograms are detected and used as high-confidence anchors. I’m also using simple phonetic matching (Soundex-style) to connect Akkadian transliterations with English spellings (e.g. Aššur ↔ Ashur).
Monotonic global alignment\nThese anchors constrain a dynamic programming / Viterbi pass so that the final alignment path stays monotonic and globally consistent, rather than making lots of locally optimal but globally incoherent matches.
Current pipeline
\nAt a high level, the workflow is:
\nA. Preprocessing: normalize Sumerograms, determinatives, and gaps
\nB. Lexical induction: unsupervised word translation learning via IBM Model 1
\nC. Scout phase: estimate an upper bound (“alignment ceiling”) for each sentence
\nD. Global optimization: dynamic programming that maximizes total satisfaction under monotonic constraints
\nWhat I’m looking for
\nI’d really appreciate feedback from anyone who has worked on:
\nsentence alignment in low-resource or historical settings
alternatives to embedding-based alignment
better ways to handle one-to-many / many-to-one sentence mappings
If you see conceptual flaws, simpler alternatives, or places where this could be tightened up, I’d love to hear it. The notebook is open-source and very much a work in progress.
\nThanks!
", "rawMarkdown": "I’m working on an English–Akkadian sentence alignment notebook aimed at creating higher-quality parallel training data for NMT in the context of the Deep Past Initiative challenge.\n\nhttps://www.kaggle.com/code/aaronbornstein/eng-akk-low-resource-sent-alignment\n\nThe goal is practical: get usable sentence-level alignments for a very low-resource, historically messy setting, where many standard tools don’t really apply.\n\n**Why this is hard**\n\n- Aligning Akkadian cuneiform with English translations breaks a lot of assumptions that common alignment methods rely on:\n\n- Damaged source text: Tablets are often fragmentary, with lacunae ([...]) and missing context.\n\n- Morphological mismatch: Akkadian is a morphologically rich Semitic language. A single Akkadian word can correspond to an entire English phrase, and word order differs a lot from English.\n\n- Data scarcity: We’re nowhere near the scale of modern parallel corpora used for contemporary MT.\n\n- No pretrained embeddings: There’s no BERT/RoBERTa-style model for Akkadian, which rules out embedding-based aligners like SimAlign.\n\n- Script complexity: The mix of syllabic writing and logograms (Sumerograms) makes straightforward lexical matching unreliable.\n\nIn practice, this means that off-the-shelf methods (e.g. Gale–Church) tend to fail pretty badly.\n\n**What I’m trying instead**\n\nThis notebook implements a custom unsupervised alignment pipeline designed around those constraints.\n\nA few key ideas:\n\n1. Normalized regret scoring\nInstead of scoring alignments purely by raw fit, each sentence is judged relative to its best possible alignment. The idea is to prevent “easy” sentences from soaking up all the alignment mass while damaged or complex sentences get dropped as NULL.\n\n2. Entity-based anchoring\nProper names, divine names, geographic names, numbers, and some Sumerograms are detected and used as high-confidence anchors. I’m also using simple phonetic matching (Soundex-style) to connect Akkadian transliterations with English spellings (e.g. Aššur ↔ Ashur).\n\n3. Monotonic global alignment\nThese anchors constrain a dynamic programming / Viterbi pass so that the final alignment path stays monotonic and globally consistent, rather than making lots of locally optimal but globally incoherent matches.\n\n**Current pipeline**\n\nAt a high level, the workflow is:\n\nA. Preprocessing: normalize Sumerograms, determinatives, and gaps\n\nB. Lexical induction: unsupervised word translation learning via IBM Model 1\n\nC. Scout phase: estimate an upper bound (“alignment ceiling”) for each sentence\n\nD. Global optimization: dynamic programming that maximizes total satisfaction under monotonic constraints\n\n**What I’m looking for**\n\nI’d really appreciate feedback from anyone who has worked on:\n\n- sentence alignment in low-resource or historical settings\n\n- alternatives to embedding-based alignment\n\n- better ways to handle one-to-many / many-to-one sentence mappings\n\nIf you see conceptual flaws, simpler alternatives, or places where this could be tightened up, I’d love to hear it. The notebook is open-source and very much a work in progress.\n\nThanks!", "votes": null }, { "id": "3388442", "postDate": "01/08/2026 21:06:31", "content": "Sample Alignments from my Pipeline
\n================================================================================
\nALIGNMENT TEST: Documents from train.csv
\n================================================================================
\nLoaded 1561 documents from train.csv
\n================================================================================
\nDoc 1: d4140158-e25a-4b80-9148-baababa6fd4a
\nSentences: 7, Tokens: 86
\nEng[0:1] → [Akk 0:9]
\nEng: To Aššur-imittī from Aššur-bāni: Why do you keep writing me about silver?
\nAkk: a-na a-šur-i-mi-ti qi-bi-ma um-ma a-šur-ba-ni-ma mi-nu-um ša a-na KU.BABBAR
\nScore: -37.26
Eng[1:2] → [Akk 9:26]
\nEng: You say: “Send me silver!” When you went away you left me 15 shekels of silver, and that was spent before Kurub-Ištar returned from Tawiniya.
\nAkk: ta-aš-ta-na-pa-ra-ni um-ma a-ta-ma KU.BABBAR še-bi-lam i-nu-mi ta-ta-al-ku 15 GIN KU.BABBAR te-zi-ba-ma a-di kur-ub-IŠTAR iš-tu ta-wi-ni-a i-tu-ra-ni ga-me-er
\nScore: -71.13
Eng[2:3] → [Akk 26:29]
\nEng: You said: “Go!
\nAkk: um-ma a-ta-ma a-la-ak
\nScore: -11.16
Eng[3:4] → [Akk 29:49]
\nEng: I shall send you tablets worth some 10 minas of silver, so you can have the silver paid, then set out and leave.” Instead of sending me tablets you warn me constantly whenever the merchant certified our tablets.
\nAkk: ṭup-pe-e ša KU.BABBAR 10 ma-na u-še-ba-la-kum-ma KU.BABBAR ša-aš-qi-il₅-ma ti-ib-a-ma a-tal-kam ki-ma ṭup-pe-e še-bu-lim a-na ṭup-pi-ni ša i-ma-ta DAM.GAR iḫ-ri-mu tu-uš-ta-na-e-di
\nScore: -175.69
Eng[4:5] → [Akk 49:61]
\nEng: And you know yourself that you left only 30 portions of food for me.
\nAkk: u a-ta-ma ti-de₈-e ki-ma a-ka-al-tam 30 te-zi-ba-ni i-na qa-qi-ri-im da-nim a-nim a-a-kam
\nScore: -59.85
Eng[5:6] → [NULL]
\nEng: From where shall I get anything so I can feed anyone under these harsh conditions?
\nScore: -100.00
Eng[6:7] → [Akk 61:86]
\nEng: And I, as you know, I must feed and raise the son of Šu-Ištar, the maid and the slave, (and) you do not feed me, so I have become indebted for at least 1 mina of silver for the food for the household.
\nAkk: la-al-qe-ma lu-ša-ki-il₅ a-na-ku wa-di a-ma DUMU šu-IŠTAR am-tam u IR ša u-ša-ku-lu u u-ra-bu-u-ni la tu ša-ki-li-ni-ma KU.BABBAR 1 ma-na a-na u-ku-ul-ti E be-tim a-ta-aḫ-ba-al
\nScore: -147.24
\nDoc Summary: 7 alignments, 1 NULLs, 0 merges
================================================================================
\nDoc 2: 23fbe6d7-8891-49f1-b812-415e01ebcdc4
\nSentences: 4, Tokens: 36
\nEng[0:1] → [Akk 0:10]
\nEng: Seal of Aššur-šamšī, son of Šu-Ištar, seal of Aššur-lamassī, seal of Dān-Aššur, son of Kususārum.
\nAkk: KIŠIB a-šur-(d)UTU-ši DUMU šu-IŠTAR KIŠIB a-šur-la-ma-si KIŠIB dan-a-šur DUMU ku-su-sa-ri-im
\nScore: -12.52
Eng[1:2] → [Akk 10:18]
\nEng: mina of refined silver Aššur-lamassī owes to Lamassutum.
\nAkk: ma-na KU ṣa-ru-pa-am i-ṣe-er a-šur-la-ma-si la-ma-sa-tum ti-šu
\nScore: 0.71
Eng[2:3] → [Akk 18:28]
\nEng: From the week of Kurub-Ištar and Iliš-tikal he will pay in 4 months.
\nAkk: iš-tu ḫa-mu-uš-tim ša kur-ub-IŠTAR u i-li-iš-ta-kal a-na 4 ITU.KAM i-ša-qal
\nScore: -12.06
Eng[3:4] → [Akk 28:36]
\nEng: If he does not pay, he will add interest in accordance with the regulation of the colony.
\nAkk: šu-ma la iš-qul ki-ma a-wa-at ka-ri-im ṣi-ib-tam u-ṣa-ab
\nScore: -56.37
\nDoc Summary: 4 alignments, 0 NULLs, 0 merges
================================================================================
\nDoc 3: 0819f9cd-c0a0-4be4-9307-040881cbb5db
\nSentences: 3, Tokens: 15
\nEng[0:1] → [Akk 0:8]
\nEng: Sealed by Buburānum; to Abiya son of Bebe and Aššur-ṭab.
\nAkk: KIŠIB bu-bu-ra-nu-um a-na a-bi₄-a DUMU be-e-be u a-šur-DU₁₀
\nScore: -23.25
Eng[1:2] → [Akk 8:11]
\nEng: Abiya.
\nAkk: -nu a-bi₄-a
\nScore: -0.59
Eng[2:3] → [Akk 11:15]
\nEng: If you are truly my brothers, take care!
\nAkk: šu-ma a-ḫu-a a-tu-nu iḫ-da
\nScore: -31.05
\nDoc Summary: 3 alignments, 0 NULLs, 0 merges
================================================================================
\nDoc 4: f7c0535b-c527-4289-911a-c2a3f26ba2cf
\nSentences: 2, Tokens: 13
\nEng[0:1] → [Akk 0:4]
\nEng: To Ennam-Aššur; seal of Amur-Ištar.
\nAkk: a-na en-nu-um-a-šur KIŠIB a-mur-IŠTAR
\nScore: 1.51
Eng[1:2] → [Akk 4:13]
\nEng: 3 minas of copper and 2 shekels of silver
\nAkk: 3 ma-na URUDU u 2 GIN KU.BABBAR a-
\nScore: 2.99
\nDoc Summary: 2 alignments, 0 NULLs, 0 merges
================================================================================
\nDoc 5: 578f8ff2-4c59-4978-82ce-80eaef1acab8
\nSentences: 5, Tokens: 89
\nEng[0:1] → [Akk 0:50]
\nEng: Ištar-lamassī and Mannum-kī-iliya and pay the silver to Mannum-kī-Aššur or to Mannum-kī-Aššur's representatives, and as to the 20 talents of wool and the -stone that Mannum-kī-Aššur holds as security, when you pay the silver let him identify Mannum-kī-iliya's seal in case the security is sealed, and then take out the tablet of the 3 minas of silver and its interest, and seal this tablet and the -stone and entrust it to Ištar-lamassī.
\nAkk: IŠTAR-la-ma-si u ma-nu-ki-DINGIR i-zi-za-ma KU.BABBAR a-na ma-nu-ki-a-šur u-la a-na ša ki-ma ma-nu-ki-a-šur šu-uq-la-ma 20 GU SIG.ḪI.A u si-ka-an-LUGAL ša a-na ša-pa-ar-tim ma-nu-ki-a-šur u-ka-lu ki-ma KU.BABBAR ta-ša-qa-la-ni a-šar ša-par-tum ka-an-ku ku-nu-ki ma-nu-ki-i-li-a lu-di-ma u ṭup-pa-am ša 3 ma-na KU.BABBAR u ṣi-ba-ti-šu še-ṣi-a-ma ṭup-pa-am šu-a-ti u si-ka-LUGAL ku-un-ka-ma a-na IŠTAR-la-ma-si pi-iq-da u
\nScore: -139.72
Eng[1:2] → [Akk 50:59]
\nEng: Also, when the silver is paid set witnesses to prove that the wool is mine.
\nAkk: i-ša-qa-al KU.BABBAR ki-ma ša-ap-tum i-a-tu-ni ši-be šu-uk-na u ša-ap-tam
\nScore: -45.56
Eng[2:3] → [Akk 59:76]
\nEng: Also, check the wool for Šalim-wardī and Mannum-kī-iliya, and set witnesses for them about how much wool is cleared, stating that the wool and 7 donkeys belong to Ennam-Aššur.
\nAkk: a-na ša-lim-wa-ar-di u ma-nu-ki-i-li-a sa-ni-qa-ma ma-la ša-ap-tam i-za-ku-a-ni ši-be-e šu-uk-na-šu-nu-ti ki-ma ša-ap-tum u 7 ANŠE.ḪI.A ša en-um-a-šur
\nScore: -105.22
Eng[3:4] → [Akk 76:83]
\nEng: Seal the wool and entrust it under your seals together with my other goods so they may conduct it here.
\nAkk: ša-ap-tam ka-ni-ka-ma ku-nu-ki-ku-nu iš-ti lu-qu-ti-a-ma pi-iq-da-ma li-ir-di-u-nim
\nScore: -92.65
Eng[4:5] → [Akk 83:89]
\nEng: Šalim-wardī shall do for them what we instructed him to there.
\nAkk: a ma-la a-na-kam ša-lim-ar-di nu-na-ḫi-du-ni le-pu-uš-šu-nu-ti
\nScore: -59.19
\nDoc Summary: 5 alignments, 0 NULLs, 0 merges
================================================================================
\nDoc 6: 4c37b764-d3cf-4532-8655-ed66a98fc370
\nSentences: 3, Tokens: 97
\nEng[0:1] → [Akk 0:48]
\nEng: Ali-ahum seized us against Enna-Suen, and concerning a tablet sealed by Enna-Suen stating that Ali-ahum owes 20 minas of silver to Iṣmid-ilum - that Enna-Suen settled the matter on behalf of his investor Aššur-malik son of Ali-ahum, so that with respect to the 20 minas of silver plus interest for which Ali-ahum was indebted to Iṣmid-ilum - if anyone raises claim against Ali-ahum, Enna-Suen will clear Ali-ahum for the silver and interest on behalf of his investor Aššur-malik.
\nAkk: a-la-ḫu-um a-na i-na-su-in iṣ-ba-at-ni-a-ti-ma ṭup-pa-am ša ku-nu-uk i-na-su-in ša 20 ma-na KU.BABBAR a-la-ḫu-um a-na iṣ-mi-di-lim i-ḫi-ib-lu i-na-su-in ki-ma a-šur-ma-lik DUMU a-la-ḫi-im um-me-a-ni-šu a-wa-tim u-ga-me-ru-ma ša 20 ma-na KU.BABBAR u ṣi-ba-ti-šu a-la-ḫu-um a-na iṣ-mi-di-lim i-ḫi-ib-lu šu-ma ma-ma-an a-na a-la-ḫi-im i-tu-a-ar i-na-su-in ki-ma a-šur-ma-lik um-me-a-ni-šu a-na KU.BABBAR u ṣi-ba-ti-šu a-la-ḫa-am
\nScore: -214.65
Eng[1:2] → [NULL]
\nEng: Ali-ahum gave a tablet sealed by Enna-Suen, stating that he will clear Ali-ahum for the silver and interest to Enna-Suen, and he has invalidated it; also, the tablet stating that Ali-ahum owes 20 minas of silver plus interest to Iṣmid-ilum - Enna-Suen gave (that) tablet to Ali-ahum and he has invalidated it.
\nScore: -100.00
Eng[2:3] → [Akk 48:97]
\nEng: Witnessed by Ṭāb-Aššur son of Al, by Dumqiya son of Concerning Kuzuzārum and Ali-ahum.
\nAkk: u-bu-bu-u ṭup-pa-am ša ku-nu-uk i-na-su-in ša a-la-ḫa-am a-na KU.BABBAR u ṣi-ba-ti-šu u-bu-bu-u a-la-ḫu-um a-na i-na-su-in i-di-in-ma i-du-uk u ṭup-pa-am ša 20 ma-na KU.BABBAR u ṣi-ba-ti-šu a-la-ḫu-um a-na iṣ-mi-di-lim i-ḫi-ib-lu i-na-su-in ṭup-pa-am a-na a-la-ḫi-im i-di-ma i-du-uk IGI ṭa-bi₄-a-šur DUMU a-la+la- IGI du-um-qi-a DUMU a ša ku-zu-za-ri-im u a-la-ḫi-im
\nScore: -21.56
\nDoc Summary: 3 alignments, 1 NULLs, 0 merges
================================================================================
\nDoc 7: 46edfe73-4380-41d6-be65-cb560f283a6e
\nSentences: 7, Tokens: 50
\nEng[0:1] → [Akk 0:18]
\nEng: From Šu-Ištar to Šuma-libši, Ilī-ālum and Amur-Ištar; specifically to Šuma-libši and Ilī-ālum: You owe 14 shekels of refined silver.
\nAkk: um-ma šu-IŠTAR-ma a-na šu-ma-li-ib-ši I-li-a-lim u a-mur-IŠTAR qi-bi-ma a-na šu-ma-li-ib-ši u I-li-a-lim qi-bi-ma 14 GIN KU.BABBAR ṣa-ru-pa-am ḫa-bu-la-tu-nu
\nScore: -46.30
Eng[1:2] → [Akk 18:21]
\nEng: I hold your certified tablet.
\nAkk: ṭup-pa-ak-nu ḫa-ar-ma-am u-ka-al
\nScore: -11.74
Eng[2:3] → [Akk 21:24]
\nEng: Your terms are overdue by a year.
\nAkk: u₄-mu-ku-nu ša-na-at e-ti-qu
\nScore: -28.43
Eng[3:4] → [Akk 24:31]
\nEng: The silver (loan) was with joint liability.
\nAkk: KU.BABBAR i qa-qa-ad šal-mi-ku-nu u ke-ni-ku-nu ra-ki-is
\nScore: -49.39
Eng[4:5] → [Akk 31:34]
\nEng: The silver was merchant-silver\".
\nAkk: KU.BABBAR KU.BABBAR-ap tam-ka-ri-im
\nScore: -24.78
Eng[5:6] → [Akk 34:41]
\nEng: Pay the silver and its interest there to Amur-Ištar.
\nAkk: a-ma-kam KU.BABBAR u ṣi-ba-su a-na a-mur-IŠTAR šu-qu-ul
\nScore: -4.07
Eng[6:7] → [Akk 41:50]
\nEng: To Amur-Ištar: Urgent, pay attention and have the silver and its interest paid.\"
\nAkk: a-na a-mur-IŠTAR qi-bi₄-ma a-pu-tum i-ḫi-id KU.BABBAR u ṣi-ba-su ša-aš-qi-il₅
\nScore: -18.57
\nDoc Summary: 7 alignments, 0 NULLs, 0 merges
================================================================================
\nDoc 8: 2d1b0c9a-4403-4255-8198-406e3501a4ab
\nSentences: 11, Tokens: 81
\nEng[0:1] → [Akk 0:7]
\nEng: 4 talents 42 minas of copper with Iddin-Aurum.
\nAkk: 4 GU 42 ma-na URUDU iš-ti i-di-(d)MAR.TU
\nScore: -26.77
Eng[1:2] → [Akk 7:12]
\nEng: 0.5 mina of silver, working capital of Kitaya.
\nAkk: 0.5 ma-na KU.BABBAR be-u-la-at ki-ta-a
\nScore: -13.04
Eng[2:3] → [Akk 12:19]
\nEng: 8 minas of silver with Šuli, son of Puzur-Abeh.
\nAkk: 8 ma-na KU.BABBAR KI šu-li DUMU PUZUR₄-a-be-eḫ
\nScore: -3.77
Eng[3:4] → [Akk 19:34]
\nEng: 4 minas 3.83333 shekels of silver with Idnāya, month IV, eponymy of Šudāya; he will add 1 1 / 4 shekel per (mina per month) as interest.
\nAkk: 4 ma-na 3.83334 GIN KU.BABBAR KI id-na-a ITU.KAM ma-ḫu-ur-i-li li-mu-um šu-da-a 1.25 GIN.TA ṣi-ib-tam u-ṣa-ab
\nScore: -86.14
Eng[4:5] → [Akk 34:47]
\nEng: 1 talent 20 minas of copper and 25 shekels with Ibaya.
\nAkk: 1 GU 20 ma-na URUDU u 0.33333 ma-na 5 GIN KU.BABBAR KI i-ba-a
\nScore: -11.85
Eng[5:6] → [Akk 47:52]
\nEng: 8 shekels of silver with Ikūn-Mīšar.
\nAkk: 8 GIN KU.BABBAR KI i-ku-mi-šar
\nScore: -1.51
Eng[6:7] → [Akk 52:61]
\nEng: 2 minas 10 shekels of silver with Aššur-dūrī, year eponymy of Iphurum.
\nAkk: 2 ma-na 10 GIN KU.BABBAR KI a-šur-du-ri li-mu-um ip-ḫu-ru-um
\nScore: -17.49
Eng[7:8] → [Akk 61:65]
\nEng: He will add 1.5 shekel per (mina per month).
\nAkk: 1.5 GIN.TA ṣi-ib-tam u-ṣa-ab
\nScore: -40.22
Eng[8:9] → [Akk 65:70]
\nEng: 0.5 mina of silver with Balṭu'um.
\nAkk: 0.5 ma-na KU.BABBAR KI ba-al-ṭu-im
\nScore: -22.76
Eng[9:10] → [Akk 70:76]
\nEng: 25 shekels (of silver) working capital of Galdaya.
\nAkk: 0.33333 ma-na 5 GIN be-u-la-at gal-da-a
\nScore: -21.05
Eng[10:11] → [Akk 76:81]
\nEng: 1 mina of silver with Galdaya.
\nAkk: 1 ma-na KU.BABBAR KI gal-da-a
\nScore: -2.51
\nDoc Summary: 11 alignments, 0 NULLs, 0 merges
================================================================================
\nOVERALL SUMMARY
\n================================================================================
\nTotal alignments: 42
\nNULL alignments: 2 (4.8%)
\nMerged alignments: 0 (0.0%)
\nAverage score: -46.24
", "rawMarkdown": "Sample Alignments from my Pipeline\n\n================================================================================\n\nALIGNMENT TEST: Documents from train.csv\n\n================================================================================\n\nLoaded 1561 documents from train.csv\n\n\n\n================================================================================\n\nDoc 1: d4140158-e25a-4b80-9148-baababa6fd4a\n\n Sentences: 7, Tokens: 86\n\n--------------------------------------------------------------------------------\n\n\n\n 1. Eng[0:1] → [Akk 0:9]\n\n Eng: To Aššur-imittī from Aššur-bāni: Why do you keep writing me about silver?\n\n Akk: a-na a-šur-i-mi-ti qi-bi-ma um-ma a-šur-ba-ni-ma mi-nu-um ša a-na KU.BABBAR\n\n Score: -37.26\n\n\n\n 2. Eng[1:2] → [Akk 9:26]\n\n Eng: You say: “Send me silver!” When you went away you left me 15 shekels of silver, and that was spent before Kurub-Ištar returned from Tawiniya.\n\n Akk: ta-aš-ta-na-pa-ra-ni um-ma a-ta-ma KU.BABBAR še-bi-lam i-nu-mi ta-ta-al-ku 15 GIN KU.BABBAR te-zi-ba-ma a-di kur-ub-IŠTAR iš-tu ta-wi-ni-a i-tu-ra-ni ga-me-er\n\n Score: -71.13\n\n\n\n 3. Eng[2:3] → [Akk 26:29]\n\n Eng: You said: “Go!\n\n Akk: um-ma a-ta-ma a-la-ak\n\n Score: -11.16\n\n\n\n 4. Eng[3:4] → [Akk 29:49]\n\n Eng: I shall send you tablets worth some 10 minas of silver, so you can have the silver paid, then set out and leave.” Instead of sending me tablets you warn me constantly whenever the merchant certified our tablets.\n\n Akk: ṭup-pe-e ša KU.BABBAR 10 ma-na u-še-ba-la-kum-ma KU.BABBAR ša-aš-qi-il₅-ma ti-ib-a-ma a-tal-kam ki-ma ṭup-pe-e še-bu-lim a-na ṭup-pi-ni ša i-ma-ta DAM.GAR iḫ-ri-mu tu-uš-ta-na-e-di\n\n Score: -175.69\n\n\n\n 5. Eng[4:5] → [Akk 49:61]\n\n Eng: And you know yourself that you left only 30 portions of food for me.\n\n Akk: u a-ta-ma ti-de₈-e ki-ma a-ka-al-tam 30 te-zi-ba-ni i-na qa-qi-ri-im da-nim a-nim a-a-kam\n\n Score: -59.85\n\n\n\n 6. Eng[5:6] → [NULL]\n\n Eng: From where shall I get anything so I can feed anyone under these harsh conditions?\n\n Score: -100.00\n\n\n\n 7. Eng[6:7] → [Akk 61:86]\n\n Eng: And I, as you know, I must feed and raise the son of Šu-Ištar, the maid and the slave, (and) you do not feed me, so I have become indebted for at least 1 mina of silver for the food for the household.\n\n Akk: la-al-qe-ma lu-ša-ki-il₅ a-na-ku wa-di a-ma DUMU šu-IŠTAR am-tam u IR ša u-ša-ku-lu u u-ra-bu-u-ni la tu ša-ki-li-ni-ma KU.BABBAR 1 ma-na a-na u-ku-ul-ti E be-tim a-ta-aḫ-ba-al\n\n Score: -147.24\n\n\n\n Doc Summary: 7 alignments, 1 NULLs, 0 merges\n\n\n\n================================================================================\n\nDoc 2: 23fbe6d7-8891-49f1-b812-415e01ebcdc4\n\n Sentences: 4, Tokens: 36\n\n--------------------------------------------------------------------------------\n\n\n\n 1. Eng[0:1] → [Akk 0:10]\n\n Eng: Seal of Aššur-šamšī, son of Šu-Ištar, seal of Aššur-lamassī, seal of Dān-Aššur, son of Kususārum.\n\n Akk: KIŠIB a-šur-(d)UTU-ši DUMU šu-IŠTAR KIŠIB a-šur-la-ma-si KIŠIB dan-a-šur DUMU ku-su-sa-ri-im\n\n Score: -12.52\n\n\n\n 2. Eng[1:2] → [Akk 10:18]\n\n Eng:Considering anchors, there are also the quantity-based fragments, e.g., 2 GÍN = 2 shekels, 5 ma-na = 5 minas, 30 GÚ = 30 talents.
", "rawMarkdown": "Considering anchors, there are also the quantity-based fragments, e.g., 2 GÍN = 2 shekels, 5 ma-na = 5 minas, 30 GÚ = 30 talents.", "votes": null }, { "id": "3388725", "postDate": "01/09/2026 13:29:29", "content": "The challenge with this though is that the units often change between the translation and source for example
\nEng: 25 shekels (of silver) working capital of Galdaya.
\nAkk: 0.33333 ma-na 5 GIN be-u-la-at gal-da-a
", "rawMarkdown": "The challenge with this though is that the units often change between the translation and source for example \n\n\nEng: 25 shekels (of silver) working capital of Galdaya.\n\nAkk: 0.33333 ma-na 5 GIN be-u-la-at gal-da-a", "votes": null }, { "id": "3388823", "postDate": "01/09/2026 16:52:15", "content": "Have you quantified the problem? I haven't seen that too often, so would think it's a rare case. If not, I'll add a section to the analysis notebook I'm working on.
", "rawMarkdown": "Have you quantified the problem? I haven't seen that too often, so would think it's a rare case. If not, I'll add a section to the analysis notebook I'm working on.", "votes": null }, { "id": "3388839", "postDate": "01/09/2026 17:14:16", "content": "There's a pretty good description of the Mesopotamian weight system here in ORACC (but their rules for formatting these numbers in transcription is different of course): https://oracc.museum.upenn.edu/doc/help/editinginatf/metrology/metrologicaltables/index.html#weight
\nIt is important to note that the fractions (which are retained as unicode fractions in the test data) were turned into decimals in the train data and in the published_texts.csv, as noted by Aaron's example:
\n\n\nEng: 25 shekels (of silver) working capital of Galdaya.
\nAkk: 0.33333 ma-na 5 GIN be-u-la-at gal-da-a
\n
So in this case, \"25 shekels\" would appear as: \"⅓ ma-na 5 GÍN\" in the test data.
", "rawMarkdown": "There's a pretty good description of the Mesopotamian weight system here in ORACC (but their rules for formatting these numbers in transcription is different of course): https://oracc.museum.upenn.edu/doc/help/editinginatf/metrology/metrologicaltables/index.html#weight\n\nIt is important to note that the fractions (which are retained as unicode fractions in the test data) were turned into decimals in the train data and in the published_texts.csv, as noted by Aaron's example:\n\n>Eng: 25 shekels (of silver) working capital of Galdaya.\n\n>Akk: 0.33333 ma-na 5 GIN be-u-la-at gal-da-a\n\nSo in this case, \"25 shekels\" would appear as: \"⅓ ma-na 5 GÍN\" in the test data.", "votes": null }, { "id": "3388900", "postDate": "01/09/2026 19:14:04", "content": "The question is though, is there an established norm that the translations follow? Or at the very least, if there are several rules is there one that stands out as the leading one?
\n\n\nAkk: 0.33333 ma-na 5 GIN be-u-la-at gal-da-a
\nEng: 25 shekels (of silver) working capital of Galdaya.
\n
What I would find intuitive is to save the original units in the translated text, i.e.:
\n\n\nAkk: ⅓ ma-na 5 GIN be-u-la-at gal-da-a
\nEng: ⅓ mina and 5 shekels (of silver) working capital of Galdaya.
\n
I don't know if that's the leading rule, though. Not for the hidden test, at least. I'll check it out for the train.csv
Adding this to my method now lets see if it helps i definitely think this should enable better modeling and fewer bytes with BYT5
", "rawMarkdown": "Adding this to my method now lets see if it helps i definitely think this should enable better modeling and fewer bytes with BYT5", "votes": null }, { "id": "3390184", "postDate": "01/12/2026 17:46:07", "content": "Yes, your intuitive approach would usually fit with the test set. The only exception is when a translator does the math in their head (e.g. 1 mina = 60 shekels, and the decimals reflect that it is usually divided into thirds) 1/3 mina + 5 shekels = 25 shekels
", "rawMarkdown": "Yes, your intuitive approach would usually fit with the test set. The only exception is when a translator does the math in their head (e.g. 1 mina = 60 shekels, and the decimals reflect that it is usually divided into thirds) 1/3 mina + 5 shekels = 25 shekels", "votes": null } ], "comments": [ { "id": 3388442, "author_name": "aaronbornstein", "author_url": "", "post_date": "01/08/2026 21:06:31", "content": "Sample Alignments from my Pipeline
\n================================================================================
\nALIGNMENT TEST: Documents from train.csv
\n================================================================================
\nLoaded 1561 documents from train.csv
\n================================================================================
\nDoc 1: d4140158-e25a-4b80-9148-baababa6fd4a
\nSentences: 7, Tokens: 86
\nEng[0:1] → [Akk 0:9]
\nEng: To Aššur-imittī from Aššur-bāni: Why do you keep writing me about silver?
\nAkk: a-na a-šur-i-mi-ti qi-bi-ma um-ma a-šur-ba-ni-ma mi-nu-um ša a-na KU.BABBAR
\nScore: -37.26
Eng[1:2] → [Akk 9:26]
\nEng: You say: “Send me silver!” When you went away you left me 15 shekels of silver, and that was spent before Kurub-Ištar returned from Tawiniya.
\nAkk: ta-aš-ta-na-pa-ra-ni um-ma a-ta-ma KU.BABBAR še-bi-lam i-nu-mi ta-ta-al-ku 15 GIN KU.BABBAR te-zi-ba-ma a-di kur-ub-IŠTAR iš-tu ta-wi-ni-a i-tu-ra-ni ga-me-er
\nScore: -71.13
Eng[2:3] → [Akk 26:29]
\nEng: You said: “Go!
\nAkk: um-ma a-ta-ma a-la-ak
\nScore: -11.16
Eng[3:4] → [Akk 29:49]
\nEng: I shall send you tablets worth some 10 minas of silver, so you can have the silver paid, then set out and leave.” Instead of sending me tablets you warn me constantly whenever the merchant certified our tablets.
\nAkk: ṭup-pe-e ša KU.BABBAR 10 ma-na u-še-ba-la-kum-ma KU.BABBAR ša-aš-qi-il₅-ma ti-ib-a-ma a-tal-kam ki-ma ṭup-pe-e še-bu-lim a-na ṭup-pi-ni ša i-ma-ta DAM.GAR iḫ-ri-mu tu-uš-ta-na-e-di
\nScore: -175.69
Eng[4:5] → [Akk 49:61]
\nEng: And you know yourself that you left only 30 portions of food for me.
\nAkk: u a-ta-ma ti-de₈-e ki-ma a-ka-al-tam 30 te-zi-ba-ni i-na qa-qi-ri-im da-nim a-nim a-a-kam
\nScore: -59.85
Eng[5:6] → [NULL]
\nEng: From where shall I get anything so I can feed anyone under these harsh conditions?
\nScore: -100.00
Eng[6:7] → [Akk 61:86]
\nEng: And I, as you know, I must feed and raise the son of Šu-Ištar, the maid and the slave, (and) you do not feed me, so I have become indebted for at least 1 mina of silver for the food for the household.
\nAkk: la-al-qe-ma lu-ša-ki-il₅ a-na-ku wa-di a-ma DUMU šu-IŠTAR am-tam u IR ša u-ša-ku-lu u u-ra-bu-u-ni la tu ša-ki-li-ni-ma KU.BABBAR 1 ma-na a-na u-ku-ul-ti E be-tim a-ta-aḫ-ba-al
\nScore: -147.24
\nDoc Summary: 7 alignments, 1 NULLs, 0 merges
================================================================================
\nDoc 2: 23fbe6d7-8891-49f1-b812-415e01ebcdc4
\nSentences: 4, Tokens: 36
\nEng[0:1] → [Akk 0:10]
\nEng: Seal of Aššur-šamšī, son of Šu-Ištar, seal of Aššur-lamassī, seal of Dān-Aššur, son of Kususārum.
\nAkk: KIŠIB a-šur-(d)UTU-ši DUMU šu-IŠTAR KIŠIB a-šur-la-ma-si KIŠIB dan-a-šur DUMU ku-su-sa-ri-im
\nScore: -12.52
Eng[1:2] → [Akk 10:18]
\nEng: mina of refined silver Aššur-lamassī owes to Lamassutum.
\nAkk: ma-na KU ṣa-ru-pa-am i-ṣe-er a-šur-la-ma-si la-ma-sa-tum ti-šu
\nScore: 0.71
Eng[2:3] → [Akk 18:28]
\nEng: From the week of Kurub-Ištar and Iliš-tikal he will pay in 4 months.
\nAkk: iš-tu ḫa-mu-uš-tim ša kur-ub-IŠTAR u i-li-iš-ta-kal a-na 4 ITU.KAM i-ša-qal
\nScore: -12.06
Eng[3:4] → [Akk 28:36]
\nEng: If he does not pay, he will add interest in accordance with the regulation of the colony.
\nAkk: šu-ma la iš-qul ki-ma a-wa-at ka-ri-im ṣi-ib-tam u-ṣa-ab
\nScore: -56.37
\nDoc Summary: 4 alignments, 0 NULLs, 0 merges
================================================================================
\nDoc 3: 0819f9cd-c0a0-4be4-9307-040881cbb5db
\nSentences: 3, Tokens: 15
\nEng[0:1] → [Akk 0:8]
\nEng: Sealed by Buburānum; to Abiya son of Bebe and Aššur-ṭab.
\nAkk: KIŠIB bu-bu-ra-nu-um a-na a-bi₄-a DUMU be-e-be u a-šur-DU₁₀
\nScore: -23.25
Eng[1:2] → [Akk 8:11]
\nEng: Abiya.
\nAkk: -nu a-bi₄-a
\nScore: -0.59
Eng[2:3] → [Akk 11:15]
\nEng: If you are truly my brothers, take care!
\nAkk: šu-ma a-ḫu-a a-tu-nu iḫ-da
\nScore: -31.05
\nDoc Summary: 3 alignments, 0 NULLs, 0 merges
================================================================================
\nDoc 4: f7c0535b-c527-4289-911a-c2a3f26ba2cf
\nSentences: 2, Tokens: 13
\nEng[0:1] → [Akk 0:4]
\nEng: To Ennam-Aššur; seal of Amur-Ištar.
\nAkk: a-na en-nu-um-a-šur KIŠIB a-mur-IŠTAR
\nScore: 1.51
Eng[1:2] → [Akk 4:13]
\nEng: 3 minas of copper and 2 shekels of silver
\nAkk: 3 ma-na URUDU u 2 GIN KU.BABBAR a-
\nScore: 2.99
\nDoc Summary: 2 alignments, 0 NULLs, 0 merges
================================================================================
\nDoc 5: 578f8ff2-4c59-4978-82ce-80eaef1acab8
\nSentences: 5, Tokens: 89
\nEng[0:1] → [Akk 0:50]
\nEng: Ištar-lamassī and Mannum-kī-iliya and pay the silver to Mannum-kī-Aššur or to Mannum-kī-Aššur's representatives, and as to the 20 talents of wool and the -stone that Mannum-kī-Aššur holds as security, when you pay the silver let him identify Mannum-kī-iliya's seal in case the security is sealed, and then take out the tablet of the 3 minas of silver and its interest, and seal this tablet and the -stone and entrust it to Ištar-lamassī.
\nAkk: IŠTAR-la-ma-si u ma-nu-ki-DINGIR i-zi-za-ma KU.BABBAR a-na ma-nu-ki-a-šur u-la a-na ša ki-ma ma-nu-ki-a-šur šu-uq-la-ma 20 GU SIG.ḪI.A u si-ka-an-LUGAL ša a-na ša-pa-ar-tim ma-nu-ki-a-šur u-ka-lu ki-ma KU.BABBAR ta-ša-qa-la-ni a-šar ša-par-tum ka-an-ku ku-nu-ki ma-nu-ki-i-li-a lu-di-ma u ṭup-pa-am ša 3 ma-na KU.BABBAR u ṣi-ba-ti-šu še-ṣi-a-ma ṭup-pa-am šu-a-ti u si-ka-LUGAL ku-un-ka-ma a-na IŠTAR-la-ma-si pi-iq-da u
\nScore: -139.72
Eng[1:2] → [Akk 50:59]
\nEng: Also, when the silver is paid set witnesses to prove that the wool is mine.
\nAkk: i-ša-qa-al KU.BABBAR ki-ma ša-ap-tum i-a-tu-ni ši-be šu-uk-na u ša-ap-tam
\nScore: -45.56
Eng[2:3] → [Akk 59:76]
\nEng: Also, check the wool for Šalim-wardī and Mannum-kī-iliya, and set witnesses for them about how much wool is cleared, stating that the wool and 7 donkeys belong to Ennam-Aššur.
\nAkk: a-na ša-lim-wa-ar-di u ma-nu-ki-i-li-a sa-ni-qa-ma ma-la ša-ap-tam i-za-ku-a-ni ši-be-e šu-uk-na-šu-nu-ti ki-ma ša-ap-tum u 7 ANŠE.ḪI.A ša en-um-a-šur
\nScore: -105.22
Eng[3:4] → [Akk 76:83]
\nEng: Seal the wool and entrust it under your seals together with my other goods so they may conduct it here.
\nAkk: ša-ap-tam ka-ni-ka-ma ku-nu-ki-ku-nu iš-ti lu-qu-ti-a-ma pi-iq-da-ma li-ir-di-u-nim
\nScore: -92.65
Eng[4:5] → [Akk 83:89]
\nEng: Šalim-wardī shall do for them what we instructed him to there.
\nAkk: a ma-la a-na-kam ša-lim-ar-di nu-na-ḫi-du-ni le-pu-uš-šu-nu-ti
\nScore: -59.19
\nDoc Summary: 5 alignments, 0 NULLs, 0 merges
================================================================================
\nDoc 6: 4c37b764-d3cf-4532-8655-ed66a98fc370
\nSentences: 3, Tokens: 97
\nEng[0:1] → [Akk 0:48]
\nEng: Ali-ahum seized us against Enna-Suen, and concerning a tablet sealed by Enna-Suen stating that Ali-ahum owes 20 minas of silver to Iṣmid-ilum - that Enna-Suen settled the matter on behalf of his investor Aššur-malik son of Ali-ahum, so that with respect to the 20 minas of silver plus interest for which Ali-ahum was indebted to Iṣmid-ilum - if anyone raises claim against Ali-ahum, Enna-Suen will clear Ali-ahum for the silver and interest on behalf of his investor Aššur-malik.
\nAkk: a-la-ḫu-um a-na i-na-su-in iṣ-ba-at-ni-a-ti-ma ṭup-pa-am ša ku-nu-uk i-na-su-in ša 20 ma-na KU.BABBAR a-la-ḫu-um a-na iṣ-mi-di-lim i-ḫi-ib-lu i-na-su-in ki-ma a-šur-ma-lik DUMU a-la-ḫi-im um-me-a-ni-šu a-wa-tim u-ga-me-ru-ma ša 20 ma-na KU.BABBAR u ṣi-ba-ti-šu a-la-ḫu-um a-na iṣ-mi-di-lim i-ḫi-ib-lu šu-ma ma-ma-an a-na a-la-ḫi-im i-tu-a-ar i-na-su-in ki-ma a-šur-ma-lik um-me-a-ni-šu a-na KU.BABBAR u ṣi-ba-ti-šu a-la-ḫa-am
\nScore: -214.65
Eng[1:2] → [NULL]
\nEng: Ali-ahum gave a tablet sealed by Enna-Suen, stating that he will clear Ali-ahum for the silver and interest to Enna-Suen, and he has invalidated it; also, the tablet stating that Ali-ahum owes 20 minas of silver plus interest to Iṣmid-ilum - Enna-Suen gave (that) tablet to Ali-ahum and he has invalidated it.
\nScore: -100.00
Eng[2:3] → [Akk 48:97]
\nEng: Witnessed by Ṭāb-Aššur son of Al, by Dumqiya son of Concerning Kuzuzārum and Ali-ahum.
\nAkk: u-bu-bu-u ṭup-pa-am ša ku-nu-uk i-na-su-in ša a-la-ḫa-am a-na KU.BABBAR u ṣi-ba-ti-šu u-bu-bu-u a-la-ḫu-um a-na i-na-su-in i-di-in-ma i-du-uk u ṭup-pa-am ša 20 ma-na KU.BABBAR u ṣi-ba-ti-šu a-la-ḫu-um a-na iṣ-mi-di-lim i-ḫi-ib-lu i-na-su-in ṭup-pa-am a-na a-la-ḫi-im i-di-ma i-du-uk IGI ṭa-bi₄-a-šur DUMU a-la+la- IGI du-um-qi-a DUMU a ša ku-zu-za-ri-im u a-la-ḫi-im
\nScore: -21.56
\nDoc Summary: 3 alignments, 1 NULLs, 0 merges
================================================================================
\nDoc 7: 46edfe73-4380-41d6-be65-cb560f283a6e
\nSentences: 7, Tokens: 50
\nEng[0:1] → [Akk 0:18]
\nEng: From Šu-Ištar to Šuma-libši, Ilī-ālum and Amur-Ištar; specifically to Šuma-libši and Ilī-ālum: You owe 14 shekels of refined silver.
\nAkk: um-ma šu-IŠTAR-ma a-na šu-ma-li-ib-ši I-li-a-lim u a-mur-IŠTAR qi-bi-ma a-na šu-ma-li-ib-ši u I-li-a-lim qi-bi-ma 14 GIN KU.BABBAR ṣa-ru-pa-am ḫa-bu-la-tu-nu
\nScore: -46.30
Eng[1:2] → [Akk 18:21]
\nEng: I hold your certified tablet.
\nAkk: ṭup-pa-ak-nu ḫa-ar-ma-am u-ka-al
\nScore: -11.74
Eng[2:3] → [Akk 21:24]
\nEng: Your terms are overdue by a year.
\nAkk: u₄-mu-ku-nu ša-na-at e-ti-qu
\nScore: -28.43
Eng[3:4] → [Akk 24:31]
\nEng: The silver (loan) was with joint liability.
\nAkk: KU.BABBAR i qa-qa-ad šal-mi-ku-nu u ke-ni-ku-nu ra-ki-is
\nScore: -49.39
Eng[4:5] → [Akk 31:34]
\nEng: The silver was merchant-silver\".
\nAkk: KU.BABBAR KU.BABBAR-ap tam-ka-ri-im
\nScore: -24.78
Eng[5:6] → [Akk 34:41]
\nEng: Pay the silver and its interest there to Amur-Ištar.
\nAkk: a-ma-kam KU.BABBAR u ṣi-ba-su a-na a-mur-IŠTAR šu-qu-ul
\nScore: -4.07
Eng[6:7] → [Akk 41:50]
\nEng: To Amur-Ištar: Urgent, pay attention and have the silver and its interest paid.\"
\nAkk: a-na a-mur-IŠTAR qi-bi₄-ma a-pu-tum i-ḫi-id KU.BABBAR u ṣi-ba-su ša-aš-qi-il₅
\nScore: -18.57
\nDoc Summary: 7 alignments, 0 NULLs, 0 merges
================================================================================
\nDoc 8: 2d1b0c9a-4403-4255-8198-406e3501a4ab
\nSentences: 11, Tokens: 81
\nEng[0:1] → [Akk 0:7]
\nEng: 4 talents 42 minas of copper with Iddin-Aurum.
\nAkk: 4 GU 42 ma-na URUDU iš-ti i-di-(d)MAR.TU
\nScore: -26.77
Eng[1:2] → [Akk 7:12]
\nEng: 0.5 mina of silver, working capital of Kitaya.
\nAkk: 0.5 ma-na KU.BABBAR be-u-la-at ki-ta-a
\nScore: -13.04
Eng[2:3] → [Akk 12:19]
\nEng: 8 minas of silver with Šuli, son of Puzur-Abeh.
\nAkk: 8 ma-na KU.BABBAR KI šu-li DUMU PUZUR₄-a-be-eḫ
\nScore: -3.77
Eng[3:4] → [Akk 19:34]
\nEng: 4 minas 3.83333 shekels of silver with Idnāya, month IV, eponymy of Šudāya; he will add 1 1 / 4 shekel per (mina per month) as interest.
\nAkk: 4 ma-na 3.83334 GIN KU.BABBAR KI id-na-a ITU.KAM ma-ḫu-ur-i-li li-mu-um šu-da-a 1.25 GIN.TA ṣi-ib-tam u-ṣa-ab
\nScore: -86.14
Eng[4:5] → [Akk 34:47]
\nEng: 1 talent 20 minas of copper and 25 shekels with Ibaya.
\nAkk: 1 GU 20 ma-na URUDU u 0.33333 ma-na 5 GIN KU.BABBAR KI i-ba-a
\nScore: -11.85
Eng[5:6] → [Akk 47:52]
\nEng: 8 shekels of silver with Ikūn-Mīšar.
\nAkk: 8 GIN KU.BABBAR KI i-ku-mi-šar
\nScore: -1.51
Eng[6:7] → [Akk 52:61]
\nEng: 2 minas 10 shekels of silver with Aššur-dūrī, year eponymy of Iphurum.
\nAkk: 2 ma-na 10 GIN KU.BABBAR KI a-šur-du-ri li-mu-um ip-ḫu-ru-um
\nScore: -17.49
Eng[7:8] → [Akk 61:65]
\nEng: He will add 1.5 shekel per (mina per month).
\nAkk: 1.5 GIN.TA ṣi-ib-tam u-ṣa-ab
\nScore: -40.22
Eng[8:9] → [Akk 65:70]
\nEng: 0.5 mina of silver with Balṭu'um.
\nAkk: 0.5 ma-na KU.BABBAR KI ba-al-ṭu-im
\nScore: -22.76
Eng[9:10] → [Akk 70:76]
\nEng: 25 shekels (of silver) working capital of Galdaya.
\nAkk: 0.33333 ma-na 5 GIN be-u-la-at gal-da-a
\nScore: -21.05
Eng[10:11] → [Akk 76:81]
\nEng: 1 mina of silver with Galdaya.
\nAkk: 1 ma-na KU.BABBAR KI gal-da-a
\nScore: -2.51
\nDoc Summary: 11 alignments, 0 NULLs, 0 merges
================================================================================
\nOVERALL SUMMARY
\n================================================================================
\nTotal alignments: 42
\nNULL alignments: 2 (4.8%)
\nMerged alignments: 0 (0.0%)
\nAverage score: -46.24
", "votes": null, "replies": [] }, { "id": 3388631, "author_name": "angantyr", "author_url": "", "post_date": "01/09/2026 09:25:09", "content": "Considering anchors, there are also the quantity-based fragments, e.g., 2 GÍN = 2 shekels, 5 ma-na = 5 minas, 30 GÚ = 30 talents.
", "votes": null, "replies": [ { "id": 3388725, "author_name": "aaronbornstein", "author_url": "", "post_date": "01/09/2026 13:29:29", "content": "The challenge with this though is that the units often change between the translation and source for example
\nEng: 25 shekels (of silver) working capital of Galdaya.
\nAkk: 0.33333 ma-na 5 GIN be-u-la-at gal-da-a
", "votes": null, "replies": [ { "id": 3388823, "author_name": "angantyr", "author_url": "", "post_date": "01/09/2026 16:52:15", "content": "Have you quantified the problem? I haven't seen that too often, so would think it's a rare case. If not, I'll add a section to the analysis notebook I'm working on.
", "votes": null, "replies": [] } ] }, { "id": 3388839, "author_name": "deeppast", "author_url": "", "post_date": "01/09/2026 17:14:16", "content": "There's a pretty good description of the Mesopotamian weight system here in ORACC (but their rules for formatting these numbers in transcription is different of course): https://oracc.museum.upenn.edu/doc/help/editinginatf/metrology/metrologicaltables/index.html#weight
\nIt is important to note that the fractions (which are retained as unicode fractions in the test data) were turned into decimals in the train data and in the published_texts.csv, as noted by Aaron's example:
\n\n\nEng: 25 shekels (of silver) working capital of Galdaya.
\nAkk: 0.33333 ma-na 5 GIN be-u-la-at gal-da-a
\n
So in this case, \"25 shekels\" would appear as: \"⅓ ma-na 5 GÍN\" in the test data.
", "votes": null, "replies": [ { "id": 3388900, "author_name": "angantyr", "author_url": "", "post_date": "01/09/2026 19:14:04", "content": "The question is though, is there an established norm that the translations follow? Or at the very least, if there are several rules is there one that stands out as the leading one?
\n\n\nAkk: 0.33333 ma-na 5 GIN be-u-la-at gal-da-a
\nEng: 25 shekels (of silver) working capital of Galdaya.
\n
What I would find intuitive is to save the original units in the translated text, i.e.:
\n\n\nAkk: ⅓ ma-na 5 GIN be-u-la-at gal-da-a
\nEng: ⅓ mina and 5 shekels (of silver) working capital of Galdaya.
\n
I don't know if that's the leading rule, though. Not for the hidden test, at least. I'll check it out for the train.csv
Yes, your intuitive approach would usually fit with the test set. The only exception is when a translator does the math in their head (e.g. 1 mina = 60 shekels, and the decimals reflect that it is usually divided into thirds) 1/3 mina + 5 shekels = 25 shekels
", "votes": null, "replies": [] } ] }, { "id": 3388964, "author_name": "aaronbornstein", "author_url": "", "post_date": "01/09/2026 22:18:38", "content": "Adding this to my method now lets see if it helps i definitely think this should enable better modeling and fewer bytes with BYT5
", "votes": null, "replies": [] } ] } ] } ], "raw_markdown_by_id": { "3388441": "I’m working on an English–Akkadian sentence alignment notebook aimed at creating higher-quality parallel training data for NMT in the context of the Deep Past Initiative challenge.\n\nhttps://www.kaggle.com/code/aaronbornstein/eng-akk-low-resource-sent-alignment\n\nThe goal is practical: get usable sentence-level alignments for a very low-resource, historically messy setting, where many standard tools don’t really apply.\n\n**Why this is hard**\n\n- Aligning Akkadian cuneiform with English translations breaks a lot of assumptions that common alignment methods rely on:\n\n- Damaged source text: Tablets are often fragmentary, with lacunae ([...]) and missing context.\n\n- Morphological mismatch: Akkadian is a morphologically rich Semitic language. A single Akkadian word can correspond to an entire English phrase, and word order differs a lot from English.\n\n- Data scarcity: We’re nowhere near the scale of modern parallel corpora used for contemporary MT.\n\n- No pretrained embeddings: There’s no BERT/RoBERTa-style model for Akkadian, which rules out embedding-based aligners like SimAlign.\n\n- Script complexity: The mix of syllabic writing and logograms (Sumerograms) makes straightforward lexical matching unreliable.\n\nIn practice, this means that off-the-shelf methods (e.g. Gale–Church) tend to fail pretty badly.\n\n**What I’m trying instead**\n\nThis notebook implements a custom unsupervised alignment pipeline designed around those constraints.\n\nA few key ideas:\n\n1. Normalized regret scoring\nInstead of scoring alignments purely by raw fit, each sentence is judged relative to its best possible alignment. The idea is to prevent “easy” sentences from soaking up all the alignment mass while damaged or complex sentences get dropped as NULL.\n\n2. Entity-based anchoring\nProper names, divine names, geographic names, numbers, and some Sumerograms are detected and used as high-confidence anchors. I’m also using simple phonetic matching (Soundex-style) to connect Akkadian transliterations with English spellings (e.g. Aššur ↔ Ashur).\n\n3. Monotonic global alignment\nThese anchors constrain a dynamic programming / Viterbi pass so that the final alignment path stays monotonic and globally consistent, rather than making lots of locally optimal but globally incoherent matches.\n\n**Current pipeline**\n\nAt a high level, the workflow is:\n\nA. Preprocessing: normalize Sumerograms, determinatives, and gaps\n\nB. Lexical induction: unsupervised word translation learning via IBM Model 1\n\nC. Scout phase: estimate an upper bound (“alignment ceiling”) for each sentence\n\nD. Global optimization: dynamic programming that maximizes total satisfaction under monotonic constraints\n\n**What I’m looking for**\n\nI’d really appreciate feedback from anyone who has worked on:\n\n- sentence alignment in low-resource or historical settings\n\n- alternatives to embedding-based alignment\n\n- better ways to handle one-to-many / many-to-one sentence mappings\n\nIf you see conceptual flaws, simpler alternatives, or places where this could be tightened up, I’d love to hear it. The notebook is open-source and very much a work in progress.\n\nThanks!", "3388442": "Sample Alignments from my Pipeline\n\n================================================================================\n\nALIGNMENT TEST: Documents from train.csv\n\n================================================================================\n\nLoaded 1561 documents from train.csv\n\n\n\n================================================================================\n\nDoc 1: d4140158-e25a-4b80-9148-baababa6fd4a\n\n Sentences: 7, Tokens: 86\n\n--------------------------------------------------------------------------------\n\n\n\n 1. Eng[0:1] → [Akk 0:9]\n\n Eng: To Aššur-imittī from Aššur-bāni: Why do you keep writing me about silver?\n\n Akk: a-na a-šur-i-mi-ti qi-bi-ma um-ma a-šur-ba-ni-ma mi-nu-um ša a-na KU.BABBAR\n\n Score: -37.26\n\n\n\n 2. Eng[1:2] → [Akk 9:26]\n\n Eng: You say: “Send me silver!” When you went away you left me 15 shekels of silver, and that was spent before Kurub-Ištar returned from Tawiniya.\n\n Akk: ta-aš-ta-na-pa-ra-ni um-ma a-ta-ma KU.BABBAR še-bi-lam i-nu-mi ta-ta-al-ku 15 GIN KU.BABBAR te-zi-ba-ma a-di kur-ub-IŠTAR iš-tu ta-wi-ni-a i-tu-ra-ni ga-me-er\n\n Score: -71.13\n\n\n\n 3. Eng[2:3] → [Akk 26:29]\n\n Eng: You said: “Go!\n\n Akk: um-ma a-ta-ma a-la-ak\n\n Score: -11.16\n\n\n\n 4. Eng[3:4] → [Akk 29:49]\n\n Eng: I shall send you tablets worth some 10 minas of silver, so you can have the silver paid, then set out and leave.” Instead of sending me tablets you warn me constantly whenever the merchant certified our tablets.\n\n Akk: ṭup-pe-e ša KU.BABBAR 10 ma-na u-še-ba-la-kum-ma KU.BABBAR ša-aš-qi-il₅-ma ti-ib-a-ma a-tal-kam ki-ma ṭup-pe-e še-bu-lim a-na ṭup-pi-ni ša i-ma-ta DAM.GAR iḫ-ri-mu tu-uš-ta-na-e-di\n\n Score: -175.69\n\n\n\n 5. Eng[4:5] → [Akk 49:61]\n\n Eng: And you know yourself that you left only 30 portions of food for me.\n\n Akk: u a-ta-ma ti-de₈-e ki-ma a-ka-al-tam 30 te-zi-ba-ni i-na qa-qi-ri-im da-nim a-nim a-a-kam\n\n Score: -59.85\n\n\n\n 6. Eng[5:6] → [NULL]\n\n Eng: From where shall I get anything so I can feed anyone under these harsh conditions?\n\n Score: -100.00\n\n\n\n 7. Eng[6:7] → [Akk 61:86]\n\n Eng: And I, as you know, I must feed and raise the son of Šu-Ištar, the maid and the slave, (and) you do not feed me, so I have become indebted for at least 1 mina of silver for the food for the household.\n\n Akk: la-al-qe-ma lu-ša-ki-il₅ a-na-ku wa-di a-ma DUMU šu-IŠTAR am-tam u IR ša u-ša-ku-lu u u-ra-bu-u-ni la tu ša-ki-li-ni-ma KU.BABBAR 1 ma-na a-na u-ku-ul-ti E be-tim a-ta-aḫ-ba-al\n\n Score: -147.24\n\n\n\n Doc Summary: 7 alignments, 1 NULLs, 0 merges\n\n\n\n================================================================================\n\nDoc 2: 23fbe6d7-8891-49f1-b812-415e01ebcdc4\n\n Sentences: 4, Tokens: 36\n\n--------------------------------------------------------------------------------\n\n\n\n 1. Eng[0:1] → [Akk 0:10]\n\n Eng: Seal of Aššur-šamšī, son of Šu-Ištar, seal of Aššur-lamassī, seal of Dān-Aššur, son of Kususārum.\n\n Akk: KIŠIB a-šur-(d)UTU-ši DUMU šu-IŠTAR KIŠIB a-šur-la-ma-si KIŠIB dan-a-šur DUMU ku-su-sa-ri-im\n\n Score: -12.52\n\n\n\n 2. Eng[1:2] → [Akk 10:18]\n\n Eng: