{
  "id": 683352,
  "title": "OA UnifiedGloss",
  "url": "/competitions/deep-past-initiative-machine-translation/discussion/683352",
  "author_name": "",
  "post_date": "2026-03-20T11:05:34.010655500Z",
  "votes": 2,
  "comment_count": 1,
  "views": 0,
  "content": "<p>I worked a lot of hours cleaning data, I transformed books and articles into rules and examples to train a model. I believe this is the right way to create a hybrid system like <a href=\"https://github.com/asahala/BabyLemmatizer\" target=\"_blank\">BabyLemmatizer</a>, but using SQLite, LLM, and Python. I only know a little (maybe nothing)  about Old Assyrian and other cuneiform languages, but this seems to be the professional method (BabyLemmatizer).</p>\n<p>I've included some in the comments; in other cases, I've simply marked them as \"public.\" All the original (raw) sources are public; this is a compilation of public sources and synthetic data generated from them.</p>\n<p>DISCLAIMER: I'm sharing all the datasets tested during the process, not necessarily the ones I used. And most importantly: it's necessary to understand that this is synthetic data, and the data is not generated by a PhD. </p>\n<ul>\n<li><a href=\"https://www.kaggle.com/datasets/davidochoacorrales/unifiedgloss-4-oare-all-words\" target=\"_blank\">UnifiedGloss 4: OARE all words</a></li>\n<li><a href=\"https://www.kaggle.com/datasets/davidochoacorrales/unifiedgloss-7-unified-ebl-cot\" target=\"_blank\">UnifiedGloss 7: unified eBL CoT</a></li>\n<li><a href=\"https://www.kaggle.com/datasets/davidochoacorrales/unifiedgloss-11-all-data-signs\" target=\"_blank\">UnifiedGloss 11: All data signs</a></li>\n<li><a href=\"https://www.kaggle.com/datasets/davidochoacorrales/unifiedgloss-15-limmu-oficials\" target=\"_blank\">UnifiedGloss 15: Limmu Oficials</a></li>\n<li><a href=\"https://www.kaggle.com/datasets/davidochoacorrales/unifiedgloss-14-kltepe-people-networking\" target=\"_blank\">UnifiedGloss 14: Kültepe People - Networking</a></li>\n<li><a href=\"https://www.kaggle.com/datasets/davidochoacorrales/unifiedgloss-13-kltepe-people-networking\" target=\"_blank\">UnifiedGloss 13: Kültepe People - Networking</a></li>\n<li><a href=\"https://www.kaggle.com/datasets/davidochoacorrales/unifiedgloss-12-kltepe-people-networking\" target=\"_blank\">UnifiedGloss 12: Kültepe People - Networking</a></li>\n<li><a href=\"https://www.kaggle.com/datasets/davidochoacorrales/the-oare-db\" target=\"_blank\">UnifiedGloss 1: The OARE DB</a></li>\n<li><a href=\"https://www.kaggle.com/datasets/davidochoacorrales/oa-entrada-lxica-estructurada-de-un-signo\" target=\"_blank\">Cuneiform Sign JSON Structure (OARE)</a></li>\n<li><a href=\"https://www.kaggle.com/datasets/davidochoacorrales/oa-date-patterns\" target=\"_blank\">Oa date patterns</a> - Perdido</li>\n<li><a href=\"https://www.kaggle.com/datasets/davidochoacorrales/oa-limmu-officials\" target=\"_blank\">Oa limmu officials</a> - copia del original</li>\n<li><a href=\"https://www.kaggle.com/datasets/davidochoacorrales/glossary\" target=\"_blank\">UnifiedGloss 2: _Glossary | Kouwenberg</a></li>\n<li><a href=\"https://www.kaggle.com/datasets/davidochoacorrales/oare-tokens\" target=\"_blank\">Tokens | OARE</a> ALL Vocal Syllabes &amp; Sings observed at OARE DB</li>\n<li><a href=\"https://www.kaggle.com/datasets/davidochoacorrales/day-unit-within-the-calendar-ccile-michel\" target=\"_blank\">Day Unit within the Calendar | Cécile Michel</a></li>\n</ul>\n<p>more</p>\n<ul>\n<li><a href=\"https://www.kaggle.com/datasets/davidochoacorrales/unified-gloss-15-oare-phrases\" target=\"_blank\">Unified Gloss 15 OARE_Phrases</a></li>\n<li><a href=\"https://www.kaggle.com/datasets/davidochoacorrales/veenhof-metrology\" target=\"_blank\">Veenhof Metrology</a></li>\n<li><a href=\"https://www.kaggle.com/datasets/davidochoacorrales/weighing-units-and-weights-ccile-michel\" target=\"_blank\">Weighing Units and Weights | Cécile Michel</a></li>\n<li><a href=\"https://www.kaggle.com/datasets/davidochoacorrales/unifiedgloss-10-glosasry-signs\" target=\"_blank\">UnifiedGloss 10: glossary Signs</a></li>\n</ul>\n<p>I loose a lot of data like all the grammar rules…</p>",
  "messages": [
    {
      "id": "3425099",
      "postDate": "03/20/2026 11:05:34",
      "content": "<p>I worked a lot of hours cleaning data, I transformed books and articles into rules and examples to train a model. I believe this is the right way to create a hybrid system like <a href=\"https://github.com/asahala/BabyLemmatizer\" target=\"_blank\">BabyLemmatizer</a>, but using SQLite, LLM, and Python. I only know a little (maybe nothing)  about Old Assyrian and other cuneiform languages, but this seems to be the professional method (BabyLemmatizer).</p>\n<p>I've included some in the comments; in other cases, I've simply marked them as \"public.\" All the original (raw) sources are public; this is a compilation of public sources and synthetic data generated from them.</p>\n<p>DISCLAIMER: I'm sharing all the datasets tested during the process, not necessarily the ones I used. And most importantly: it's necessary to understand that this is synthetic data, and the data is not generated by a PhD. </p>\n<ul>\n<li><a href=\"https://www.kaggle.com/datasets/davidochoacorrales/unifiedgloss-4-oare-all-words\" target=\"_blank\">UnifiedGloss 4: OARE all words</a></li>\n<li><a href=\"https://www.kaggle.com/datasets/davidochoacorrales/unifiedgloss-7-unified-ebl-cot\" target=\"_blank\">UnifiedGloss 7: unified eBL CoT</a></li>\n<li><a href=\"https://www.kaggle.com/datasets/davidochoacorrales/unifiedgloss-11-all-data-signs\" target=\"_blank\">UnifiedGloss 11: All data signs</a></li>\n<li><a href=\"https://www.kaggle.com/datasets/davidochoacorrales/unifiedgloss-15-limmu-oficials\" target=\"_blank\">UnifiedGloss 15: Limmu Oficials</a></li>\n<li><a href=\"https://www.kaggle.com/datasets/davidochoacorrales/unifiedgloss-14-kltepe-people-networking\" target=\"_blank\">UnifiedGloss 14: Kültepe People - Networking</a></li>\n<li><a href=\"https://www.kaggle.com/datasets/davidochoacorrales/unifiedgloss-13-kltepe-people-networking\" target=\"_blank\">UnifiedGloss 13: Kültepe People - Networking</a></li>\n<li><a href=\"https://www.kaggle.com/datasets/davidochoacorrales/unifiedgloss-12-kltepe-people-networking\" target=\"_blank\">UnifiedGloss 12: Kültepe People - Networking</a></li>\n<li><a href=\"https://www.kaggle.com/datasets/davidochoacorrales/the-oare-db\" target=\"_blank\">UnifiedGloss 1: The OARE DB</a></li>\n<li><a href=\"https://www.kaggle.com/datasets/davidochoacorrales/oa-entrada-lxica-estructurada-de-un-signo\" target=\"_blank\">Cuneiform Sign JSON Structure (OARE)</a></li>\n<li><a href=\"https://www.kaggle.com/datasets/davidochoacorrales/oa-date-patterns\" target=\"_blank\">Oa date patterns</a> - Perdido</li>\n<li><a href=\"https://www.kaggle.com/datasets/davidochoacorrales/oa-limmu-officials\" target=\"_blank\">Oa limmu officials</a> - copia del original</li>\n<li><a href=\"https://www.kaggle.com/datasets/davidochoacorrales/glossary\" target=\"_blank\">UnifiedGloss 2: _Glossary | Kouwenberg</a></li>\n<li><a href=\"https://www.kaggle.com/datasets/davidochoacorrales/oare-tokens\" target=\"_blank\">Tokens | OARE</a> ALL Vocal Syllabes &amp; Sings observed at OARE DB</li>\n<li><a href=\"https://www.kaggle.com/datasets/davidochoacorrales/day-unit-within-the-calendar-ccile-michel\" target=\"_blank\">Day Unit within the Calendar | Cécile Michel</a></li>\n</ul>\n<p>more</p>\n<ul>\n<li><a href=\"https://www.kaggle.com/datasets/davidochoacorrales/unified-gloss-15-oare-phrases\" target=\"_blank\">Unified Gloss 15 OARE_Phrases</a></li>\n<li><a href=\"https://www.kaggle.com/datasets/davidochoacorrales/veenhof-metrology\" target=\"_blank\">Veenhof Metrology</a></li>\n<li><a href=\"https://www.kaggle.com/datasets/davidochoacorrales/weighing-units-and-weights-ccile-michel\" target=\"_blank\">Weighing Units and Weights | Cécile Michel</a></li>\n<li><a href=\"https://www.kaggle.com/datasets/davidochoacorrales/unifiedgloss-10-glosasry-signs\" target=\"_blank\">UnifiedGloss 10: glossary Signs</a></li>\n</ul>\n<p>I loose a lot of data like all the grammar rules…</p>",
      "rawMarkdown": "I worked a lot of hours cleaning data, I transformed books and articles into rules and examples to train a model. I believe this is the right way to create a hybrid system like [BabyLemmatizer](https://github.com/asahala/BabyLemmatizer), but using SQLite, LLM, and Python. I only know a little (maybe nothing)  about Old Assyrian and other cuneiform languages, but this seems to be the professional method (BabyLemmatizer).\n\nI've included some in the comments; in other cases, I've simply marked them as \"public.\" All the original (raw) sources are public; this is a compilation of public sources and synthetic data generated from them.\n\nDISCLAIMER: I'm sharing all the datasets tested during the process, not necessarily the ones I used. And most importantly: it's necessary to understand that this is synthetic data, and the data is not generated by a PhD. \n\n- [UnifiedGloss 4: OARE all words](https://www.kaggle.com/datasets/davidochoacorrales/unifiedgloss-4-oare-all-words)\n- [UnifiedGloss 7: unified eBL CoT](https://www.kaggle.com/datasets/davidochoacorrales/unifiedgloss-7-unified-ebl-cot)\n- [UnifiedGloss 11: All data signs](https://www.kaggle.com/datasets/davidochoacorrales/unifiedgloss-11-all-data-signs)\n- [UnifiedGloss 15: Limmu Oficials](https://www.kaggle.com/datasets/davidochoacorrales/unifiedgloss-15-limmu-oficials)\n- [UnifiedGloss 14: Kültepe People - Networking](https://www.kaggle.com/datasets/davidochoacorrales/unifiedgloss-14-kltepe-people-networking)\n- [UnifiedGloss 13: Kültepe People - Networking](https://www.kaggle.com/datasets/davidochoacorrales/unifiedgloss-13-kltepe-people-networking)\n- [UnifiedGloss 12: Kültepe People - Networking](https://www.kaggle.com/datasets/davidochoacorrales/unifiedgloss-12-kltepe-people-networking)\n- [UnifiedGloss 1: The OARE DB](https://www.kaggle.com/datasets/davidochoacorrales/the-oare-db)\n- [Cuneiform Sign JSON Structure (OARE)](https://www.kaggle.com/datasets/davidochoacorrales/oa-entrada-lxica-estructurada-de-un-signo)\n- [Oa date patterns](https://www.kaggle.com/datasets/davidochoacorrales/oa-date-patterns) - Perdido\n- [Oa limmu officials](https://www.kaggle.com/datasets/davidochoacorrales/oa-limmu-officials) - copia del original\n- [UnifiedGloss 2: _Glossary | Kouwenberg](https://www.kaggle.com/datasets/davidochoacorrales/glossary)\n- [Tokens | OARE](https://www.kaggle.com/datasets/davidochoacorrales/oare-tokens) ALL Vocal Syllabes & Sings observed at OARE DB\n- [Day Unit within the Calendar | Cécile Michel](https://www.kaggle.com/datasets/davidochoacorrales/day-unit-within-the-calendar-ccile-michel)\n\nmore\n\n- [Unified Gloss 15 OARE_Phrases](https://www.kaggle.com/datasets/davidochoacorrales/unified-gloss-15-oare-phrases)\n- [Veenhof Metrology](https://www.kaggle.com/datasets/davidochoacorrales/veenhof-metrology)\n- [Weighing Units and Weights | Cécile Michel](https://www.kaggle.com/datasets/davidochoacorrales/weighing-units-and-weights-ccile-michel)\n- [UnifiedGloss 10: glossary Signs](https://www.kaggle.com/datasets/davidochoacorrales/unifiedgloss-10-glosasry-signs)\n\nI loose a lot of data like all the grammar rules...",
      "votes": null
    },
    {
      "id": "3425941",
      "postDate": "03/21/2026 15:09:29",
      "content": "<p>I added more datasets</p>",
      "rawMarkdown": "I added more datasets",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 3425941,
      "author_name": "davidochoacorrales",
      "author_url": "",
      "post_date": "03/21/2026 15:09:29",
      "content": "<p>I added more datasets</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "3425099": "I worked a lot of hours cleaning data, I transformed books and articles into rules and examples to train a model. I believe this is the right way to create a hybrid system like [BabyLemmatizer](https://github.com/asahala/BabyLemmatizer), but using SQLite, LLM, and Python. I only know a little (maybe nothing)  about Old Assyrian and other cuneiform languages, but this seems to be the professional method (BabyLemmatizer).\n\nI've included some in the comments; in other cases, I've simply marked them as \"public.\" All the original (raw) sources are public; this is a compilation of public sources and synthetic data generated from them.\n\nDISCLAIMER: I'm sharing all the datasets tested during the process, not necessarily the ones I used. And most importantly: it's necessary to understand that this is synthetic data, and the data is not generated by a PhD. \n\n- [UnifiedGloss 4: OARE all words](https://www.kaggle.com/datasets/davidochoacorrales/unifiedgloss-4-oare-all-words)\n- [UnifiedGloss 7: unified eBL CoT](https://www.kaggle.com/datasets/davidochoacorrales/unifiedgloss-7-unified-ebl-cot)\n- [UnifiedGloss 11: All data signs](https://www.kaggle.com/datasets/davidochoacorrales/unifiedgloss-11-all-data-signs)\n- [UnifiedGloss 15: Limmu Oficials](https://www.kaggle.com/datasets/davidochoacorrales/unifiedgloss-15-limmu-oficials)\n- [UnifiedGloss 14: Kültepe People - Networking](https://www.kaggle.com/datasets/davidochoacorrales/unifiedgloss-14-kltepe-people-networking)\n- [UnifiedGloss 13: Kültepe People - Networking](https://www.kaggle.com/datasets/davidochoacorrales/unifiedgloss-13-kltepe-people-networking)\n- [UnifiedGloss 12: Kültepe People - Networking](https://www.kaggle.com/datasets/davidochoacorrales/unifiedgloss-12-kltepe-people-networking)\n- [UnifiedGloss 1: The OARE DB](https://www.kaggle.com/datasets/davidochoacorrales/the-oare-db)\n- [Cuneiform Sign JSON Structure (OARE)](https://www.kaggle.com/datasets/davidochoacorrales/oa-entrada-lxica-estructurada-de-un-signo)\n- [Oa date patterns](https://www.kaggle.com/datasets/davidochoacorrales/oa-date-patterns) - Perdido\n- [Oa limmu officials](https://www.kaggle.com/datasets/davidochoacorrales/oa-limmu-officials) - copia del original\n- [UnifiedGloss 2: _Glossary | Kouwenberg](https://www.kaggle.com/datasets/davidochoacorrales/glossary)\n- [Tokens | OARE](https://www.kaggle.com/datasets/davidochoacorrales/oare-tokens) ALL Vocal Syllabes & Sings observed at OARE DB\n- [Day Unit within the Calendar | Cécile Michel](https://www.kaggle.com/datasets/davidochoacorrales/day-unit-within-the-calendar-ccile-michel)\n\nmore\n\n- [Unified Gloss 15 OARE_Phrases](https://www.kaggle.com/datasets/davidochoacorrales/unified-gloss-15-oare-phrases)\n- [Veenhof Metrology](https://www.kaggle.com/datasets/davidochoacorrales/veenhof-metrology)\n- [Weighing Units and Weights | Cécile Michel](https://www.kaggle.com/datasets/davidochoacorrales/weighing-units-and-weights-ccile-michel)\n- [UnifiedGloss 10: glossary Signs](https://www.kaggle.com/datasets/davidochoacorrales/unifiedgloss-10-glosasry-signs)\n\nI loose a lot of data like all the grammar rules...",
    "3425941": "I added more datasets"
  },
  "source": "meta"
}