{
  "id": 663998,
  "title": "CDLI/ORACC overlap",
  "url": "/competitions/deep-past-initiative-machine-translation/discussion/663998",
  "author_name": "",
  "post_date": "2025-12-21T23:39:17.518949500Z",
  "votes": 2,
  "comment_count": 2,
  "views": 0,
  "content": "<p>Hi all,</p>\n<p>General question - do the test data exist in the CDLI or ORACC repositories at all? I'm worried about data leakage when training models. </p>",
  "messages": [
    {
      "id": "3380221",
      "postDate": "12/21/2025 23:39:17",
      "content": "<p>Hi all,</p>\n<p>General question - do the test data exist in the CDLI or ORACC repositories at all? I'm worried about data leakage when training models. </p>",
      "rawMarkdown": "Hi all,\n\nGeneral question - do the test data exist in the CDLI or ORACC repositories at all? I'm worried about data leakage when training models.",
      "votes": null
    },
    {
      "id": "3380226",
      "postDate": "12/22/2025 00:34:28",
      "content": "<p>Maybe not. By searching with theoare_id, I only found around 19 training-related entries on aicuneiform.com. Secondly, after adding other data for training, my scores have barely improved. Although I'm not sure if it's because I didn't handle the additional data properly.</p>",
      "rawMarkdown": "Maybe not. By searching with theoare_id, I only found around 19 training-related entries on aicuneiform.com. Secondly, after adding other data for training, my scores have barely improved. Although I'm not sure if it's because I didn't handle the additional data properly.",
      "votes": null
    },
    {
      "id": "3381032",
      "postDate": "12/23/2025 16:02:46",
      "content": "<p>No, not at all. We did not use any published texts for the test data. We made sure of that. Also, there are no Old Assyrian texts in Oracc at this time. The CDLI has a little less than 5000 texts, which are all published. </p>",
      "rawMarkdown": "No, not at all. We did not use any published texts for the test data. We made sure of that. Also, there are no Old Assyrian texts in Oracc at this time. The CDLI has a little less than 5000 texts, which are all published.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 3380226,
      "author_name": "qifeihhh666",
      "author_url": "",
      "post_date": "12/22/2025 00:34:28",
      "content": "<p>Maybe not. By searching with theoare_id, I only found around 19 training-related entries on aicuneiform.com. Secondly, after adding other data for training, my scores have barely improved. Although I'm not sure if it's because I didn't handle the additional data properly.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 3381032,
      "author_name": "deeppast",
      "author_url": "",
      "post_date": "12/23/2025 16:02:46",
      "content": "<p>No, not at all. We did not use any published texts for the test data. We made sure of that. Also, there are no Old Assyrian texts in Oracc at this time. The CDLI has a little less than 5000 texts, which are all published. </p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "3380221": "Hi all,\n\nGeneral question - do the test data exist in the CDLI or ORACC repositories at all? I'm worried about data leakage when training models.",
    "3380226": "Maybe not. By searching with theoare_id, I only found around 19 training-related entries on aicuneiform.com. Secondly, after adding other data for training, my scores have barely improved. Although I'm not sure if it's because I didn't handle the additional data properly.",
    "3381032": "No, not at all. We did not use any published texts for the test data. We made sure of that. Also, there are no Old Assyrian texts in Oracc at this time. The CDLI has a little less than 5000 texts, which are all published."
  },
  "source": "meta"
}