{
  "id": 665102,
  "title": "Finetuned model generates repeated gibberish",
  "url": "/competitions/deep-past-initiative-machine-translation/discussion/665102",
  "author_name": "",
  "post_date": "2025-12-30T06:16:51.580662300Z",
  "votes": 5,
  "comment_count": 3,
  "views": 0,
  "content": "<p>My finetuned ByT5 model generates translations like, when setting num_beam to &gt; 1.  any one knows why?</p>\n<pre><code>'From Kanesh to Kanesh to Kanesh to Ali-ahum from the City and the City and the City and the City and the City and the City and the City and the City and the City and the City and the City and the City and the City and the City and the City and the City and the City and the City and the City and the City and the City and the City and the City and the City and the City and the City and the City and the City and the City and the City and the City and the City and the City and the City and the City and the City'\n</code></pre>",
  "messages": [
    {
      "id": "3383437",
      "postDate": "12/30/2025 06:16:51",
      "content": "<p>My finetuned ByT5 model generates translations like, when setting num_beam to &gt; 1.  any one knows why?</p>\n<pre><code>'From Kanesh to Kanesh to Kanesh to Ali-ahum from the City and the City and the City and the City and the City and the City and the City and the City and the City and the City and the City and the City and the City and the City and the City and the City and the City and the City and the City and the City and the City and the City and the City and the City and the City and the City and the City and the City and the City and the City and the City and the City and the City and the City and the City and the City'\n</code></pre>",
      "rawMarkdown": "My finetuned ByT5 model generates translations like, when setting num_beam to > 1.  any one knows why?\n```\n'From Kanesh to Kanesh to Kanesh to Ali-ahum from the City and the City and the City and the City and the City and the City and the City and the City and the City and the City and the City and the City and the City and the City and the City and the City and the City and the City and the City and the City and the City and the City and the City and the City and the City and the City and the City and the City and the City and the City and the City and the City and the City and the City and the City and the City'\n```",
      "votes": null
    },
    {
      "id": "3383462",
      "postDate": "12/30/2025 07:54:05",
      "content": "<p>This is my idea.Lower the <strong>learning rate</strong> a bit and reduce the number of <strong>epochs</strong>.</p>",
      "rawMarkdown": "This is my idea.Lower the **learning rate** a bit and reduce the number of **epochs**.",
      "votes": null
    },
    {
      "id": "3383467",
      "postDate": "12/30/2025 08:13:34",
      "content": "<p>let me try thanks</p>",
      "rawMarkdown": "let me try thanks",
      "votes": null
    },
    {
      "id": "3383664",
      "postDate": "12/30/2025 15:56:28",
      "content": "<p>While I can't say exactly why the model is apparently hallucinating, my suspicion is that it has to do with the named entities (especially personal names) in the texts not having a robust embedding. Since the introduction of these texts includes personal names, it would make sense to me that without the ability to properly predict these names, the model is unable to predict any subsequent words either.</p>",
      "rawMarkdown": "While I can't say exactly why the model is apparently hallucinating, my suspicion is that it has to do with the named entities (especially personal names) in the texts not having a robust embedding. Since the introduction of these texts includes personal names, it would make sense to me that without the ability to properly predict these names, the model is unable to predict any subsequent words either.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 3383462,
      "author_name": "qifeihhh666",
      "author_url": "",
      "post_date": "12/30/2025 07:54:05",
      "content": "<p>This is my idea.Lower the <strong>learning rate</strong> a bit and reduce the number of <strong>epochs</strong>.</p>",
      "votes": null,
      "replies": [
        {
          "id": 3383467,
          "author_name": "zhangyue199",
          "author_url": "",
          "post_date": "12/30/2025 08:13:34",
          "content": "<p>let me try thanks</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 3383664,
      "author_name": "deeppast",
      "author_url": "",
      "post_date": "12/30/2025 15:56:28",
      "content": "<p>While I can't say exactly why the model is apparently hallucinating, my suspicion is that it has to do with the named entities (especially personal names) in the texts not having a robust embedding. Since the introduction of these texts includes personal names, it would make sense to me that without the ability to properly predict these names, the model is unable to predict any subsequent words either.</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "3383437": "My finetuned ByT5 model generates translations like, when setting num_beam to > 1.  any one knows why?\n```\n'From Kanesh to Kanesh to Kanesh to Ali-ahum from the City and the City and the City and the City and the City and the City and the City and the City and the City and the City and the City and the City and the City and the City and the City and the City and the City and the City and the City and the City and the City and the City and the City and the City and the City and the City and the City and the City and the City and the City and the City and the City and the City and the City and the City and the City'\n```",
    "3383462": "This is my idea.Lower the **learning rate** a bit and reduce the number of **epochs**.",
    "3383467": "let me try thanks",
    "3383664": "While I can't say exactly why the model is apparently hallucinating, my suspicion is that it has to do with the named entities (especially personal names) in the texts not having a robust embedding. Since the introduction of these texts includes personal names, it would make sense to me that without the ability to properly predict these names, the model is unable to predict any subsequent words either."
  },
  "source": "meta"
}