{
  "id": 679497,
  "title": "Lora on ByT5 large",
  "url": "/competitions/deep-past-initiative-machine-translation/discussion/679497",
  "author_name": "",
  "post_date": "2026-03-01T18:25:05.944665100Z",
  "votes": 4,
  "comment_count": 1,
  "views": 0,
  "content": "<p>Just wanted to try out lora on a model bigger than ByT5-small, didn't really work well, the best submission caps out at 18.0 . \nNotebook - <a href=\"https://www.kaggle.com/code/pshikk/ffn-lora-deep-past\" target=\"_blank\">https://www.kaggle.com/code/pshikk/ffn-lora-deep-past</a>\nOutputs - <a href=\"https://www.kaggle.com/datasets/pshikk/lora-5-byte\" target=\"_blank\">https://www.kaggle.com/datasets/pshikk/lora-5-byte</a></p>",
  "messages": [
    {
      "id": "3415975",
      "postDate": "03/01/2026 18:25:05",
      "content": "<p>Just wanted to try out lora on a model bigger than ByT5-small, didn't really work well, the best submission caps out at 18.0 . \nNotebook - <a href=\"https://www.kaggle.com/code/pshikk/ffn-lora-deep-past\" target=\"_blank\">https://www.kaggle.com/code/pshikk/ffn-lora-deep-past</a>\nOutputs - <a href=\"https://www.kaggle.com/datasets/pshikk/lora-5-byte\" target=\"_blank\">https://www.kaggle.com/datasets/pshikk/lora-5-byte</a></p>",
      "rawMarkdown": "Just wanted to try out lora on a model bigger than ByT5-small, didn't really work well, the best submission caps out at 18.0 . \nNotebook - https://www.kaggle.com/code/pshikk/ffn-lora-deep-past\nOutputs - https://www.kaggle.com/datasets/pshikk/lora-5-byte",
      "votes": null
    },
    {
      "id": "3417009",
      "postDate": "03/04/2026 10:23:47",
      "content": "<p>LoRA is best for adapting or specializing a pretrained model. It can learn new behaviors, but teaching completely new knowledge or capabilities is much harder than with full fine tuning</p>",
      "rawMarkdown": "LoRA is best for adapting or specializing a pretrained model. It can learn new behaviors, but teaching completely new knowledge or capabilities is much harder than with full fine tuning",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 3417009,
      "author_name": "adarsh2626",
      "author_url": "",
      "post_date": "03/04/2026 10:23:47",
      "content": "<p>LoRA is best for adapting or specializing a pretrained model. It can learn new behaviors, but teaching completely new knowledge or capabilities is much harder than with full fine tuning</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "3415975": "Just wanted to try out lora on a model bigger than ByT5-small, didn't really work well, the best submission caps out at 18.0 . \nNotebook - https://www.kaggle.com/code/pshikk/ffn-lora-deep-past\nOutputs - https://www.kaggle.com/datasets/pshikk/lora-5-byte",
    "3417009": "LoRA is best for adapting or specializing a pretrained model. It can learn new behaviors, but teaching completely new knowledge or capabilities is much harder than with full fine tuning"
  },
  "source": "meta"
}