{
  "id": 419056,
  "title": "Finetune a LLM on 2 GPUs",
  "url": "/competitions/cafa-5-protein-function-prediction/discussion/419056",
  "author_name": "Simon Veitner",
  "post_date": "2023-06-23T22:05:14.611000",
  "votes": 5,
  "comment_count": 0,
  "views": 0,
  "content": "<p>I published the following <a href=\"https://www.kaggle.com/simonveitner/finetuning-on-2-gpus\" target=\"_blank\">code</a> which implements a finetuning of a protein language model. for now i used the smallest esm. without any finetuning (i left the hyperparameters as they were in the notebook i modified from other competitions) and it archieves from training on 1 fold an LB of 0.329.</p>\n<p>Now we can start to tune hyperparmeters, use different models etc etc.<br>\nIf somebody finds a mistake in the notebook, please let me know as i am not super experienced.<br>\nAlso of course i would be happy if you find something interesting and report here.</p>",
  "messages": [
    {
      "id": 2315181,
      "postDate": "2023-06-23T22:05:14.610Z",
      "content": "<p>I published the following <a href=\"https://www.kaggle.com/simonveitner/finetuning-on-2-gpus\" target=\"_blank\">code</a> which implements a finetuning of a protein language model. for now i used the smallest esm. without any finetuning (i left the hyperparameters as they were in the notebook i modified from other competitions) and it archieves from training on 1 fold an LB of 0.329.</p>\n<p>Now we can start to tune hyperparmeters, use different models etc etc.<br>\nIf somebody finds a mistake in the notebook, please let me know as i am not super experienced.<br>\nAlso of course i would be happy if you find something interesting and report here.</p>",
      "rawMarkdown": "I published the following [code](https://www.kaggle.com/simonveitner/finetuning-on-2-gpus) which implements a finetuning of a protein language model. for now i used the smallest esm. without any finetuning (i left the hyperparameters as they were in the notebook i modified from other competitions) and it archieves from training on 1 fold an LB of 0.329.\n\nNow we can start to tune hyperparmeters, use different models etc etc.\nIf somebody finds a mistake in the notebook, please let me know as i am not super experienced.\nAlso of course i would be happy if you find something interesting and report here.",
      "votes": 5
    }
  ],
  "comments": [],
  "raw_markdown_by_id": {
    "2315181": "I published the following [code](https://www.kaggle.com/simonveitner/finetuning-on-2-gpus) which implements a finetuning of a protein language model. for now i used the smallest esm. without any finetuning (i left the hyperparameters as they were in the notebook i modified from other competitions) and it archieves from training on 1 fold an LB of 0.329.\n\nNow we can start to tune hyperparmeters, use different models etc etc.\nIf somebody finds a mistake in the notebook, please let me know as i am not super experienced.\nAlso of course i would be happy if you find something interesting and report here."
  }
}