{
  "id": 446936,
  "title": "167th Place Solution [Deberta Large with LoRA]",
  "content": "<h3>Submission Overview</h3>\n<table>\n<thead>\n<tr>\n<th>Models</th>\n<th>CV</th>\n<th>Public</th>\n<th>Private</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>Deberta base + lgb</td>\n<td>508</td>\n<td>447</td>\n<td>485</td>\n</tr>\n<tr>\n<td>Deberta large LoRA + lgb</td>\n<td>501</td>\n<td>441</td>\n<td>495</td>\n</tr>\n<tr>\n<td>Result blend + lgb</td>\n<td>492</td>\n<td>435</td>\n<td>482</td>\n</tr>\n</tbody>\n</table>\n<p><br></p>\n<h3>Things which didn't work</h3>\n<p>Fine-tuning like GPT models for regression<br>\nSFT Llama for direct target generation<br>\nAdditional data from ChatGPT, SFT Llama2<br>\n<br></p>\n<h3>Example code how to fine-tuning with LoRA</h3>\n<p><a href=\"https://www.kaggle.com/code/alexcei/fine-tuning-deberta-large-with-lora-adapter\" target=\"_blank\">fine-tuning-deberta-large-with-lora-adapter</a></p>",
  "comments": [
    {
      "id": 2481280,
      "author_name": "DL",
      "author_url": "",
      "post_date": "2023-10-14T02:08:07.377000",
      "content": "<p>thanks for sharing，but maybe additional data from ChatGPT is against the rules</p>",
      "votes": 1,
      "replies": [
        {
          "id": 2481593,
          "author_name": "Somesh88",
          "author_url": "",
          "post_date": "2023-10-14T08:52:10.137000",
          "content": "<p>I think the solution shared don't have use of gpt generated data. </p>",
          "votes": 1,
          "replies": []
        }
      ]
    },
    {
      "id": 2481151,
      "author_name": "Nissim Matatov",
      "author_url": "",
      "post_date": "2023-10-13T20:58:08.520000",
      "content": "<p>Thank you for the code . I think deberta-large with lora this a fundamental technique to be used  in future competitions</p>",
      "votes": 1,
      "replies": []
    }
  ],
  "comment_count": 3,
  "messages": [
    {
      "id": 2480926,
      "postDate": "2023-10-13T16:32:48.403Z",
      "content": "<h3>Submission Overview</h3>\n<table>\n<thead>\n<tr>\n<th>Models</th>\n<th>CV</th>\n<th>Public</th>\n<th>Private</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>Deberta base + lgb</td>\n<td>508</td>\n<td>447</td>\n<td>485</td>\n</tr>\n<tr>\n<td>Deberta large LoRA + lgb</td>\n<td>501</td>\n<td>441</td>\n<td>495</td>\n</tr>\n<tr>\n<td>Result blend + lgb</td>\n<td>492</td>\n<td>435</td>\n<td>482</td>\n</tr>\n</tbody>\n</table>\n<p><br></p>\n<h3>Things which didn't work</h3>\n<p>Fine-tuning like GPT models for regression<br>\nSFT Llama for direct target generation<br>\nAdditional data from ChatGPT, SFT Llama2<br>\n<br></p>\n<h3>Example code how to fine-tuning with LoRA</h3>\n<p><a href=\"https://www.kaggle.com/code/alexcei/fine-tuning-deberta-large-with-lora-adapter\" target=\"_blank\">fine-tuning-deberta-large-with-lora-adapter</a></p>",
      "rawMarkdown": "### Submission Overview\n| Models | CV | Public | Private |\n| --- | --- | --- |\n| Deberta base + lgb | 508 | 447 | 485 |\n| Deberta large LoRA + lgb | 501 | 441 | 495 |\n| Result blend + lgb | 492 | 435 | 482 |\n\n<br>\n### Things which didn't work\nFine-tuning like GPT models for regression\nSFT Llama for direct target generation\nAdditional data from ChatGPT, SFT Llama2\n<br>\n### Example code how to fine-tuning with LoRA\n[fine-tuning-deberta-large-with-lora-adapter](https://www.kaggle.com/code/alexcei/fine-tuning-deberta-large-with-lora-adapter)",
      "votes": 2
    },
    {
      "id": 2481280,
      "postDate": "2023-10-14T02:08:07.377Z",
      "content": "<p>thanks for sharing，but maybe additional data from ChatGPT is against the rules</p>",
      "rawMarkdown": "thanks for sharing，but maybe additional data from ChatGPT is against the rules",
      "votes": 1,
      "replies": [
        {
          "id": 2481593,
          "postDate": "2023-10-14T08:52:10.137Z",
          "content": "<p>I think the solution shared don't have use of gpt generated data. </p>",
          "rawMarkdown": "I think the solution shared don't have use of gpt generated data. ",
          "votes": 1
        }
      ]
    },
    {
      "id": 2481151,
      "postDate": "2023-10-13T20:58:08.520Z",
      "content": "<p>Thank you for the code . I think deberta-large with lora this a fundamental technique to be used  in future competitions</p>",
      "rawMarkdown": "Thank you for the code . I think deberta-large with lora this a fundamental technique to be used  in future competitions",
      "votes": 1
    }
  ],
  "complete": true,
  "source": "live",
  "fetched_at": "2026-10-19T23:05:45Z",
  "competition_id": 53482,
  "competition_slug": "commonlit-evaluate-student-summaries",
  "team_id": 10685867,
  "solution_url": "/competitions/commonlit-evaluate-student-summaries/writeups/alexey-latskov-167th-place-solution-deberta-large-"
}