{
  "id": 371602,
  "title": "37th place solution",
  "content": "<p>First of all, thanks to the host for an interesting competition and congratulations to all the winners!</p>\n<p><strong>Overview</strong></p>\n<ul>\n<li>Our final submission is ensemble of 13 models. The weights of the models are determined using nelder-mead.</li>\n<li>cv0.44384/PublicLB0.435426/PrivateLB0.435417</li>\n</ul>\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F7393510%2F297a629de174ed1245d6fb524837a9ab%2F324.png?generation=1670742662019516&amp;alt=media\" alt=\"\"></p>\n<p><strong>What Worked</strong></p>\n<ul>\n<li><p>Ensemble including roberta and funnel<br>\n      - The best PrivateLB for the deberta-only ensemble model was 0.435807.<br>\n      - The best PrivateLB in every submit was 0.435103.</p></li>\n<li><p>Optimization by nelder-mead<br>\n　　　- Our bestCV(0.44287) and bestPrivate(0.435103) were ensemble models with weights optimized by nelder-mead.<br>\n　　　- In the final submission, the best model is the ensemble model of cv0.44384,Private0.435417 optimized by nelder-mead</p></li>\n</ul>\n<p><strong>What Didn’t Work</strong></p>\n<ul>\n<li>adding LSTM to model</li>\n<li>Embedding SVR with fine-tuned models</li>\n<li>Different loss functions</li>\n</ul>\n<p><strong>Important ciations</strong></p>\n<ul>\n<li><a href=\"https://www.kaggle.com/code/kojimar/fb3-deberta-family-inference-9-28-updated\" target=\"_blank\">FB3 Deberta Family Inference</a></li>\n<li><a href=\"https://www.kaggle.com/code/yasufuminakama/fb3-deberta-v3-base-baseline-train\" target=\"_blank\">FB3 / Deberta-v3-base baseline</a></li>\n<li><a href=\"https://www.kaggle.com/code/yasufuminakama/fb3-deberta-v3-base-baseline-inference/notebook\" target=\"_blank\">FB3 / Deberta-v3-base baseline [inference]</a></li>\n</ul>\n<p><strong>Thanks and Acknowledgements:</strong><br>\nThanks to our hosts for hosting the competition. And thanks to the Kaggler's for sharing their helpful notebooks and discussions.Special thanks to my teammates for such a perfect teamwork</p>\n<p><strong>Team Members:</strong></p>\n<ul>\n<li><a href=\"https://www.kaggle.com/rachisotaro\" target=\"_blank\">@rachisotaro</a> </li>\n<li><a href=\"https://www.kaggle.com/shinuk\" target=\"_blank\">@shinuk</a> </li>\n</ul>",
  "comments": [],
  "comment_count": 0,
  "messages": [
    {
      "id": 2061519,
      "postDate": "2022-12-11T07:42:36.480Z",
      "content": "<p>First of all, thanks to the host for an interesting competition and congratulations to all the winners!</p>\n<p><strong>Overview</strong></p>\n<ul>\n<li>Our final submission is ensemble of 13 models. The weights of the models are determined using nelder-mead.</li>\n<li>cv0.44384/PublicLB0.435426/PrivateLB0.435417</li>\n</ul>\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F7393510%2F297a629de174ed1245d6fb524837a9ab%2F324.png?generation=1670742662019516&amp;alt=media\" alt=\"\"></p>\n<p><strong>What Worked</strong></p>\n<ul>\n<li><p>Ensemble including roberta and funnel<br>\n      - The best PrivateLB for the deberta-only ensemble model was 0.435807.<br>\n      - The best PrivateLB in every submit was 0.435103.</p></li>\n<li><p>Optimization by nelder-mead<br>\n　　　- Our bestCV(0.44287) and bestPrivate(0.435103) were ensemble models with weights optimized by nelder-mead.<br>\n　　　- In the final submission, the best model is the ensemble model of cv0.44384,Private0.435417 optimized by nelder-mead</p></li>\n</ul>\n<p><strong>What Didn’t Work</strong></p>\n<ul>\n<li>adding LSTM to model</li>\n<li>Embedding SVR with fine-tuned models</li>\n<li>Different loss functions</li>\n</ul>\n<p><strong>Important ciations</strong></p>\n<ul>\n<li><a href=\"https://www.kaggle.com/code/kojimar/fb3-deberta-family-inference-9-28-updated\" target=\"_blank\">FB3 Deberta Family Inference</a></li>\n<li><a href=\"https://www.kaggle.com/code/yasufuminakama/fb3-deberta-v3-base-baseline-train\" target=\"_blank\">FB3 / Deberta-v3-base baseline</a></li>\n<li><a href=\"https://www.kaggle.com/code/yasufuminakama/fb3-deberta-v3-base-baseline-inference/notebook\" target=\"_blank\">FB3 / Deberta-v3-base baseline [inference]</a></li>\n</ul>\n<p><strong>Thanks and Acknowledgements:</strong><br>\nThanks to our hosts for hosting the competition. And thanks to the Kaggler's for sharing their helpful notebooks and discussions.Special thanks to my teammates for such a perfect teamwork</p>\n<p><strong>Team Members:</strong></p>\n<ul>\n<li><a href=\"https://www.kaggle.com/rachisotaro\" target=\"_blank\">@rachisotaro</a> </li>\n<li><a href=\"https://www.kaggle.com/shinuk\" target=\"_blank\">@shinuk</a> </li>\n</ul>",
      "rawMarkdown": "First of all, thanks to the host for an interesting competition and congratulations to all the winners!\n\n**Overview**\n- Our final submission is ensemble of 13 models. The weights of the models are determined using nelder-mead.\n- cv0.44384/PublicLB0.435426/PrivateLB0.435417\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F7393510%2F297a629de174ed1245d6fb524837a9ab%2F324.png?generation=1670742662019516&alt=media)\n\n**What Worked**\n- Ensemble including roberta and funnel\n          - The best PrivateLB for the deberta-only ensemble model was 0.435807.\n          - The best PrivateLB in every submit was 0.435103.\n\n- Optimization by nelder-mead\n　　　- Our bestCV(0.44287) and bestPrivate(0.435103) were ensemble models with weights optimized by nelder-mead.\n　　　- In the final submission, the best model is the ensemble model of cv0.44384,Private0.435417 optimized by nelder-mead\n\n**What Didn’t Work**\n- adding LSTM to model\n- Embedding SVR with fine-tuned models\n- Different loss functions\n\n**Important ciations**\n- [FB3 Deberta Family Inference](https://www.kaggle.com/code/kojimar/fb3-deberta-family-inference-9-28-updated)\n- [FB3 / Deberta-v3-base baseline](https://www.kaggle.com/code/yasufuminakama/fb3-deberta-v3-base-baseline-train)\n- [FB3 / Deberta-v3-base baseline [inference]](https://www.kaggle.com/code/yasufuminakama/fb3-deberta-v3-base-baseline-inference/notebook)\n\n**Thanks and Acknowledgements:**\nThanks to our hosts for hosting the competition. And thanks to the Kaggler's for sharing their helpful notebooks and discussions.Special thanks to my teammates for such a perfect teamwork\n\n**Team Members:**\n- @rachisotaro \n- @shinuk ",
      "votes": 4
    }
  ],
  "complete": true,
  "source": "live",
  "fetched_at": "2026-10-19T20:06:31Z",
  "competition_id": 38321,
  "competition_slug": "feedback-prize-english-language-learning",
  "team_id": 9361775,
  "solution_url": "/competitions/feedback-prize-english-language-learning/writeups/team-sawayaka-37th-place-solution"
}