![](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F2701619%2Fa64d6d2130b7138cf7b226553d40c909%2Fshape.png?generation=1670207602959148&alt=media\")
First of all, I would like to thank you for organizing this competition.
\nHonestly, I did not expect to be 2nd in efficiency prize.
\nThere has been some unfortunate event in my personal life, this makes it difficult to prepare according to the Guidelines.
\nBut I think it is better to provide my solution asap, so I am writing a summary.
MultiLabelStratifiedKFold 4fold for targets and topics from BERTopic.
\nAfter adjusting the parameters, I used all train data (1fold) .
I'd like to use this opportunity to also thank @kashiwaba. I joined this competition after reading the article you summarized about FB2(external link : Japanese).
\nMy first NLP competition was so fun. Thanks to the organizers for holding this interesting competition. Thanks to the kagglers for sharing very helpful codes and discussions.
\nPlease take good care of yourself.
", "comments": [], "comment_count": 0, "messages": [ { "id": 2050405, "postDate": "2022-11-30T17:56:09.817Z", "content": "First of all, I would like to thank you for organizing this competition.
\nHonestly, I did not expect to be 2nd in efficiency prize.
\nThere has been some unfortunate event in my personal life, this makes it difficult to prepare according to the Guidelines.
\nBut I think it is better to provide my solution asap, so I am writing a summary.
MultiLabelStratifiedKFold 4fold for targets and topics from BERTopic.
\nAfter adjusting the parameters, I used all train data (1fold) .
I'd like to use this opportunity to also thank @kashiwaba. I joined this competition after reading the article you summarized about FB2(external link : Japanese).
\nMy first NLP competition was so fun. Thanks to the organizers for holding this interesting competition. Thanks to the kagglers for sharing very helpful codes and discussions.
\nPlease take good care of yourself.
", "rawMarkdown": "First of all, I would like to thank you for organizing this competition.\nHonestly, I did not expect to be 2nd in efficiency prize. \nThere has been some unfortunate event in my personal life, this makes it difficult to prepare according to the [Guidelines](https://www.kaggle.com/WinningModelDocumentationGuidelines).\nBut I think it is better to provide my solution asap, so I am writing a summary.\n\n# OverView\n- **Leaderboard:** I used multiple Deberta series models, SVR, as well as pseudo-labels based on FB1 data and classify argumentative elements models from FB2 provides meta data. Finally, 13 models were weight-tuned with L-BFGS-B method.\n- **Efficiency:** I used deberta-v3-xsmall with max_len:512, knowledge distillation by above-mentioned ensemble model and uniform length batching. I did not do any special tricks and spent a lot of time tuning.\n\n\n\n# CV strategy\nMultiLabelStratifiedKFold 4fold for targets and topics from BERTopic.\nAfter adjusting the parameters, I used all train data (1fold) .\n\n\n# What Worked Efficency LB\n- [High Imapct] deberta-v3-xsmall\n - I think it was important to use xsmall. My verification confirmed that the CV is higher than small but can be made up of processing speed.\n- [High Imapct] uniform length batching\n - Increased processing speed compared to the normal method (inference) .\n- [High Imapct] knowledge distillation\n - Even the 82th model was useful enough.\n- [Midium Imapct] adjust_max_len\n - 512 was consistently good performance for me.\n\n# What Didn't Work Efficency LB\n- Torch.jit\n - I continued to apply torch.jit except for uniform length batching. But there was almost no difference in results.\n- increase max_len\n - I noticed that I get better CV by increasing xsmall_max_len slightly (e.g.576) . However, they did not useful in private.\n\n# Important Citations:\n - [Dyanmic Padding and Uniform Length Batching](https://www.kaggle.com/code/rhtsingh/speeding-up-transformer-w-optimization-strategies)\n - @rhtsingh 's notebooks is very good learning experience.\n - [FB3 / Deberta-v3-base baseline [train]](https://www.kaggle.com/code/yasufuminakama/fb3-deberta-v3-base-baseline-train)\n - I used @yasufuminakama 's starter code, this was really helpful.\n - [RAPIDS SVR - CV 0.450 - LB 0.44x](https://www.kaggle.com/code/cdeotte/rapids-svr-cv-0-450-lb-0-44x)\n - I always admire @cdeotte. Check out about the SVR.\n - [[Train]Deberta-v3-base with BERTopic](https://www.kaggle.com/code/mujrush/train-deberta-v3-base-with-bertopic#Deberta-v3-base-baseline)\n - It worked for my CV strategy.\n\n---\n I'd like to use this opportunity to also thank @kashiwaba. I joined this competition after reading the article you summarized about FB2[(external link : Japanese)](https://zenn.dev/yume_neko/articles/6e8f78a12a1d2a).\n\n# Thanks and Acknowledgements:\nMy first NLP competition was so fun. Thanks to the organizers for holding this interesting competition. Thanks to the kagglers for sharing very helpful codes and discussions. \n\nPlease take good care of yourself.",
"votes": 17
}
],
"complete": true,
"source": "live",
"fetched_at": "2026-10-19T20:02:44Z",
"competition_id": 38321,
"competition_slug": "feedback-prize-english-language-learning",
"team_id": 9092408,
"solution_url": "/competitions/feedback-prize-english-language-learning/writeups/run-out-of-ideas-efficiency-2nd-solution-private-8"
}