{ "id": 369529, "title": "67th place solution(Silver medal)", "content": "

Thank you to Kaggle and Host of the comp host for providing this opportunity.

\n

To be honest, I was almost heartbroken because the LB did not go up at all until the end, but in my case it was Trust CV. Here is a brief description of the solution.

\n

Multi task learning

\n

Since there were 6 targets in this competition, I added the following new targets and performed multi-task learning, which improved CV/LB.
\nIn the end, only the targets with the best CV, target_min and target_max, were added to the competition targets for multi-task learning.

\n\n

Layer-wise Learning Rate Decay(LLRD)

\n

I divided the layers into 4 groups and set the following learning_rate for each group, and trained the first layer with a smaller learning_rate.

\n\n

Ensemble

\n

Ensemble with blending was performed on 8 models.
\nNelder-Mead was used for blending.

\n

8model

\n\n

Blending

\n\n

Other

\n", "comments": [], "comment_count": 0, "messages": [ { "id": 2049849, "postDate": "2022-11-30T11:31:38.273Z", "content": "

Thank you to Kaggle and Host of the comp host for providing this opportunity.

\n

To be honest, I was almost heartbroken because the LB did not go up at all until the end, but in my case it was Trust CV. Here is a brief description of the solution.

\n

Multi task learning

\n

Since there were 6 targets in this competition, I added the following new targets and performed multi-task learning, which improved CV/LB.
\nIn the end, only the targets with the best CV, target_min and target_max, were added to the competition targets for multi-task learning.

\n\n

Layer-wise Learning Rate Decay(LLRD)

\n

I divided the layers into 4 groups and set the following learning_rate for each group, and trained the first layer with a smaller learning_rate.

\n\n

Ensemble

\n

Ensemble with blending was performed on 8 models.
\nNelder-Mead was used for blending.

\n

8model

\n\n

Blending

\n\n

Other

\n", "rawMarkdown": "Thank you to Kaggle and Host of the comp host for providing this opportunity.\n\nTo be honest, I was almost heartbroken because the LB did not go up at all until the end, but in my case it was Trust CV. Here is a brief description of the solution.\n\n\n### Multi task learning \nSince there were 6 targets in this competition, I added the following new targets and performed multi-task learning, which improved CV/LB.\nIn the end, only the targets with the best CV, target_min and target_max, were added to the competition targets for multi-task learning.\n- target sum\n- target mean\n- target median\n- target min\n- target max\n\n### Layer-wise Learning Rate Decay(LLRD)\nI divided the layers into 4 groups and set the following learning_rate for each group, and trained the first layer with a smaller learning_rate.\n- Learning_rate 2.5-e5\n- learning_rate per group\n　group1: learning_rate/5\n　group2: learning_rate/4\n　group3: learning_rate/2\n　group4: learning_rate\n\n### Ensemble\nEnsemble with blending was performed on 8 models.\nNelder-Mead was used for blending.\n\n8model\n- microsoft/deberta-v3-base\n- microsoft/deberta-v3-large\n- roberta large\n- funnel-transformer/large\n- microsoft/deberta-large\n- microsoft/deberta-v2-xlarge\n- facebook/muppet-roberta-large\n- google/bigbird-roberta-large\n\nBlending\n- Nelder-Mead\n\n### Other\n- Mean pooling\n- MultilabelStratifiedKFold\n- MultilabelStratifiedKFold5\n- No initialization of final layer\n", "votes": 11 } ], "complete": true, "source": "live", "fetched_at": "2026-10-19T19:59:49Z", "competition_id": 38321, "competition_slug": "feedback-prize-english-language-learning", "team_id": 9101581, "solution_url": "/competitions/feedback-prize-english-language-learning/writeups/muj-rush-67th-place-solution-silver-medal" }