{
  "id": 347420,
  "title": "73th Place Solution(feature design, MLM, stacking)",
  "content": "<p>First of all, I would like to thank Kaggle and host for hosting such an interesting competition!</p>\n<h1>Summary</h1>\n<h4>1.feature design</h4>\n<p>The score was boosted by adding a discourse_type of the same essay_id for each discourse_text.<br>\n Also, score was better with discourse_type capitalized.<br>\n<code>dis_type.upper() +  ‘ '  + dis_text+ ‘ ‘ + dis_type of same essay_id + ‘ ' + dis_text of same essay_id  + ‘ ' + dis_type of same essay_id + ‘ ' + dis_text of same essay_id ...</code></p>\n<h4>2.feedback2021 MLM</h4>\n<p>Model was learned MLM, after feedback2021 dataset is converted to feedback2022 baseline format and feature designed.<br>\n・before: raw feedback2021 essay_text</p>\n<pre><code>dis_text, \ndis_text, \ndis_text, …\n</code></pre>\n<p>・after: feedback2022 baseline format &amp; feature design</p>\n<pre><code>dis_type.upper() +  ‘ '  + dis_text+ ‘ ‘ + dis_type of essay_id + ‘ ' + dis_text of essay_id  + ‘ ' + dis_type of essay_id + ‘ ' + dis_text of essay_id …,\ndis_type.upper() +  ‘ '  + dis_text+ ‘ ‘ + dis_type of essay_id + ‘ ' + dis_text of essay_id  + ‘ ' + dis_type of essay_id + ‘ ' + dis_text of essay_id …,\ndis_type.upper() +  ‘ '  + dis_text+ ‘ ‘ + dis_type of essay_id + ‘ ' + dis_text of essay_id  + ‘ ' + dis_type of essay_id + ‘ ' + dis_text of essay_id …,\n</code></pre>\n<h4>3.stacking</h4>\n<p>stage1. 4 deberta-v3-large models ,   stage2.LGBM<br>\nThe score was boosted by stacking, while the weighted average ensemble only boosted the score a little.<br>\nmodel1 : feature design + MLM</p>\n<pre><code>dis_type.upper() +  ‘ '  + dis_text+ ‘[SEP]‘ + dis_type of essay_id + ‘ ' + dis_text of essay_id  + ‘ ' + dis_type of essay_id + ‘ ' + dis_text of essay_id ...\n</code></pre>\n<p>model2 : feature design + MLM</p>\n<pre><code>dis_type.upper() +  ‘[SEP]'  + dis_text+ ‘[SEP]‘ + dis_type of essay_id + ‘ ' + dis_text of essay_id  + ‘ ' + dis_type of essay_id + ‘ ' + dis_text of essay_id ...\n</code></pre>\n<p>model3 : feature design</p>\n<pre><code>dis_type.upper() +  ‘ '  + dis_text+ ‘ ‘ + dis_type of essay_id + ‘[SEP]' + dis_text of essay_id  + ‘ ' + dis_type of essay_id + ‘ ' + dis_text of essay_id ...\n</code></pre>\n<p>model4 : feature design</p>\n<pre><code>dis_type.upper() +  ‘ '  + dis_text+ ‘[SIGN]' + dis_types of essay_id + ‘ ' + dis_texts of essay_id\n</code></pre>\n<h1>Other</h1>\n<p>・approach : text classification<br>\n・model : only deberta v3 large<br>\n・max length  : 512<br>\n・CV strategy :StratifiedGroupKFold<br>\n・head : cls embedding<br>\n・epoch : 2</p>\n<h1>Score</h1>\n<table>\n<thead>\n<tr>\n<th>CV</th>\n<th>public LB</th>\n<th>private LB</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>0.5920</td>\n<td>0.585</td>\n<td>0.589</td>\n</tr>\n<tr>\n<td>0.5919</td>\n<td>0.586</td>\n<td>0.590</td>\n</tr>\n</tbody>\n</table>",
  "comments": [],
  "comment_count": 0,
  "messages": [
    {
      "id": 1911362,
      "postDate": "2022-08-24T03:48:57.863Z",
      "content": "<p>First of all, I would like to thank Kaggle and host for hosting such an interesting competition!</p>\n<h1>Summary</h1>\n<h4>1.feature design</h4>\n<p>The score was boosted by adding a discourse_type of the same essay_id for each discourse_text.<br>\n Also, score was better with discourse_type capitalized.<br>\n<code>dis_type.upper() +  ‘ '  + dis_text+ ‘ ‘ + dis_type of same essay_id + ‘ ' + dis_text of same essay_id  + ‘ ' + dis_type of same essay_id + ‘ ' + dis_text of same essay_id ...</code></p>\n<h4>2.feedback2021 MLM</h4>\n<p>Model was learned MLM, after feedback2021 dataset is converted to feedback2022 baseline format and feature designed.<br>\n・before: raw feedback2021 essay_text</p>\n<pre><code>dis_text, \ndis_text, \ndis_text, …\n</code></pre>\n<p>・after: feedback2022 baseline format &amp; feature design</p>\n<pre><code>dis_type.upper() +  ‘ '  + dis_text+ ‘ ‘ + dis_type of essay_id + ‘ ' + dis_text of essay_id  + ‘ ' + dis_type of essay_id + ‘ ' + dis_text of essay_id …,\ndis_type.upper() +  ‘ '  + dis_text+ ‘ ‘ + dis_type of essay_id + ‘ ' + dis_text of essay_id  + ‘ ' + dis_type of essay_id + ‘ ' + dis_text of essay_id …,\ndis_type.upper() +  ‘ '  + dis_text+ ‘ ‘ + dis_type of essay_id + ‘ ' + dis_text of essay_id  + ‘ ' + dis_type of essay_id + ‘ ' + dis_text of essay_id …,\n</code></pre>\n<h4>3.stacking</h4>\n<p>stage1. 4 deberta-v3-large models ,   stage2.LGBM<br>\nThe score was boosted by stacking, while the weighted average ensemble only boosted the score a little.<br>\nmodel1 : feature design + MLM</p>\n<pre><code>dis_type.upper() +  ‘ '  + dis_text+ ‘[SEP]‘ + dis_type of essay_id + ‘ ' + dis_text of essay_id  + ‘ ' + dis_type of essay_id + ‘ ' + dis_text of essay_id ...\n</code></pre>\n<p>model2 : feature design + MLM</p>\n<pre><code>dis_type.upper() +  ‘[SEP]'  + dis_text+ ‘[SEP]‘ + dis_type of essay_id + ‘ ' + dis_text of essay_id  + ‘ ' + dis_type of essay_id + ‘ ' + dis_text of essay_id ...\n</code></pre>\n<p>model3 : feature design</p>\n<pre><code>dis_type.upper() +  ‘ '  + dis_text+ ‘ ‘ + dis_type of essay_id + ‘[SEP]' + dis_text of essay_id  + ‘ ' + dis_type of essay_id + ‘ ' + dis_text of essay_id ...\n</code></pre>\n<p>model4 : feature design</p>\n<pre><code>dis_type.upper() +  ‘ '  + dis_text+ ‘[SIGN]' + dis_types of essay_id + ‘ ' + dis_texts of essay_id\n</code></pre>\n<h1>Other</h1>\n<p>・approach : text classification<br>\n・model : only deberta v3 large<br>\n・max length  : 512<br>\n・CV strategy :StratifiedGroupKFold<br>\n・head : cls embedding<br>\n・epoch : 2</p>\n<h1>Score</h1>\n<table>\n<thead>\n<tr>\n<th>CV</th>\n<th>public LB</th>\n<th>private LB</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>0.5920</td>\n<td>0.585</td>\n<td>0.589</td>\n</tr>\n<tr>\n<td>0.5919</td>\n<td>0.586</td>\n<td>0.590</td>\n</tr>\n</tbody>\n</table>",
      "rawMarkdown": "First of all, I would like to thank Kaggle and host for hosting such an interesting competition!\n\n# Summary\n#### 1.feature design\nThe score was boosted by adding a discourse_type of the same essay_id for each discourse_text.\n Also, score was better with discourse_type capitalized.\n`dis_type.upper() +  ‘ '  + dis_text+ ‘ ‘ + dis_type of same essay_id + ‘ ' + dis_text of same essay_id  + ‘ ' + dis_type of same essay_id + ‘ ' + dis_text of same essay_id ...`\n\n#### 2.feedback2021 MLM\nModel was learned MLM, after feedback2021 dataset is converted to feedback2022 baseline format and feature designed.\n・before: raw feedback2021 essay_text\n```\ndis_text, \ndis_text, \ndis_text, …\n```\n・after: feedback2022 baseline format & feature design\n```\ndis_type.upper() +  ‘ '  + dis_text+ ‘ ‘ + dis_type of essay_id + ‘ ' + dis_text of essay_id  + ‘ ' + dis_type of essay_id + ‘ ' + dis_text of essay_id …,\ndis_type.upper() +  ‘ '  + dis_text+ ‘ ‘ + dis_type of essay_id + ‘ ' + dis_text of essay_id  + ‘ ' + dis_type of essay_id + ‘ ' + dis_text of essay_id …,\ndis_type.upper() +  ‘ '  + dis_text+ ‘ ‘ + dis_type of essay_id + ‘ ' + dis_text of essay_id  + ‘ ' + dis_type of essay_id + ‘ ' + dis_text of essay_id …,\n```\n\n#### 3.stacking\nstage1. 4 deberta-v3-large models ,   stage2.LGBM\nThe score was boosted by stacking, while the weighted average ensemble only boosted the score a little.\nmodel1 : feature design + MLM\n```\ndis_type.upper() +  ‘ '  + dis_text+ ‘[SEP]‘ + dis_type of essay_id + ‘ ' + dis_text of essay_id  + ‘ ' + dis_type of essay_id + ‘ ' + dis_text of essay_id ...\n\n```\nmodel2 : feature design + MLM\n```\ndis_type.upper() +  ‘[SEP]'  + dis_text+ ‘[SEP]‘ + dis_type of essay_id + ‘ ' + dis_text of essay_id  + ‘ ' + dis_type of essay_id + ‘ ' + dis_text of essay_id ...\n\n```\nmodel3 : feature design\n```\ndis_type.upper() +  ‘ '  + dis_text+ ‘ ‘ + dis_type of essay_id + ‘[SEP]' + dis_text of essay_id  + ‘ ' + dis_type of essay_id + ‘ ' + dis_text of essay_id ...\n\n```\nmodel4 : feature design\n```\ndis_type.upper() +  ‘ '  + dis_text+ ‘[SIGN]' + dis_types of essay_id + ‘ ' + dis_texts of essay_id\n\n```\n# Other\n・approach : text classification\n・model : only deberta v3 large\n・max length  : 512\n・CV strategy :StratifiedGroupKFold\n・head : cls embedding\n・epoch : 2\n\n#Score\n\n|  CV | public LB | private LB |\n| --- | --- | --- |\n| 0.5920 | 0.585 | 0.589 |\n| 0.5919 | 0.586 | 0.590 |\n",
      "votes": 11
    }
  ],
  "complete": true,
  "source": "live",
  "fetched_at": "2026-10-19T17:54:38Z",
  "competition_id": 35308,
  "competition_slug": "feedback-prize-effectiveness",
  "team_id": 8677190,
  "solution_url": "/competitions/feedback-prize-effectiveness/writeups/muj-rush-73th-place-solution-feature-design-mlm-st"
}