{
  "id": 446591,
  "title": "35th Place Solution",
  "content": "<p>Thanks for organizing the challenging competition. Though we got a silver medal finally, we learned a lot from the experienments. And thanks my teammates <a href=\"https://www.kaggle.com/penpentled\" target=\"_blank\">@penpentled</a> <a href=\"https://www.kaggle.com/shigeria\" target=\"_blank\">@shigeria</a> <a href=\"https://www.kaggle.com/chaudharypriyanshu\" target=\"_blank\">@chaudharypriyanshu</a> <a href=\"https://www.kaggle.com/takanashihumbert\" target=\"_blank\">@takanashihumbert</a>, they have been really helpful these months. </p>\n<h1>1. Summary</h1>\n<p>Our best private solution submitted is a ensemble of models, which contain a model with prompt_text, 3 wording models and a content+lgbm model without prompt_text. The inference time is about 8hrs. </p>\n<table>\n<thead>\n<tr>\n<th>Sub</th>\n<th>Type</th>\n<th>CV</th>\n<th>LB</th>\n<th>PB</th>\n<th>Comment</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>No.1</td>\n<td>best lb</td>\n<td>0.4655</td>\n<td>0.426</td>\n<td>0.466</td>\n<td>model with prompt_text training max_len=896 infer 1024</td>\n</tr>\n<tr>\n<td>No.2</td>\n<td>best cv</td>\n<td>0.463</td>\n<td>0.427</td>\n<td>0.467</td>\n<td>substitute the content model with a distilled model+lgbm</td>\n</tr>\n<tr>\n<td>No.3</td>\n<td>conservative</td>\n<td>0.476</td>\n<td>0.439</td>\n<td>0.484</td>\n<td>only models without prompt_text</td>\n</tr>\n</tbody>\n</table>\n<p>Though we have a model scored 0.461 for pb, we didn't select it since it only scored 0.445 for lb. It contains training on 2 folds validation on other 2 folds models by <a href=\"https://www.kaggle.com/takanashihumbert\" target=\"_blank\">@takanashihumbert</a> and the base model with prompt_text.</p>\n<h1>2. Training</h1>\n<p>Models submitted are shown below.</p>\n<table>\n<thead>\n<tr>\n<th>ID</th>\n<th>Type</th>\n<th>Train max_len</th>\n<th>Infer max_len</th>\n<th>Text Cleaning</th>\n<th>CV</th>\n<th>Model</th>\n<th>With prompt_text</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>base</td>\n<td>both</td>\n<td>896</td>\n<td>1024</td>\n<td>public methods</td>\n<td>0.477</td>\n<td>v3-large</td>\n<td>✓</td>\n</tr>\n<tr>\n<td>w2</td>\n<td>wording</td>\n<td>512</td>\n<td>512</td>\n<td>replaced copied text with [PASSAGE]</td>\n<td>0.556</td>\n<td>v3-large</td>\n<td></td>\n</tr>\n<tr>\n<td>w4</td>\n<td>wording</td>\n<td>768</td>\n<td>768</td>\n<td>replaced copied text with [PASSAGE]</td>\n<td>0.559</td>\n<td>v3-large</td>\n<td></td>\n</tr>\n<tr>\n<td>w6</td>\n<td>wording</td>\n<td>512</td>\n<td>512</td>\n<td>replaced copied text with [PASSAGE] and [REFERENCE]</td>\n<td>0.5561</td>\n<td>v3-base</td>\n<td></td>\n</tr>\n<tr>\n<td>c1</td>\n<td>content</td>\n<td>1024</td>\n<td>1024</td>\n<td>public methods</td>\n<td>0.4445</td>\n<td>v3-large</td>\n<td></td>\n</tr>\n<tr>\n<td>distill</td>\n<td>content</td>\n<td>1024</td>\n<td>1024</td>\n<td>joined copied text with [PASSAGE] and [REFERENCE] tokens</td>\n<td>0.488</td>\n<td>v3-base</td>\n<td></td>\n</tr>\n</tbody>\n</table>\n<h1>3. inference</h1>\n<p>We used optuna and nelder-mead to optimize for ensemble, and features that public notebooks shared for lgbm. We strived for both better cv and lb when ensemble.</p>\n<h1>4. Not working for us</h1>\n<ul>\n<li>Models except for v3</li>\n<li>Mean Pooling, Concat Pooling, …</li>\n</ul>\n<h1>5. Acknowledgements</h1>\n<ul>\n<li><a href=\"https://www.kaggle.com/tsunotsuno\" target=\"_blank\">@tsunotsuno</a> idea of lgbm and feature engineering</li>\n</ul>\n<h1>6.Notebooks</h1>\n<p>Best PB submission: <a href=\"https://www.kaggle.com/code/snorfyang/commonlit-exp-e6-shigeria1/notebook\" target=\"_blank\">https://www.kaggle.com/code/snorfyang/commonlit-exp-e6-shigeria1/notebook</a><br>\nText cleaning, ensemble methods, final submissions and other notebooks may be published later.</p>",
  "comments": [],
  "comment_count": 0,
  "messages": [
    {
      "id": 2478888,
      "postDate": "2023-10-12T08:55:11.853Z",
      "content": "<p>Thanks for organizing the challenging competition. Though we got a silver medal finally, we learned a lot from the experienments. And thanks my teammates <a href=\"https://www.kaggle.com/penpentled\" target=\"_blank\">@penpentled</a> <a href=\"https://www.kaggle.com/shigeria\" target=\"_blank\">@shigeria</a> <a href=\"https://www.kaggle.com/chaudharypriyanshu\" target=\"_blank\">@chaudharypriyanshu</a> <a href=\"https://www.kaggle.com/takanashihumbert\" target=\"_blank\">@takanashihumbert</a>, they have been really helpful these months. </p>\n<h1>1. Summary</h1>\n<p>Our best private solution submitted is a ensemble of models, which contain a model with prompt_text, 3 wording models and a content+lgbm model without prompt_text. The inference time is about 8hrs. </p>\n<table>\n<thead>\n<tr>\n<th>Sub</th>\n<th>Type</th>\n<th>CV</th>\n<th>LB</th>\n<th>PB</th>\n<th>Comment</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>No.1</td>\n<td>best lb</td>\n<td>0.4655</td>\n<td>0.426</td>\n<td>0.466</td>\n<td>model with prompt_text training max_len=896 infer 1024</td>\n</tr>\n<tr>\n<td>No.2</td>\n<td>best cv</td>\n<td>0.463</td>\n<td>0.427</td>\n<td>0.467</td>\n<td>substitute the content model with a distilled model+lgbm</td>\n</tr>\n<tr>\n<td>No.3</td>\n<td>conservative</td>\n<td>0.476</td>\n<td>0.439</td>\n<td>0.484</td>\n<td>only models without prompt_text</td>\n</tr>\n</tbody>\n</table>\n<p>Though we have a model scored 0.461 for pb, we didn't select it since it only scored 0.445 for lb. It contains training on 2 folds validation on other 2 folds models by <a href=\"https://www.kaggle.com/takanashihumbert\" target=\"_blank\">@takanashihumbert</a> and the base model with prompt_text.</p>\n<h1>2. Training</h1>\n<p>Models submitted are shown below.</p>\n<table>\n<thead>\n<tr>\n<th>ID</th>\n<th>Type</th>\n<th>Train max_len</th>\n<th>Infer max_len</th>\n<th>Text Cleaning</th>\n<th>CV</th>\n<th>Model</th>\n<th>With prompt_text</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>base</td>\n<td>both</td>\n<td>896</td>\n<td>1024</td>\n<td>public methods</td>\n<td>0.477</td>\n<td>v3-large</td>\n<td>✓</td>\n</tr>\n<tr>\n<td>w2</td>\n<td>wording</td>\n<td>512</td>\n<td>512</td>\n<td>replaced copied text with [PASSAGE]</td>\n<td>0.556</td>\n<td>v3-large</td>\n<td></td>\n</tr>\n<tr>\n<td>w4</td>\n<td>wording</td>\n<td>768</td>\n<td>768</td>\n<td>replaced copied text with [PASSAGE]</td>\n<td>0.559</td>\n<td>v3-large</td>\n<td></td>\n</tr>\n<tr>\n<td>w6</td>\n<td>wording</td>\n<td>512</td>\n<td>512</td>\n<td>replaced copied text with [PASSAGE] and [REFERENCE]</td>\n<td>0.5561</td>\n<td>v3-base</td>\n<td></td>\n</tr>\n<tr>\n<td>c1</td>\n<td>content</td>\n<td>1024</td>\n<td>1024</td>\n<td>public methods</td>\n<td>0.4445</td>\n<td>v3-large</td>\n<td></td>\n</tr>\n<tr>\n<td>distill</td>\n<td>content</td>\n<td>1024</td>\n<td>1024</td>\n<td>joined copied text with [PASSAGE] and [REFERENCE] tokens</td>\n<td>0.488</td>\n<td>v3-base</td>\n<td></td>\n</tr>\n</tbody>\n</table>\n<h1>3. inference</h1>\n<p>We used optuna and nelder-mead to optimize for ensemble, and features that public notebooks shared for lgbm. We strived for both better cv and lb when ensemble.</p>\n<h1>4. Not working for us</h1>\n<ul>\n<li>Models except for v3</li>\n<li>Mean Pooling, Concat Pooling, …</li>\n</ul>\n<h1>5. Acknowledgements</h1>\n<ul>\n<li><a href=\"https://www.kaggle.com/tsunotsuno\" target=\"_blank\">@tsunotsuno</a> idea of lgbm and feature engineering</li>\n</ul>\n<h1>6.Notebooks</h1>\n<p>Best PB submission: <a href=\"https://www.kaggle.com/code/snorfyang/commonlit-exp-e6-shigeria1/notebook\" target=\"_blank\">https://www.kaggle.com/code/snorfyang/commonlit-exp-e6-shigeria1/notebook</a><br>\nText cleaning, ensemble methods, final submissions and other notebooks may be published later.</p>",
      "rawMarkdown": "Thanks for organizing the challenging competition. Though we got a silver medal finally, we learned a lot from the experienments. And thanks my teammates @penpentled @shigeria @chaudharypriyanshu @takanashihumbert, they have been really helpful these months. \n\n# 1. Summary\n\nOur best private solution submitted is a ensemble of models, which contain a model with prompt_text, 3 wording models and a content+lgbm model without prompt_text. The inference time is about 8hrs. \n| Sub | Type | CV | LB | PB | Comment |\n| --- | --- | --- | --- | --- | --- | \n| No.1 | best lb | 0.4655 | 0.426 | 0.466 | model with prompt_text training max_len=896 infer 1024 |\n| No.2 | best cv | 0.463 | 0.427 | 0.467 | substitute the content model with a distilled model+lgbm |\n| No.3 | conservative | 0.476 | 0.439 | 0.484 | only models without prompt_text |  \n\n\nThough we have a model scored 0.461 for pb, we didn't select it since it only scored 0.445 for lb. It contains training on 2 folds validation on other 2 folds models by @takanashihumbert and the base model with prompt_text.\n\n# 2. Training\nModels submitted are shown below.\n| ID | Type | Train max_len | Infer max_len | Text Cleaning | CV | Model | With prompt_text |\n| --- | --- | --- | --- | --- | --- | --- |\n| base | both | 896 | 1024 | public methods | 0.477 | v3-large | ✓ |\n| w2 | wording | 512 | 512 | replaced copied text with [PASSAGE] | 0.556 | v3-large |\n| w4 | wording | 768 | 768 | replaced copied text with [PASSAGE] | 0.559 | v3-large |\n| w6 | wording | 512 | 512 | replaced copied text with [PASSAGE] and [REFERENCE] | 0.5561 | v3-base |\n| c1 | content | 1024 | 1024 | public methods | 0.4445 | v3-large |\n| distill | content | 1024 | 1024 | joined copied text with [PASSAGE] and [REFERENCE] tokens | 0.488 | v3-base |\n\n# 3. inference\nWe used optuna and nelder-mead to optimize for ensemble, and features that public notebooks shared for lgbm. We strived for both better cv and lb when ensemble.\n\n# 4. Not working for us\n- Models except for v3\n- Mean Pooling, Concat Pooling, ...\n\n# 5. Acknowledgements\n- @tsunotsuno idea of lgbm and feature engineering\n\n# 6.Notebooks\nBest PB submission: https://www.kaggle.com/code/snorfyang/commonlit-exp-e6-shigeria1/notebook\nText cleaning, ensemble methods, final submissions and other notebooks may be published later.\n\n",
      "votes": 12
    }
  ],
  "complete": true,
  "source": "live",
  "fetched_at": "2026-10-19T23:02:57Z",
  "competition_id": 53482,
  "competition_slug": "commonlit-evaluate-student-summaries",
  "team_id": 10676887,
  "solution_url": "/competitions/commonlit-evaluate-student-summaries/writeups/machine-not-learning-35th-place-solution"
}