{ "id": 371467, "title": "44th place solution", "content": "

Overview

\n

Our solution is an ensemble of ridge regression and support vector regression, trained on predictions of 8 fine-tuned deberta models and 2 classical ml algorithms using 10-fold cross-validation and optimized with Optuna (see Fig. 1).\"Figure
\nThe scores are listed in the table below:

\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
samplescore
cross-validation0.44355
public test0.43845
private test0.435586
\n

Our code

\n

What worked:

\n\n

What didn't work:

\n\n

Important Citations:

\n\n

Thanks and Acknowledgements:

\n\n

Team members:

\n", "comments": [], "comment_count": 0, "messages": [ { "id": 2060668, "postDate": "2022-12-10T09:06:56.910Z", "content": "

Overview

\n

Our solution is an ensemble of ridge regression and support vector regression, trained on predictions of 8 fine-tuned deberta models and 2 classical ml algorithms using 10-fold cross-validation and optimized with Optuna (see Fig. 1).\"Figure
\nThe scores are listed in the table below:

\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
samplescore
cross-validation0.44355
public test0.43845
private test0.435586
\n

Our code

\n

What worked:

\n\n

What didn't work:

\n\n

Important Citations:

\n\n

Thanks and Acknowledgements:

\n\n

Team members:

\n", "rawMarkdown": "## Overview\nOur solution is an ensemble of ridge regression and support vector regression, trained on predictions of 8 fine-tuned deberta models and 2 classical ml algorithms using 10-fold cross-validation and optimized with *Optuna* (see Fig. 1).![Figure 1: Our solution scheme](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F303188%2F6a3d10f8713e526a45932203882ddc1e%2Ffb3%20(1).png?generation=1670625664430131&alt=media)\nThe scores are listed in the table below:\n| **sample** | **score** |\n| --- | --- |\n| cross-validation | 0.44355 |\n| public test | 0.43845 |\n| private test | 0.435586 |\n\n## [Our code](https://www.kaggle.com/code/andreyustinov/inference-notebook-stacking?scriptVersionId=112415397)\n\n## What worked:\n- Very careful cross-validating on 10 folds and own coefficients for each target field\n- Mix of debertas\n- Adding some classical ml (even with features based on deberta)\n- Automatic weight tune\n## What didn't work:\n- Handcrafted and semi-automatic features (although some of them were not so bad)\n- Sentence embeddings (especially for a `vocabulary` mark)\n## Important Citations:\n- [Ten debertas notebook](https://www.kaggle.com/code/jingwora1/fb3-deberta-family-inference-weight-tune)\n- [Optuna](https://optuna.org)\n## Thanks and Acknowledgements:\n- My teammate Andrey Ustinov\n- All those great and honorable persons who publish high-score notebooks (even if it blows the leaderboard up)\n## Team members:\n- [Andrey Ustinov](https://www.kaggle.com/andreyustinov) \n- [Innokentiy Humonen](https://www.kaggle.com/user303188)", "votes": 4 } ], "complete": true, "source": "live", "fetched_at": "2026-10-19T20:06:22Z", "competition_id": 38321, "competition_slug": "feedback-prize-english-language-learning", "team_id": 9110121, "solution_url": "/competitions/feedback-prize-english-language-learning/writeups/44th-place-solution" }