{ "id": 588473, "title": "Ensemble strategy", "url": "/competitions/cmi-detect-behavior-with-sensor-data/discussion/588473", "author_name": "", "post_date": "2025-07-06T16:49:00.775654200Z", "votes": 4, "comment_count": 7, "views": 0, "content": "

In my experiments about folds model ensemble, voting ＞ median ＞ logits average. What about you?

", "messages": [ { "id": "3243007", "postDate": "07/06/2025 16:49:00", "content": "

In my experiments about folds model ensemble, voting ＞ median ＞ logits average. What about you?

", "rawMarkdown": "In my experiments about folds model ensemble, voting ＞ median ＞ logits average. What about you?", "votes": null }, { "id": "3243373", "postDate": "07/07/2025 04:23:33", "content": "

[ Your Notebook ] Using IMU and full sensor models is a good strategy the top 5 probably do the same! Mine also achieved 0.81, but with just a single model.

\n

Regarding voting, I suggest looking at the confusion matrices for all your models, especially because they tend to confuse 'Neck - pinch skin' and 'Neck - scratch'. Calculating the median of top-1 logits across models isn’t always effective — if one model is off, there's a high chance others might also misclassify in the same way, especially with full sensor inputs.

\n

This is just my understanding, but alternatively, adjusting custom weights helped me boost my leaderboard score by 0.02 (from 0.79 to 0.81) without retraining anything!

", "rawMarkdown": "[[ Your Notebook ]](https://www.kaggle.com/code/kurisew/an-example-about-swiching-model) Using IMU and full sensor models is a good strategy the top 5 probably do the same! Mine also achieved 0.81, but with just a single model.\n\nRegarding voting, I suggest looking at the confusion matrices for all your models, especially because they tend to confuse 'Neck - pinch skin' and 'Neck - scratch'. Calculating the median of top-1 logits across models isn’t always effective — if one model is off, there's a high chance others might also misclassify in the same way, especially with full sensor inputs.\n\nThis is just my understanding, but alternatively, adjusting custom weights helped me boost my leaderboard score by 0.02 (from 0.79 to 0.81) without retraining anything!", "votes": null }, { "id": "3243390", "postDate": "07/07/2025 04:52:53", "content": "

Thanks! Good idea.

", "rawMarkdown": "Thanks! Good idea.", "votes": null }, { "id": "3244849", "postDate": "07/08/2025 16:21:02", "content": "

Thank you for sharing you ensembling approach, but don't you think that adjusting weights of fold-wise models can lead to the overfitting on the public leaderboard?

", "rawMarkdown": "Thank you for sharing you ensembling approach, but don't you think that adjusting weights of fold-wise models can lead to the overfitting on the public leaderboard?", "votes": null }, { "id": "3249239", "postDate": "07/16/2025 04:41:42", "content": "

Voting is promising if we have massive and diverse models.

", "rawMarkdown": "Voting is promising if we have massive and diverse models.", "votes": null }, { "id": "3249288", "postDate": "07/16/2025 07:11:58", "content": "

Agree with that, because we just need to predict 18 classes.

", "rawMarkdown": "Agree with that, because we just need to predict 18 classes.", "votes": null }, { "id": "3249374", "postDate": "07/16/2025 10:34:38", "content": "

I just test voting and logits average with 5 fold models. In my condition , logits average is better than voting(+0.005~0.01LB). Not sure if it's overfitting and need more testing.

", "rawMarkdown": "I just test voting and logits average with 5 fold models. In my condition , logits average is better than voting(+0.005~0.01LB). Not sure if it's overfitting and need more testing.", "votes": null }, { "id": "3249411", "postDate": "07/16/2025 12:23:38", "content": "

like if we have a good way to do similar thing like random forest but on the model side, then voting is a good choice.

", "rawMarkdown": "like if we have a good way to do similar thing like random forest but on the model side, then voting is a good choice.", "votes": null } ], "comments": [ { "id": 3243373, "author_name": "sangrampatil5150", "author_url": "", "post_date": "07/07/2025 04:23:33", "content": "

[ Your Notebook ] Using IMU and full sensor models is a good strategy the top 5 probably do the same! Mine also achieved 0.81, but with just a single model.

\n

Regarding voting, I suggest looking at the confusion matrices for all your models, especially because they tend to confuse 'Neck - pinch skin' and 'Neck - scratch'. Calculating the median of top-1 logits across models isn’t always effective — if one model is off, there's a high chance others might also misclassify in the same way, especially with full sensor inputs.

\n

This is just my understanding, but alternatively, adjusting custom weights helped me boost my leaderboard score by 0.02 (from 0.79 to 0.81) without retraining anything!

", "votes": null, "replies": [ { "id": 3243390, "author_name": "kurisew", "author_url": "", "post_date": "07/07/2025 04:52:53", "content": "

Thanks! Good idea.

", "votes": null, "replies": [] }, { "id": 3244849, "author_name": "maximumshot", "author_url": "", "post_date": "07/08/2025 16:21:02", "content": "

Thank you for sharing you ensembling approach, but don't you think that adjusting weights of fold-wise models can lead to the overfitting on the public leaderboard?

", "votes": null, "replies": [] } ] }, { "id": 3249239, "author_name": "tom99763", "author_url": "", "post_date": "07/16/2025 04:41:42", "content": "

Voting is promising if we have massive and diverse models.

", "votes": null, "replies": [ { "id": 3249288, "author_name": "kurisew", "author_url": "", "post_date": "07/16/2025 07:11:58", "content": "

Agree with that, because we just need to predict 18 classes.

", "votes": null, "replies": [ { "id": 3249411, "author_name": "tom99763", "author_url": "", "post_date": "07/16/2025 12:23:38", "content": "

like if we have a good way to do similar thing like random forest but on the model side, then voting is a good choice.

", "votes": null, "replies": [] } ] } ] }, { "id": 3249374, "author_name": "minerppdy", "author_url": "", "post_date": "07/16/2025 10:34:38", "content": "

I just test voting and logits average with 5 fold models. In my condition , logits average is better than voting(+0.005~0.01LB). Not sure if it's overfitting and need more testing.

", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "3243007": "In my experiments about folds model ensemble, voting ＞ median ＞ logits average. What about you?", "3243373": "[[ Your Notebook ]](https://www.kaggle.com/code/kurisew/an-example-about-swiching-model) Using IMU and full sensor models is a good strategy the top 5 probably do the same! Mine also achieved 0.81, but with just a single model.\n\nRegarding voting, I suggest looking at the confusion matrices for all your models, especially because they tend to confuse 'Neck - pinch skin' and 'Neck - scratch'. Calculating the median of top-1 logits across models isn’t always effective — if one model is off, there's a high chance others might also misclassify in the same way, especially with full sensor inputs.\n\nThis is just my understanding, but alternatively, adjusting custom weights helped me boost my leaderboard score by 0.02 (from 0.79 to 0.81) without retraining anything!", "3243390": "Thanks! Good idea.", "3244849": "Thank you for sharing you ensembling approach, but don't you think that adjusting weights of fold-wise models can lead to the overfitting on the public leaderboard?", "3249239": "Voting is promising if we have massive and diverse models.", "3249288": "Agree with that, because we just need to predict 18 classes.", "3249374": "I just test voting and logits average with 5 fold models. In my condition , logits average is better than voting(+0.005~0.01LB). Not sure if it's overfitting and need more testing.", "3249411": "like if we have a good way to do similar thing like random forest but on the model side, then voting is a good choice." }, "source": "meta" }