![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F6454680%2F640d593cb269bf5bd12f8b75186384a1%2Fraw_input_img2.png?generation=1756915210477535&alt=media\")
Congratulations to all the winners! I would also like to express my gratitude to the organizers for preparing such an exciting competition.
\nI devoted a significant amount of time and effort to this competition from the very beginning, so I am truly delighted that this has resulted in a gold medal.
The key factors in my approach were (1) building diverse models and ensembling them, and (2) applying post-processing by leveraging previously inferred test samples. The idea of creating diverse models was inspired by my past gold medal solution in the HMS competition. Since HMS also dealt with time-series sensor data, some of the model ideas in this competition came directly from that experience.
\nThis idea was introduced by a teammate in the HMS competition and achieved the best performance among my five models. The input images for the 2DCNN are as follows. All signals except TOF were stacked into 16 pixels, while TOF was left unstacked. Upscaling the width to double (448 pixels in width) further improved the score. The model architecture is EfficientNetV2-B2 (B1 for IMU-only).
\n
This was the idea I personally used in HMS. The input images for the 2DCNN are as follows. TOF part processed the same as in raw-2DCNN. The model architecture is EfficientNetV2-B1.
\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F6454680%2F43dde4e1be2c63372e160b6152392b5b%2Fplot_input_img2.png?generation=1756915265089482&alt=media\")
A lightly modified version of a public notebook model.
\nSame as iii), except the GRU-Attention module was replaced with a BERT.
\nBased on models used in past competition solutions, but adjusted with smaller dim, ksize, and fewer layers.
\nTwo days before the competition deadline, I discovered this technique and achieved a significant score boost (~+0.010). The basic idea is similar to the one discussed in this thread.
\nI modified the prediction() function in the submission code into a class method, storing the predictions of previously inferred samples in self.history variable. This allowed the model to access past predictions. For each subject ID, I counted predictions per class, then penalized predictions belonging to classes with high counts.
| Model | \nIMU only CV | \nAll CV | \nPublic LB | \nPrivate LB | \n
|---|---|---|---|---|
| raw-2DCNN | \n0.7972 | \n0.8773 | \n- | \n- | \n
| plot-2DCNN | \n0.7695 | \n0.8632 | \n- | \n- | \n
| 1DCNN-GRU-Attention | \n0.7999 | \n0.8437 | \n- | \n- | \n
| 1DCNN-BERT | \n0.7834 | \n0.8385 | \n- | \n- | \n
| Mix of 1DCNN & Transformer | \n0.7883 | \n0.8282 | \n- | \n- | \n
| 5 Models Ensemble | \n0.8211 | \n0.8945 | \n0.868 | \n0.846 | \n
| Ensemble + Post-process | \n- | \n0.9044 | \n0.884 | \n0.858 | \n
Congratulations to all the winners! I would also like to express my gratitude to the organizers for preparing such an exciting competition.
\nI devoted a significant amount of time and effort to this competition from the very beginning, so I am truly delighted that this has resulted in a gold medal.
The key factors in my approach were (1) building diverse models and ensembling them, and (2) applying post-processing by leveraging previously inferred test samples. The idea of creating diverse models was inspired by my past gold medal solution in the HMS competition. Since HMS also dealt with time-series sensor data, some of the model ideas in this competition came directly from that experience.
\nThis idea was introduced by a teammate in the HMS competition and achieved the best performance among my five models. The input images for the 2DCNN are as follows. All signals except TOF were stacked into 16 pixels, while TOF was left unstacked. Upscaling the width to double (448 pixels in width) further improved the score. The model architecture is EfficientNetV2-B2 (B1 for IMU-only).
\n
This was the idea I personally used in HMS. The input images for the 2DCNN are as follows. TOF part processed the same as in raw-2DCNN. The model architecture is EfficientNetV2-B1.
\n
A lightly modified version of a public notebook model.
\nSame as iii), except the GRU-Attention module was replaced with a BERT.
\nBased on models used in past competition solutions, but adjusted with smaller dim, ksize, and fewer layers.
\nTwo days before the competition deadline, I discovered this technique and achieved a significant score boost (~+0.010). The basic idea is similar to the one discussed in this thread.
\nI modified the prediction() function in the submission code into a class method, storing the predictions of previously inferred samples in self.history variable. This allowed the model to access past predictions. For each subject ID, I counted predictions per class, then penalized predictions belonging to classes with high counts.
| Model | \nIMU only CV | \nAll CV | \nPublic LB | \nPrivate LB | \n
|---|---|---|---|---|
| raw-2DCNN | \n0.7972 | \n0.8773 | \n- | \n- | \n
| plot-2DCNN | \n0.7695 | \n0.8632 | \n- | \n- | \n
| 1DCNN-GRU-Attention | \n0.7999 | \n0.8437 | \n- | \n- | \n
| 1DCNN-BERT | \n0.7834 | \n0.8385 | \n- | \n- | \n
| Mix of 1DCNN & Transformer | \n0.7883 | \n0.8282 | \n- | \n- | \n
| 5 Models Ensemble | \n0.8211 | \n0.8945 | \n0.868 | \n0.846 | \n
| Ensemble + Post-process | \n- | \n0.9044 | \n0.884 | \n0.858 | \n
Congrats on the gold! Your image models are very strong, surprisingly they came out better than 1d cnn. I couldn't even get the image models close to raw 1d models
", "rawMarkdown": "Congrats on the gold! Your image models are very strong, surprisingly they came out better than 1d cnn. I couldn't even get the image models close to raw 1d models", "votes": null }, { "id": "3281111", "postDate": "09/03/2025 16:35:54", "content": "Congrats on your solution I really liked it, I knew it all the time I even saw your solution in the HMS 2 months ago and i was almost sure you and light source used cnn models on plots or spectogram, I should've worked more on the idea myself and stop following public baselines but that is a lesson for me
", "rawMarkdown": "Congrats on your solution I really liked it, I knew it all the time I even saw your solution in the HMS 2 months ago and i was almost sure you and light source used cnn models on plots or spectogram, I should've worked more on the idea myself and stop following public baselines but that is a lesson for me", "votes": null }, { "id": "3281257", "postDate": "09/03/2025 23:47:19", "content": "Congratulations on finishing in the gold zone!
\nI'm truly impressed by your excellent 2D-based solution. (I should have spent more time digging into 2D models…)
Since we both joined this competition from the early stages, I’ve personally seen you as a friendly rival throughout.
\nI’m really happy that both of us were able to finish with good results in the end :)
Due to GPU limitations and long training time, I give up the 2D model to process ToF. It's amazing to see your method.
", "rawMarkdown": "Due to GPU limitations and long training time, I give up the 2D model to process ToF. It's amazing to see your method.", "votes": null }, { "id": "3281525", "postDate": "09/04/2025 13:27:38", "content": "I had the idea of combining raw data with a type of image data. However, I failed in using spectrograms because the number of samples over time was quite small. I also thought about plotting images but haven't tried it yet. Thank you for letting me know that plotting images could be a workable solution. However, it seems my concern was correct. The accuracy on the IMU-only model is somewhat low when only using plot images.
", "rawMarkdown": "I had the idea of combining raw data with a type of image data. However, I failed in using spectrograms because the number of samples over time was quite small. I also thought about plotting images but haven't tried it yet. Thank you for letting me know that plotting images could be a workable solution. However, it seems my concern was correct. The accuracy on the IMU-only model is somewhat low when only using plot images.", "votes": null }, { "id": "3282911", "postDate": "09/07/2025 01:55:26", "content": "Congratulations!
\n\n\nAll signals except TOF were stacked into 16 pixels,
\n
25x16, do you have 25 features? What is this number 16?
", "rawMarkdown": "Congratulations!\n\n> All signals except TOF were stacked into 16 pixels,\n\n25x16, do you have 25 features? What is this number 16?", "votes": null } ], "comments": [ { "id": 3281096, "author_name": "l1ghtsource", "author_url": "", "post_date": "09/03/2025 16:15:32", "content": "Congrats on the gold! Your image models are very strong, surprisingly they came out better than 1d cnn. I couldn't even get the image models close to raw 1d models
", "votes": null, "replies": [] }, { "id": 3281111, "author_name": "tahaalshatiri", "author_url": "", "post_date": "09/03/2025 16:35:54", "content": "Congrats on your solution I really liked it, I knew it all the time I even saw your solution in the HMS 2 months ago and i was almost sure you and light source used cnn models on plots or spectogram, I should've worked more on the idea myself and stop following public baselines but that is a lesson for me
", "votes": null, "replies": [] }, { "id": 3281257, "author_name": "ryosukesaito", "author_url": "", "post_date": "09/03/2025 23:47:19", "content": "Congratulations on finishing in the gold zone!
\nI'm truly impressed by your excellent 2D-based solution. (I should have spent more time digging into 2D models…)
Since we both joined this competition from the early stages, I’ve personally seen you as a friendly rival throughout.
\nI’m really happy that both of us were able to finish with good results in the end :)
Due to GPU limitations and long training time, I give up the 2D model to process ToF. It's amazing to see your method.
", "votes": null, "replies": [] }, { "id": 3281525, "author_name": "quan0095", "author_url": "", "post_date": "09/04/2025 13:27:38", "content": "I had the idea of combining raw data with a type of image data. However, I failed in using spectrograms because the number of samples over time was quite small. I also thought about plotting images but haven't tried it yet. Thank you for letting me know that plotting images could be a workable solution. However, it seems my concern was correct. The accuracy on the IMU-only model is somewhat low when only using plot images.
", "votes": null, "replies": [] }, { "id": 3282911, "author_name": "minghigh", "author_url": "", "post_date": "09/07/2025 01:55:26", "content": "Congratulations!
\n\n\nAll signals except TOF were stacked into 16 pixels,
\n
25x16, do you have 25 features? What is this number 16?
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "3281087": "Congratulations to all the winners! I would also like to express my gratitude to the organizers for preparing such an exciting competition.\nI devoted a significant amount of time and effort to this competition from the very beginning, so I am truly delighted that this has resulted in a gold medal.\n\n# Overall Summary\n\nThe key factors in my approach were (1) building diverse models and ensembling them, and (2) applying post-processing by leveraging previously inferred test samples. The idea of creating diverse models was inspired by my past gold medal solution in the [HMS competition](https://www.kaggle.com/competitions/hms-harmful-brain-activity-classification). Since HMS also dealt with time-series sensor data, some of the model ideas in this competition came directly from that experience.\n\n# Models\n\n## Common Settings Across All Models\n\n* Trained separate models for **IMU only** and **IMU + TOF + THM (All)**.\n* Trained five different models (i–v) and ensembled them via weighted averaging.\n* Used publicly shared IMU feature sets.\n* 18-class output.\n* Employed standard **Cross-Entropy loss**.\n* Augmentation methods varied slightly by model, but **mixup** was consistently effective (except for plot-2DCNN).\n* For sequences longer than 224, used the last 224 timesteps; for shorter sequences, applied padding.\n* Preprocessing: applied **mean-std normalization** (except for plot-2DCNN, which used fixed-value clipping).\n* StratifiedGroupKFold (5-folds)\n\n## i) raw-2DCNN\n\nThis idea was introduced by a teammate in the HMS competition and achieved the best performance among my five models. The input images for the 2DCNN are as follows. All signals except TOF were stacked into 16 pixels, while TOF was left unstacked. Upscaling the width to double (448 pixels in width) further improved the score. The model architecture is EfficientNetV2-B2 (B1 for IMU-only).\n\n\n\n## ii) plot-2DCNN\n\nThis was the idea I personally used in HMS. The input images for the 2DCNN are as follows. TOF part processed the same as in raw-2DCNN. The model architecture is EfficientNetV2-B1.\n\n\n\n## iii) 1DCNN-GRU-Attention\n\nA lightly modified version of a public notebook model.\n\n## iv) 1DCNN-BERT\n\nSame as iii), except the GRU-Attention module was replaced with a BERT.\n\n## v) Mix of 1DCNN & Transformer\n\nBased on [models used in past competition solutions](https://www.kaggle.com/competitions/asl-signs/writeups/hoyeol-sohn-1st-place-solution-1dcnn-combined-with), but adjusted with smaller dim, ksize, and fewer layers.\n\n# Post-processing\n\nTwo days before the competition deadline, I discovered this technique and achieved a significant score boost (\\~+0.010). The basic idea is similar to the one discussed in [this thread](https://www.kaggle.com/competitions/cmi-detect-behavior-with-sensor-data/discussion/603544).\nI modified the `prediction()` function in the submission code into a class method, storing the predictions of previously inferred samples in `self.history` variable. This allowed the model to access past predictions. For each subject ID, I counted predictions per class, then penalized predictions belonging to classes with high counts.\n\n# CV & LB\n\n|Model|IMU only CV|All CV|Public LB|Private LB|\n| --- | --- | --- | --- | --- |\n|raw-2DCNN|0.7972|0.8773|-|-|\n|plot-2DCNN|0.7695|0.8632|-|-|\n|1DCNN-GRU-Attention|0.7999|0.8437|-|-|\n|1DCNN-BERT|0.7834|0.8385|-|-|\n|Mix of 1DCNN & Transformer|0.7883|0.8282|-|-|\n|5 Models Ensemble|0.8211|0.8945|0.868|0.846|\n|Ensemble + Post-process|-|0.9044|0.884|0.858|", "3281096": "Congrats on the gold! Your image models are very strong, surprisingly they came out better than 1d cnn. I couldn't even get the image models close to raw 1d models", "3281111": "Congrats on your solution I really liked it, I knew it all the time I even saw your solution in the HMS 2 months ago and i was almost sure you and light source used cnn models on plots or spectogram, I should've worked more on the idea myself and stop following public baselines but that is a lesson for me", "3281257": "Congratulations on finishing in the gold zone!\nI'm truly impressed by your excellent 2D-based solution. (I should have spent more time digging into 2D models…)\n\nSince we both joined this competition from the early stages, I’ve personally seen you as a friendly rival throughout.\nI’m really happy that both of us were able to finish with good results in the end :)", "3281404": "Due to GPU limitations and long training time, I give up the 2D model to process ToF. It's amazing to see your method.", "3281525": "I had the idea of combining raw data with a type of image data. However, I failed in using spectrograms because the number of samples over time was quite small. I also thought about plotting images but haven't tried it yet. Thank you for letting me know that plotting images could be a workable solution. However, it seems my concern was correct. The accuracy on the IMU-only model is somewhat low when only using plot images.", "3282911": "Congratulations!\n\n> All signals except TOF were stacked into 16 pixels,\n\n25x16, do you have 25 features? What is this number 16?" }, "source": "meta" }