{
  "id": 603684,
  "title": "107th place solution",
  "url": "/competitions/cmi-detect-behavior-with-sensor-data/discussion/603684",
  "author_name": "Pi",
  "post_date": "2025-09-03T17:28:55.865000",
  "votes": 4,
  "comment_count": 0,
  "views": 0,
  "content": "<p>Huge thanks to the organizers and fellow participants for making this competition such a fun and rewarding journey! 🎉 Congrats to the winners, and cheers to everyone who learned, shared, and grew along the way 🚀</p>\n<p>Especially, we would like to thank these following people together with their amazing notebooks from which we learned how to process data:<br>\n(1) <a href=\"https://www.kaggle.com/jiazhuang\" target=\"_blank\">@jiazhuang</a> for <a href=\"https://www.kaggle.com/code/jiazhuang/cmi-imu-only-lstm\" target=\"_blank\">https://www.kaggle.com/code/jiazhuang/cmi-imu-only-lstm</a><br>\n(2) <a href=\"https://www.kaggle.com/pepushi\" target=\"_blank\">@pepushi</a> for <a href=\"https://www.kaggle.com/code/pepushi/imu-tof-thm-fusion-with-physics-fe-mixup-lb0-8\" target=\"_blank\">https://www.kaggle.com/code/pepushi/imu-tof-thm-fusion-with-physics-fe-mixup-lb0-8</a><br>\n(3) <a href=\"https://www.kaggle.com/wasupandceacar\" target=\"_blank\">@wasupandceacar</a> for <a href=\"https://www.kaggle.com/code/wasupandceacar/lb-0-841-5fold-single-model-with-split-sensors\" target=\"_blank\">https://www.kaggle.com/code/wasupandceacar/lb-0-841-5fold-single-model-with-split-sensors</a> </p>\n<h2>Strategy</h2>\n<p><strong>(1) K-fold cross-validation with StratifiedGroupKFold</strong><br>\n2 ways for splitting data with StratifiedGroupKFold:</p>\n<blockquote>\n  <ul>\n  <li>group: subjects, stratify: gestures (SS1).</li>\n  <li>group: subjects, stratify: gestures + missing Helios data (THM/TOF) (SS2).</li>\n  </ul>\n</blockquote>\n<p><strong>(2) Diversify model architectures and training parameters for boosting ensemble.</strong> </p>\n<p>We found this article very useful to have better training strategy for ensemble: <a href=\"https://dl.acm.org/doi/pdf/10.5555/1046920.1194899\" target=\"_blank\">Managing Diversity in Regression Ensembles</a>.</p>\n<p>The single error can be decomposed into:</p>\n<p>$$<br>\n\\mathbb{E}[(y-\\hat{f}(x))^2] = \\text{Bias}^2 \\ (\\text{underfitting}) + \\text{Variance} \\ (\\text{overfitting})<br>\n$$</p>\n<p>The ensemble error:<br>\n$$<br>\nE\\left[(\\bar{f} - t)^2\\right] = \\overline{\\text{bias}}^2 + \\frac{1}{M} \\, \\overline{\\text{var}} + \\left(1 - \\frac{1}{M} \\right) \\overline{\\text{covar}}<br>\n$$</p>\n<p>where:</p>\n<ul>\n<li>\\(\\overline{\\text{bias}}\\) is the averaged bias of the ensemble members</li>\n<li>\\(\\overline{\\text{var}}\\) is the averaged variance of the ensemble members</li>\n<li>\\(\\overline{\\text{covar}}\\) is the averaged covariance of the ensemble members</li>\n</ul>\n<p>Based on this, the ensemble will help to decrease variance, thus decreasing the error. However at the same time if the models we used for ensemble are correlated, it will increase the error through this component \\((1 - \\frac{1}{M}) \\overline{\\text{covar}}\\). Therefore, we try to diversify the model by:</p>\n<blockquote>\n  <ul>\n  <li>training models on different frameworks: Tensorflow and Pytorch</li>\n  <li>splitting more folds 10-20 folds and each fold train with different seeds</li>\n  <li>building model with different input features</li>\n  <li>different outputs: (a) 18 target and non-target gestures; (b) 8 target gestures + 1 grouped non-target gesture.</li>\n  </ul>\n</blockquote>\n<p><strong>(3) Analyze the model performance to gain insights and directions to improve the results.</strong></p>\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F878726%2Fb5f7df036e70e5a040079374a6514a3b%2Fchart.svg?generation=1757512135711327&amp;alt=media\" alt=\"performance-analysis\"></p>\n<p>We started by building a performance analysis pipeline to see how model accuracy relates to different features and to spot opportunities for improvement.  <br>\nSome findings when analyzing IMU only and IMU-THM-TOF models:</p>\n<blockquote>\n  <ul>\n  <li>When THM and TOF data are missing, the IMU only model outperforms the IMU-THM-TOF model --&gt; asign higher weights for IMU_only model when THM &amp; TOF are not available.</li>\n  <li>Both models achieve higher accuracy on adults, males, and right hands compared to children, females, and left hands. However, this gap may be influenced by the co-occurrence of missing rotation/THM/TOF data with these groups. --&gt; upsampling for children, females, and left hands during training.</li>\n  <li>No significant performance differences are observed across height, shoulder-to-wrist length, or elbow-to-wrist length. --&gt; no need to use these features during training.</li>\n  </ul>\n</blockquote>\n<p>For example, here is the performance when TOF is missing:</p>\n<ul>\n<li><p>IMU only model: the results of TOF missing or present are not different significantly.<br>\n<img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F878726%2F3b4861850677c6dc2e3b462f72cf009b%2FIMU_only.png?generation=1756920162531734&amp;alt=media\" alt=\"IMU-only\"></p></li>\n<li><p>IMU-THM-TOF: When TOF is missing, the results drop significantly.<br>\n<img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F878726%2Fe709f7ba35941c0a4b62c55d68af0669%2FIMU_THM_TOF.png?generation=1756920222650364&amp;alt=media\" alt=\" IMU-THM-TOF\"></p></li>\n</ul>\n<h2>Implementation</h2>\n<p>Some selections of our trained models:</p>\n<table>\n<thead>\n<tr>\n<th>model ID</th>\n<th>features</th>\n<th>architecture</th>\n<th>splitting</th>\n<th>framework</th>\n<th>CV</th>\n<th>public LB</th>\n<th>private LB</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>M1</td>\n<td>IMU only (maxlen=256, seed=2025)</td>\n<td><a href=\"https://www.kaggle.com/code/jiazhuang/cmi-imu-only-lstm\" target=\"_blank\">cmi imu-only lstm by jiazhuang</a></td>\n<td>20-fold SS1</td>\n<td>Pytorch</td>\n<td>0.789</td>\n<td>0.799</td>\n<td>0.789</td>\n</tr>\n<tr>\n<td>M1</td>\n<td>IMU only (maxlen=256, seed=42)</td>\n<td><a href=\"https://www.kaggle.com/code/jiazhuang/cmi-imu-only-lstm\" target=\"_blank\">cmi imu-only lstm from jiazhuang</a></td>\n<td>20-fold SS1</td>\n<td>Pytorch</td>\n<td>0.785</td>\n<td>0.798</td>\n<td>0.791</td>\n</tr>\n<tr>\n<td>M1+M2</td>\n<td></td>\n<td></td>\n<td></td>\n<td></td>\n<td></td>\n<td>0.801</td>\n<td>0.796</td>\n</tr>\n<tr>\n<td>M3</td>\n<td>IMU+THM+TOF</td>\n<td><a href=\"https://www.kaggle.com/code/pepushi/imu-tof-thm-fusion-with-physics-fe-mixup-lb0-8\" target=\"_blank\">IMU × TOF/THM Fusion with Physics‑FE &amp; MixUp by pepushi</a></td>\n<td>20-fold SS1</td>\n<td>Tensorflow</td>\n<td>0.826</td>\n<td>0.825</td>\n<td>0.813</td>\n</tr>\n<tr>\n<td>M4</td>\n<td>IMU+THM+TOF + raw TOF</td>\n<td><a href=\"https://www.kaggle.com/code/wasupandceacar/lb-0-841-5fold-single-model-with-split-sensors\" target=\"_blank\">wasupandceacar's architecture</a> with modified some channels</td>\n<td>10-fold SS2</td>\n<td>Pytorch</td>\n<td></td>\n<td>0.833</td>\n<td>0.815</td>\n</tr>\n<tr>\n<td>M5</td>\n<td>IMU+THM+TOF + raw TOF</td>\n<td><a href=\"https://www.kaggle.com/code/wasupandceacar/lb-0-841-5fold-single-model-with-split-sensors\" target=\"_blank\">wasupandceacar's architecture</a> with modified some channels</td>\n<td>10-fold SS2</td>\n<td>Tensorflow</td>\n<td></td>\n<td>0.837</td>\n<td>0.815</td>\n</tr>\n<tr>\n<td>M4+M5</td>\n<td></td>\n<td></td>\n<td></td>\n<td></td>\n<td></td>\n<td>0.837</td>\n<td>0.819</td>\n</tr>\n<tr>\n<td>M6</td>\n<td>IMU+THM+TOF</td>\n<td><a href=\"https://www.kaggle.com/code/pepushi/imu-tof-thm-fusion-with-physics-fe-mixup-lb0-8\" target=\"_blank\">IMU × TOF/THM Fusion with Physics‑FE &amp; MixUp by pepushi</a> with 9-class output</td>\n<td>20-fold SS1</td>\n<td>Tensorflow</td>\n<td>0.811</td>\n<td>0.794</td>\n<td>0.800</td>\n</tr>\n</tbody>\n</table>\n<hr>\n<p>Public shared models</p>\n<table>\n<thead>\n<tr>\n<th>model ID</th>\n<th>link</th>\n<th>version</th>\n<th>CV</th>\n<th>public LB</th>\n<th>private LB</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>P1</td>\n<td><a href=\"https://www.kaggle.com/code/wasupandceacar/lb-0-841-5fold-single-model-with-split-sensors\" target=\"_blank\">LB 0.841-5fold single model with split sensors</a> by <a href=\"https://www.kaggle.com/wasupandceacar\" target=\"_blank\">wasupandceacar</a></td>\n<td>v4</td>\n<td>0.861</td>\n<td>0.841</td>\n<td>0.824</td>\n</tr>\n<tr>\n<td>P2</td>\n<td><a href=\"https://www.kaggle.com/code/pepushi/gated-gru-hybrid-ensemble-v02\" target=\"_blank\">Gated GRU + Hybrid Ensemble_v02</a> by <a href=\"https://www.kaggle.com/pepushi\" target=\"_blank\">pepushi</a></td>\n<td>v10</td>\n<td></td>\n<td>0.835</td>\n<td>0.816</td>\n</tr>\n<tr>\n<td>P3</td>\n<td><a href=\"https://www.kaggle.com/code/wasupandceacar/lb-0-82-5fold-single-bert-model\" target=\"_blank\">LB 0.82-5fold single bert model</a> by<a href=\"https://www.kaggle.com/wasupandceacar\" target=\"_blank\">wasupandceacar</a></td>\n<td>v7</td>\n<td></td>\n<td>0.829</td>\n<td>0.809</td>\n</tr>\n</tbody>\n</table>\n<hr>\n<p>Ensemble(our self-trained models in bold)</p>\n<table>\n<thead>\n<tr>\n<th>ensemble ID</th>\n<th>weights</th>\n<th>Public LB</th>\n<th>private LB</th>\n<th>selected</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>E1</td>\n<td>if THM &amp; TOF are missing {<strong>M1: 0.5</strong>, P1: 0.5} else {<strong>M3: 0.5</strong>, P1: 0.5}</td>\n<td>0.849</td>\n<td>0.834</td>\n<td>no</td>\n</tr>\n<tr>\n<td>E2</td>\n<td>if THM &amp; TOF are missing {<strong>M1: 0.4</strong>, P1: 0.4, P2: 0.2} else {<strong>M3: 0.25</strong>, P1: 0.5, P2: 0.25}</td>\n<td>0.849</td>\n<td>0.837</td>\n<td>no</td>\n</tr>\n<tr>\n<td>E3</td>\n<td>if THM &amp; TOF are missing {<strong>M1: 0.4</strong>, P1: 0.4, P2: 0.2} else {<strong>M3: 0.5</strong>, P1: 0.5}</td>\n<td>0.850</td>\n<td><strong>0.838</strong></td>\n<td>no</td>\n</tr>\n<tr>\n<td>E4</td>\n<td>if THM &amp; TOF are missing {<strong>M1+M2: 0.3</strong>, P1: 0.3, P2: 0.2, P3: 0.2} else {<strong>M3: 0.1</strong>, P1: 0.4, P2: 0.25, P3: 0.25}</td>\n<td>0.855</td>\n<td>0.834</td>\n<td>yes</td>\n</tr>\n<tr>\n<td>E4</td>\n<td>if THM &amp; TOF are missing {<strong>M1+M2: 0.3</strong>, P1: 0.3, P2: 0.2, P3: 0.2} else {P1: 0.4, P2: 0.35, P3: 0.35}</td>\n<td><strong>0.857</strong></td>\n<td>0.834</td>\n<td>yes</td>\n</tr>\n</tbody>\n</table>",
  "messages": [
    {
      "id": 3281132,
      "postDate": "2025-09-03T17:28:55.867Z",
      "content": "<p>Huge thanks to the organizers and fellow participants for making this competition such a fun and rewarding journey! 🎉 Congrats to the winners, and cheers to everyone who learned, shared, and grew along the way 🚀</p>\n<p>Especially, we would like to thank these following people together with their amazing notebooks from which we learned how to process data:<br>\n(1) <a href=\"https://www.kaggle.com/jiazhuang\" target=\"_blank\">@jiazhuang</a> for <a href=\"https://www.kaggle.com/code/jiazhuang/cmi-imu-only-lstm\" target=\"_blank\">https://www.kaggle.com/code/jiazhuang/cmi-imu-only-lstm</a><br>\n(2) <a href=\"https://www.kaggle.com/pepushi\" target=\"_blank\">@pepushi</a> for <a href=\"https://www.kaggle.com/code/pepushi/imu-tof-thm-fusion-with-physics-fe-mixup-lb0-8\" target=\"_blank\">https://www.kaggle.com/code/pepushi/imu-tof-thm-fusion-with-physics-fe-mixup-lb0-8</a><br>\n(3) <a href=\"https://www.kaggle.com/wasupandceacar\" target=\"_blank\">@wasupandceacar</a> for <a href=\"https://www.kaggle.com/code/wasupandceacar/lb-0-841-5fold-single-model-with-split-sensors\" target=\"_blank\">https://www.kaggle.com/code/wasupandceacar/lb-0-841-5fold-single-model-with-split-sensors</a> </p>\n<h2>Strategy</h2>\n<p><strong>(1) K-fold cross-validation with StratifiedGroupKFold</strong><br>\n2 ways for splitting data with StratifiedGroupKFold:</p>\n<blockquote>\n  <ul>\n  <li>group: subjects, stratify: gestures (SS1).</li>\n  <li>group: subjects, stratify: gestures + missing Helios data (THM/TOF) (SS2).</li>\n  </ul>\n</blockquote>\n<p><strong>(2) Diversify model architectures and training parameters for boosting ensemble.</strong> </p>\n<p>We found this article very useful to have better training strategy for ensemble: <a href=\"https://dl.acm.org/doi/pdf/10.5555/1046920.1194899\" target=\"_blank\">Managing Diversity in Regression Ensembles</a>.</p>\n<p>The single error can be decomposed into:</p>\n<p>$$<br>\n\\mathbb{E}[(y-\\hat{f}(x))^2] = \\text{Bias}^2 \\ (\\text{underfitting}) + \\text{Variance} \\ (\\text{overfitting})<br>\n$$</p>\n<p>The ensemble error:<br>\n$$<br>\nE\\left[(\\bar{f} - t)^2\\right] = \\overline{\\text{bias}}^2 + \\frac{1}{M} \\, \\overline{\\text{var}} + \\left(1 - \\frac{1}{M} \\right) \\overline{\\text{covar}}<br>\n$$</p>\n<p>where:</p>\n<ul>\n<li>\\(\\overline{\\text{bias}}\\) is the averaged bias of the ensemble members</li>\n<li>\\(\\overline{\\text{var}}\\) is the averaged variance of the ensemble members</li>\n<li>\\(\\overline{\\text{covar}}\\) is the averaged covariance of the ensemble members</li>\n</ul>\n<p>Based on this, the ensemble will help to decrease variance, thus decreasing the error. However at the same time if the models we used for ensemble are correlated, it will increase the error through this component \\((1 - \\frac{1}{M}) \\overline{\\text{covar}}\\). Therefore, we try to diversify the model by:</p>\n<blockquote>\n  <ul>\n  <li>training models on different frameworks: Tensorflow and Pytorch</li>\n  <li>splitting more folds 10-20 folds and each fold train with different seeds</li>\n  <li>building model with different input features</li>\n  <li>different outputs: (a) 18 target and non-target gestures; (b) 8 target gestures + 1 grouped non-target gesture.</li>\n  </ul>\n</blockquote>\n<p><strong>(3) Analyze the model performance to gain insights and directions to improve the results.</strong></p>\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F878726%2Fb5f7df036e70e5a040079374a6514a3b%2Fchart.svg?generation=1757512135711327&amp;alt=media\" alt=\"performance-analysis\"></p>\n<p>We started by building a performance analysis pipeline to see how model accuracy relates to different features and to spot opportunities for improvement.  <br>\nSome findings when analyzing IMU only and IMU-THM-TOF models:</p>\n<blockquote>\n  <ul>\n  <li>When THM and TOF data are missing, the IMU only model outperforms the IMU-THM-TOF model --&gt; asign higher weights for IMU_only model when THM &amp; TOF are not available.</li>\n  <li>Both models achieve higher accuracy on adults, males, and right hands compared to children, females, and left hands. However, this gap may be influenced by the co-occurrence of missing rotation/THM/TOF data with these groups. --&gt; upsampling for children, females, and left hands during training.</li>\n  <li>No significant performance differences are observed across height, shoulder-to-wrist length, or elbow-to-wrist length. --&gt; no need to use these features during training.</li>\n  </ul>\n</blockquote>\n<p>For example, here is the performance when TOF is missing:</p>\n<ul>\n<li><p>IMU only model: the results of TOF missing or present are not different significantly.<br>\n<img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F878726%2F3b4861850677c6dc2e3b462f72cf009b%2FIMU_only.png?generation=1756920162531734&amp;alt=media\" alt=\"IMU-only\"></p></li>\n<li><p>IMU-THM-TOF: When TOF is missing, the results drop significantly.<br>\n<img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F878726%2Fe709f7ba35941c0a4b62c55d68af0669%2FIMU_THM_TOF.png?generation=1756920222650364&amp;alt=media\" alt=\" IMU-THM-TOF\"></p></li>\n</ul>\n<h2>Implementation</h2>\n<p>Some selections of our trained models:</p>\n<table>\n<thead>\n<tr>\n<th>model ID</th>\n<th>features</th>\n<th>architecture</th>\n<th>splitting</th>\n<th>framework</th>\n<th>CV</th>\n<th>public LB</th>\n<th>private LB</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>M1</td>\n<td>IMU only (maxlen=256, seed=2025)</td>\n<td><a href=\"https://www.kaggle.com/code/jiazhuang/cmi-imu-only-lstm\" target=\"_blank\">cmi imu-only lstm by jiazhuang</a></td>\n<td>20-fold SS1</td>\n<td>Pytorch</td>\n<td>0.789</td>\n<td>0.799</td>\n<td>0.789</td>\n</tr>\n<tr>\n<td>M1</td>\n<td>IMU only (maxlen=256, seed=42)</td>\n<td><a href=\"https://www.kaggle.com/code/jiazhuang/cmi-imu-only-lstm\" target=\"_blank\">cmi imu-only lstm from jiazhuang</a></td>\n<td>20-fold SS1</td>\n<td>Pytorch</td>\n<td>0.785</td>\n<td>0.798</td>\n<td>0.791</td>\n</tr>\n<tr>\n<td>M1+M2</td>\n<td></td>\n<td></td>\n<td></td>\n<td></td>\n<td></td>\n<td>0.801</td>\n<td>0.796</td>\n</tr>\n<tr>\n<td>M3</td>\n<td>IMU+THM+TOF</td>\n<td><a href=\"https://www.kaggle.com/code/pepushi/imu-tof-thm-fusion-with-physics-fe-mixup-lb0-8\" target=\"_blank\">IMU × TOF/THM Fusion with Physics‑FE &amp; MixUp by pepushi</a></td>\n<td>20-fold SS1</td>\n<td>Tensorflow</td>\n<td>0.826</td>\n<td>0.825</td>\n<td>0.813</td>\n</tr>\n<tr>\n<td>M4</td>\n<td>IMU+THM+TOF + raw TOF</td>\n<td><a href=\"https://www.kaggle.com/code/wasupandceacar/lb-0-841-5fold-single-model-with-split-sensors\" target=\"_blank\">wasupandceacar's architecture</a> with modified some channels</td>\n<td>10-fold SS2</td>\n<td>Pytorch</td>\n<td></td>\n<td>0.833</td>\n<td>0.815</td>\n</tr>\n<tr>\n<td>M5</td>\n<td>IMU+THM+TOF + raw TOF</td>\n<td><a href=\"https://www.kaggle.com/code/wasupandceacar/lb-0-841-5fold-single-model-with-split-sensors\" target=\"_blank\">wasupandceacar's architecture</a> with modified some channels</td>\n<td>10-fold SS2</td>\n<td>Tensorflow</td>\n<td></td>\n<td>0.837</td>\n<td>0.815</td>\n</tr>\n<tr>\n<td>M4+M5</td>\n<td></td>\n<td></td>\n<td></td>\n<td></td>\n<td></td>\n<td>0.837</td>\n<td>0.819</td>\n</tr>\n<tr>\n<td>M6</td>\n<td>IMU+THM+TOF</td>\n<td><a href=\"https://www.kaggle.com/code/pepushi/imu-tof-thm-fusion-with-physics-fe-mixup-lb0-8\" target=\"_blank\">IMU × TOF/THM Fusion with Physics‑FE &amp; MixUp by pepushi</a> with 9-class output</td>\n<td>20-fold SS1</td>\n<td>Tensorflow</td>\n<td>0.811</td>\n<td>0.794</td>\n<td>0.800</td>\n</tr>\n</tbody>\n</table>\n<hr>\n<p>Public shared models</p>\n<table>\n<thead>\n<tr>\n<th>model ID</th>\n<th>link</th>\n<th>version</th>\n<th>CV</th>\n<th>public LB</th>\n<th>private LB</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>P1</td>\n<td><a href=\"https://www.kaggle.com/code/wasupandceacar/lb-0-841-5fold-single-model-with-split-sensors\" target=\"_blank\">LB 0.841-5fold single model with split sensors</a> by <a href=\"https://www.kaggle.com/wasupandceacar\" target=\"_blank\">wasupandceacar</a></td>\n<td>v4</td>\n<td>0.861</td>\n<td>0.841</td>\n<td>0.824</td>\n</tr>\n<tr>\n<td>P2</td>\n<td><a href=\"https://www.kaggle.com/code/pepushi/gated-gru-hybrid-ensemble-v02\" target=\"_blank\">Gated GRU + Hybrid Ensemble_v02</a> by <a href=\"https://www.kaggle.com/pepushi\" target=\"_blank\">pepushi</a></td>\n<td>v10</td>\n<td></td>\n<td>0.835</td>\n<td>0.816</td>\n</tr>\n<tr>\n<td>P3</td>\n<td><a href=\"https://www.kaggle.com/code/wasupandceacar/lb-0-82-5fold-single-bert-model\" target=\"_blank\">LB 0.82-5fold single bert model</a> by<a href=\"https://www.kaggle.com/wasupandceacar\" target=\"_blank\">wasupandceacar</a></td>\n<td>v7</td>\n<td></td>\n<td>0.829</td>\n<td>0.809</td>\n</tr>\n</tbody>\n</table>\n<hr>\n<p>Ensemble(our self-trained models in bold)</p>\n<table>\n<thead>\n<tr>\n<th>ensemble ID</th>\n<th>weights</th>\n<th>Public LB</th>\n<th>private LB</th>\n<th>selected</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>E1</td>\n<td>if THM &amp; TOF are missing {<strong>M1: 0.5</strong>, P1: 0.5} else {<strong>M3: 0.5</strong>, P1: 0.5}</td>\n<td>0.849</td>\n<td>0.834</td>\n<td>no</td>\n</tr>\n<tr>\n<td>E2</td>\n<td>if THM &amp; TOF are missing {<strong>M1: 0.4</strong>, P1: 0.4, P2: 0.2} else {<strong>M3: 0.25</strong>, P1: 0.5, P2: 0.25}</td>\n<td>0.849</td>\n<td>0.837</td>\n<td>no</td>\n</tr>\n<tr>\n<td>E3</td>\n<td>if THM &amp; TOF are missing {<strong>M1: 0.4</strong>, P1: 0.4, P2: 0.2} else {<strong>M3: 0.5</strong>, P1: 0.5}</td>\n<td>0.850</td>\n<td><strong>0.838</strong></td>\n<td>no</td>\n</tr>\n<tr>\n<td>E4</td>\n<td>if THM &amp; TOF are missing {<strong>M1+M2: 0.3</strong>, P1: 0.3, P2: 0.2, P3: 0.2} else {<strong>M3: 0.1</strong>, P1: 0.4, P2: 0.25, P3: 0.25}</td>\n<td>0.855</td>\n<td>0.834</td>\n<td>yes</td>\n</tr>\n<tr>\n<td>E4</td>\n<td>if THM &amp; TOF are missing {<strong>M1+M2: 0.3</strong>, P1: 0.3, P2: 0.2, P3: 0.2} else {P1: 0.4, P2: 0.35, P3: 0.35}</td>\n<td><strong>0.857</strong></td>\n<td>0.834</td>\n<td>yes</td>\n</tr>\n</tbody>\n</table>",
      "rawMarkdown": "Huge thanks to the organizers and fellow participants for making this competition such a fun and rewarding journey! 🎉 Congrats to the winners, and cheers to everyone who learned, shared, and grew along the way 🚀\n\nEspecially, we would like to thank these following people together with their amazing notebooks from which we learned how to process data:\n(1) @jiazhuang for https://www.kaggle.com/code/jiazhuang/cmi-imu-only-lstm\n(2) @pepushi for https://www.kaggle.com/code/pepushi/imu-tof-thm-fusion-with-physics-fe-mixup-lb0-8\n(3) @wasupandceacar for https://www.kaggle.com/code/wasupandceacar/lb-0-841-5fold-single-model-with-split-sensors \n\n## Strategy\n\n**(1) K-fold cross-validation with StratifiedGroupKFold**\n2 ways for splitting data with StratifiedGroupKFold:\n>* group: subjects, stratify: gestures (SS1).\n* group: subjects, stratify: gestures + missing Helios data (THM/TOF) (SS2).\n\n**(2) Diversify model architectures and training parameters for boosting ensemble.** \n\nWe found this article very useful to have better training strategy for ensemble: [Managing Diversity in Regression Ensembles](https://dl.acm.org/doi/pdf/10.5555/1046920.1194899).\n\nThe single error can be decomposed into:\n\n$$\n\\mathbb{E}[(y-\\hat{f}(x))^2] = \\text{Bias}^2 \\ (\\text{underfitting}) + \\text{Variance} \\ (\\text{overfitting})\n$$\n\nThe ensemble error:\n$$\nE\\left[(\\bar{f} - t)^2\\right] = \\overline{\\text{bias}}^2 + \\frac{1}{M} \\, \\overline{\\text{var}} + \\left(1 - \\frac{1}{M} \\right) \\overline{\\text{covar}}\n$$\n\nwhere:\n* \\\\(\\overline{\\text{bias}}\\\\) is the averaged bias of the ensemble members\n*  \\\\(\\overline{\\text{var}}\\\\) is the averaged variance of the ensemble members\n*  \\\\(\\overline{\\text{covar}}\\\\) is the averaged covariance of the ensemble members\n\nBased on this, the ensemble will help to decrease variance, thus decreasing the error. However at the same time if the models we used for ensemble are correlated, it will increase the error through this component \\\\((1 - \\frac{1}{M}) \\overline{\\text{covar}}\\\\). Therefore, we try to diversify the model by:\n\n>* training models on different frameworks: Tensorflow and Pytorch\n* splitting more folds 10-20 folds and each fold train with different seeds\n* building model with different input features\n* different outputs: (a) 18 target and non-target gestures; (b) 8 target gestures + 1 grouped non-target gesture.\n\n**(3) Analyze the model performance to gain insights and directions to improve the results.**\n\n![performance-analysis](https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F878726%2Fb5f7df036e70e5a040079374a6514a3b%2Fchart.svg?generation=1757512135711327&alt=media)\n\nWe started by building a performance analysis pipeline to see how model accuracy relates to different features and to spot opportunities for improvement.  \nSome findings when analyzing IMU only and IMU-THM-TOF models:\n\n>* When THM and TOF data are missing, the IMU only model outperforms the IMU-THM-TOF model --> asign higher weights for IMU_only model when THM & TOF are not available.\n* Both models achieve higher accuracy on adults, males, and right hands compared to children, females, and left hands. However, this gap may be influenced by the co-occurrence of missing rotation/THM/TOF data with these groups. --> upsampling for children, females, and left hands during training.\n* No significant performance differences are observed across height, shoulder-to-wrist length, or elbow-to-wrist length. --> no need to use these features during training.\n\nFor example, here is the performance when TOF is missing:\n* IMU only model: the results of TOF missing or present are not different significantly.\n![IMU-only](https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F878726%2F3b4861850677c6dc2e3b462f72cf009b%2FIMU_only.png?generation=1756920162531734&alt=media)\n\n* IMU-THM-TOF: When TOF is missing, the results drop significantly.\n![ IMU-THM-TOF](https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F878726%2Fe709f7ba35941c0a4b62c55d68af0669%2FIMU_THM_TOF.png?generation=1756920222650364&alt=media)\n\n## Implementation\n\nSome selections of our trained models:\n\n| model ID | features |architecture|splitting|framework |CV|public LB|private LB|\n| --- | --- | --- | --- | --- | --- | --- | --- |\n| M1 | IMU only (maxlen=256, seed=2025)|[cmi imu-only lstm by jiazhuang](https://www.kaggle.com/code/jiazhuang/cmi-imu-only-lstm)|20-fold SS1|Pytorch|0.789|0.799|0.789|\n| M1 | IMU only (maxlen=256, seed=42)|[cmi imu-only lstm from jiazhuang](https://www.kaggle.com/code/jiazhuang/cmi-imu-only-lstm)|20-fold SS1|Pytorch|0.785|0.798|0.791|\n|M1+M2||||||0.801|0.796|\n|M3|IMU+THM+TOF|[IMU × TOF/THM Fusion with Physics‑FE & MixUp by pepushi](https://www.kaggle.com/code/pepushi/imu-tof-thm-fusion-with-physics-fe-mixup-lb0-8)|20-fold SS1|Tensorflow|0.826|0.825|0.813|\n|M4|IMU+THM+TOF + raw TOF|[wasupandceacar's architecture](https://www.kaggle.com/code/wasupandceacar/lb-0-841-5fold-single-model-with-split-sensors) with modified some channels|10-fold SS2|Pytorch||0.833|0.815|\n|M5|IMU+THM+TOF + raw TOF|[wasupandceacar's architecture](https://www.kaggle.com/code/wasupandceacar/lb-0-841-5fold-single-model-with-split-sensors) with modified some channels|10-fold SS2|Tensorflow||0.837|0.815|\n|M4+M5||||||0.837|0.819|\n|M6|IMU+THM+TOF|[IMU × TOF/THM Fusion with Physics‑FE & MixUp by pepushi](https://www.kaggle.com/code/pepushi/imu-tof-thm-fusion-with-physics-fe-mixup-lb0-8) with 9-class output|20-fold SS1|Tensorflow|0.811|0.794|0.800|\n\n---\n\nPublic shared models\n\n| model ID | link |version|CV|public LB|private LB|\n| --- | --- | --- | --- | --- | --- |\n|P1|[LB 0.841-5fold single model with split sensors](https://www.kaggle.com/code/wasupandceacar/lb-0-841-5fold-single-model-with-split-sensors) by [wasupandceacar](https://www.kaggle.com/wasupandceacar) |v4|0.861|0.841|0.824|\n|P2|[Gated GRU + Hybrid Ensemble_v02](https://www.kaggle.com/code/pepushi/gated-gru-hybrid-ensemble-v02) by [pepushi](https://www.kaggle.com/pepushi)|v10||0.835|0.816|\n|P3|[LB 0.82-5fold single bert model](https://www.kaggle.com/code/wasupandceacar/lb-0-82-5fold-single-bert-model) by[wasupandceacar](https://www.kaggle.com/wasupandceacar)|v7||0.829|0.809|\n\n---\n\nEnsemble(our self-trained models in bold)\n\n|ensemble ID|weights|Public LB|private LB|selected|\n| --- | --- | --- | --- | --- | \n| E1 | if THM & TOF are missing {**M1: 0.5**, P1: 0.5} else {**M3: 0.5**, P1: 0.5}|0.849|0.834|no|\n| E2 | if THM & TOF are missing {**M1: 0.4**, P1: 0.4, P2: 0.2} else {**M3: 0.25**, P1: 0.5, P2: 0.25}|0.849|0.837|no|\n| E3 | if THM & TOF are missing {**M1: 0.4**, P1: 0.4, P2: 0.2} else {**M3: 0.5**, P1: 0.5}|0.850|**0.838**|no|\n| E4 | if THM & TOF are missing {**M1+M2: 0.3**, P1: 0.3, P2: 0.2, P3: 0.2} else {**M3: 0.1**, P1: 0.4, P2: 0.25, P3: 0.25}|0.855|0.834|yes|\n| E4 | if THM & TOF are missing {**M1+M2: 0.3**, P1: 0.3, P2: 0.2, P3: 0.2} else {P1: 0.4, P2: 0.35, P3: 0.35}|**0.857**|0.834|yes|\n\n\n",
      "votes": 4
    }
  ],
  "comments": [],
  "raw_markdown_by_id": {
    "3281132": "Huge thanks to the organizers and fellow participants for making this competition such a fun and rewarding journey! 🎉 Congrats to the winners, and cheers to everyone who learned, shared, and grew along the way 🚀\n\nEspecially, we would like to thank these following people together with their amazing notebooks from which we learned how to process data:\n(1) @jiazhuang for https://www.kaggle.com/code/jiazhuang/cmi-imu-only-lstm\n(2) @pepushi for https://www.kaggle.com/code/pepushi/imu-tof-thm-fusion-with-physics-fe-mixup-lb0-8\n(3) @wasupandceacar for https://www.kaggle.com/code/wasupandceacar/lb-0-841-5fold-single-model-with-split-sensors \n\n## Strategy\n\n**(1) K-fold cross-validation with StratifiedGroupKFold**\n2 ways for splitting data with StratifiedGroupKFold:\n>* group: subjects, stratify: gestures (SS1).\n* group: subjects, stratify: gestures + missing Helios data (THM/TOF) (SS2).\n\n**(2) Diversify model architectures and training parameters for boosting ensemble.** \n\nWe found this article very useful to have better training strategy for ensemble: [Managing Diversity in Regression Ensembles](https://dl.acm.org/doi/pdf/10.5555/1046920.1194899).\n\nThe single error can be decomposed into:\n\n$$\n\\mathbb{E}[(y-\\hat{f}(x))^2] = \\text{Bias}^2 \\ (\\text{underfitting}) + \\text{Variance} \\ (\\text{overfitting})\n$$\n\nThe ensemble error:\n$$\nE\\left[(\\bar{f} - t)^2\\right] = \\overline{\\text{bias}}^2 + \\frac{1}{M} \\, \\overline{\\text{var}} + \\left(1 - \\frac{1}{M} \\right) \\overline{\\text{covar}}\n$$\n\nwhere:\n* \\\\(\\overline{\\text{bias}}\\\\) is the averaged bias of the ensemble members\n*  \\\\(\\overline{\\text{var}}\\\\) is the averaged variance of the ensemble members\n*  \\\\(\\overline{\\text{covar}}\\\\) is the averaged covariance of the ensemble members\n\nBased on this, the ensemble will help to decrease variance, thus decreasing the error. However at the same time if the models we used for ensemble are correlated, it will increase the error through this component \\\\((1 - \\frac{1}{M}) \\overline{\\text{covar}}\\\\). Therefore, we try to diversify the model by:\n\n>* training models on different frameworks: Tensorflow and Pytorch\n* splitting more folds 10-20 folds and each fold train with different seeds\n* building model with different input features\n* different outputs: (a) 18 target and non-target gestures; (b) 8 target gestures + 1 grouped non-target gesture.\n\n**(3) Analyze the model performance to gain insights and directions to improve the results.**\n\n![performance-analysis](https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F878726%2Fb5f7df036e70e5a040079374a6514a3b%2Fchart.svg?generation=1757512135711327&alt=media)\n\nWe started by building a performance analysis pipeline to see how model accuracy relates to different features and to spot opportunities for improvement.  \nSome findings when analyzing IMU only and IMU-THM-TOF models:\n\n>* When THM and TOF data are missing, the IMU only model outperforms the IMU-THM-TOF model --> asign higher weights for IMU_only model when THM & TOF are not available.\n* Both models achieve higher accuracy on adults, males, and right hands compared to children, females, and left hands. However, this gap may be influenced by the co-occurrence of missing rotation/THM/TOF data with these groups. --> upsampling for children, females, and left hands during training.\n* No significant performance differences are observed across height, shoulder-to-wrist length, or elbow-to-wrist length. --> no need to use these features during training.\n\nFor example, here is the performance when TOF is missing:\n* IMU only model: the results of TOF missing or present are not different significantly.\n![IMU-only](https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F878726%2F3b4861850677c6dc2e3b462f72cf009b%2FIMU_only.png?generation=1756920162531734&alt=media)\n\n* IMU-THM-TOF: When TOF is missing, the results drop significantly.\n![ IMU-THM-TOF](https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F878726%2Fe709f7ba35941c0a4b62c55d68af0669%2FIMU_THM_TOF.png?generation=1756920222650364&alt=media)\n\n## Implementation\n\nSome selections of our trained models:\n\n| model ID | features |architecture|splitting|framework |CV|public LB|private LB|\n| --- | --- | --- | --- | --- | --- | --- | --- |\n| M1 | IMU only (maxlen=256, seed=2025)|[cmi imu-only lstm by jiazhuang](https://www.kaggle.com/code/jiazhuang/cmi-imu-only-lstm)|20-fold SS1|Pytorch|0.789|0.799|0.789|\n| M1 | IMU only (maxlen=256, seed=42)|[cmi imu-only lstm from jiazhuang](https://www.kaggle.com/code/jiazhuang/cmi-imu-only-lstm)|20-fold SS1|Pytorch|0.785|0.798|0.791|\n|M1+M2||||||0.801|0.796|\n|M3|IMU+THM+TOF|[IMU × TOF/THM Fusion with Physics‑FE & MixUp by pepushi](https://www.kaggle.com/code/pepushi/imu-tof-thm-fusion-with-physics-fe-mixup-lb0-8)|20-fold SS1|Tensorflow|0.826|0.825|0.813|\n|M4|IMU+THM+TOF + raw TOF|[wasupandceacar's architecture](https://www.kaggle.com/code/wasupandceacar/lb-0-841-5fold-single-model-with-split-sensors) with modified some channels|10-fold SS2|Pytorch||0.833|0.815|\n|M5|IMU+THM+TOF + raw TOF|[wasupandceacar's architecture](https://www.kaggle.com/code/wasupandceacar/lb-0-841-5fold-single-model-with-split-sensors) with modified some channels|10-fold SS2|Tensorflow||0.837|0.815|\n|M4+M5||||||0.837|0.819|\n|M6|IMU+THM+TOF|[IMU × TOF/THM Fusion with Physics‑FE & MixUp by pepushi](https://www.kaggle.com/code/pepushi/imu-tof-thm-fusion-with-physics-fe-mixup-lb0-8) with 9-class output|20-fold SS1|Tensorflow|0.811|0.794|0.800|\n\n---\n\nPublic shared models\n\n| model ID | link |version|CV|public LB|private LB|\n| --- | --- | --- | --- | --- | --- |\n|P1|[LB 0.841-5fold single model with split sensors](https://www.kaggle.com/code/wasupandceacar/lb-0-841-5fold-single-model-with-split-sensors) by [wasupandceacar](https://www.kaggle.com/wasupandceacar) |v4|0.861|0.841|0.824|\n|P2|[Gated GRU + Hybrid Ensemble_v02](https://www.kaggle.com/code/pepushi/gated-gru-hybrid-ensemble-v02) by [pepushi](https://www.kaggle.com/pepushi)|v10||0.835|0.816|\n|P3|[LB 0.82-5fold single bert model](https://www.kaggle.com/code/wasupandceacar/lb-0-82-5fold-single-bert-model) by[wasupandceacar](https://www.kaggle.com/wasupandceacar)|v7||0.829|0.809|\n\n---\n\nEnsemble(our self-trained models in bold)\n\n|ensemble ID|weights|Public LB|private LB|selected|\n| --- | --- | --- | --- | --- | \n| E1 | if THM & TOF are missing {**M1: 0.5**, P1: 0.5} else {**M3: 0.5**, P1: 0.5}|0.849|0.834|no|\n| E2 | if THM & TOF are missing {**M1: 0.4**, P1: 0.4, P2: 0.2} else {**M3: 0.25**, P1: 0.5, P2: 0.25}|0.849|0.837|no|\n| E3 | if THM & TOF are missing {**M1: 0.4**, P1: 0.4, P2: 0.2} else {**M3: 0.5**, P1: 0.5}|0.850|**0.838**|no|\n| E4 | if THM & TOF are missing {**M1+M2: 0.3**, P1: 0.3, P2: 0.2, P3: 0.2} else {**M3: 0.1**, P1: 0.4, P2: 0.25, P3: 0.25}|0.855|0.834|yes|\n| E4 | if THM & TOF are missing {**M1+M2: 0.3**, P1: 0.3, P2: 0.2, P3: 0.2} else {P1: 0.4, P2: 0.35, P3: 0.35}|**0.857**|0.834|yes|\n\n\n"
  },
  "complete": true,
  "source": "existing_discussion",
  "competition_id": 102335,
  "competition_slug": "cmi-detect-behavior-with-sensor-data",
  "team_id": 13914844,
  "solution_url": "/competitions/cmi-detect-behavior-with-sensor-data/writeups/107th-solution"
}