![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F2832287%2F53827cb5dd407cd86a2c26c316f55e73%2FScreenshot%202025-09-04%20at%2005.58.25.png?generation=1756940329828136&alt=media\")
\nCV: 0.825
The key difference in my solution is the utilization of computer vision to train the sequence model.
\nI found that there are four key factors that significantly impact the final result:
\nI used a computer vision model because of its fast training speed and quick convergence (22 epochs, each taking about 40 seconds). The pipeline using 1DCNN + GRU is only used for the IMU-only model (it requires 120 epochs to converge, with each epoch taking about 110 seconds). The computer vision model I chose is Dino patch 8. Why Dino? Because among the vision transformer models, Dino offers faster convergence and better final results (likely due to good pretraining). Why patch 8? Because the TOF features are organized in an 8x8 matrix. I tried Dinov2 patch 14, but the results were not as good as Dino patch 8 (the score difference was about 1%). In the Dino family (Dino, Dinov2, Dinov3), only two models meet the patch 8 condition: ViT-S/8 and xcit_small_12_p8. Among them, xcit small performs significantly better than ViT small, but the training speed of ViT small is 1.5 times faster.
\n
\nCV: 0.825
| Model | \nCV | \n
|---|---|
| xcit_small without softlabel | \n0.875 | \n
| xcit_small withsoftlabel | \n0.895 | \n
![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F2832287%2F122b235117dbe4d002075b0ca210e23a%2FScreenshot%202025-09-04%20at%2006.34.08.png?generation=1756942505782669&alt=media\") | \n\n |
I use CutMix, MixUp, and RandomErasing. MixUp and CutMix contribute significantly to improving CV.
\nPadding method:
\nI discovered that the gesture performance segment only falls within the last 5 seconds (approximately 50 frames). However, when I cropped it to only the last 64 frames to make the model focus on the gesture segment, the accuracy decreased. This indicates that the transition segment also seems to play a role in providing additional useful information for the model.
\nAdditionally, when padding, I intended to maintain the time interval between frames by inserting zeros at the beginning of the sequence. However, that method seemed ineffective. Ultimately, I performed padding by duplicating the elements to fill in the gaps around that element.
Correct wrong subject:
\nThanks to the authors of these two notebooks, I used features from them.
\nOutput shape: (1, 24, 192)
\nNormalize: not used
Output shape: (192, 17)
\nNormalize: not used
Output shape: (192, 17)
\nNormalize: not used
Output shape: (1, 352, 192)
\nNormalize: IMU not use. THM + TOF use
\nThe special aspect here is that I treat the feature dimension as the height (h) of the model and the sequence length (seq len) as the width (w) of the model. When the model shifts the patch window, it slides along the time dimension. If this is reversed, the model's accuracy decreases significantly.
I use StratifiedGroupKFold to split data (group by subject, stratified by gesture_handedness_height_cm_bin)
\nI use the last epoch as advised by @theoviel https://www.kaggle.com/competitions/cmi-detect-behavior-with-sensor-data/discussion/596787
After training the k-fold models, I used the k models to predict the OOF (Out-Of-Fold) results. Then, I combined the predicted probabilities with the real labels at a ratio of 3:7 to obtain soft labels. Using these soft labels for training improved the CV score by 2.x%.
\nI used the ensemble technique one model at a time. Since I made improvements but didn't have time to retrain the old model, the CV score of the old model is lower than that of the new model.
\nResult: Public=0.874. Private=0.849
The key difference in my solution is the utilization of computer vision to train the sequence model.
\nI found that there are four key factors that significantly impact the final result:
\nI used a computer vision model because of its fast training speed and quick convergence (22 epochs, each taking about 40 seconds). The pipeline using 1DCNN + GRU is only used for the IMU-only model (it requires 120 epochs to converge, with each epoch taking about 110 seconds). The computer vision model I chose is Dino patch 8. Why Dino? Because among the vision transformer models, Dino offers faster convergence and better final results (likely due to good pretraining). Why patch 8? Because the TOF features are organized in an 8x8 matrix. I tried Dinov2 patch 14, but the results were not as good as Dino patch 8 (the score difference was about 1%). In the Dino family (Dino, Dinov2, Dinov3), only two models meet the patch 8 condition: ViT-S/8 and xcit_small_12_p8. Among them, xcit small performs significantly better than ViT small, but the training speed of ViT small is 1.5 times faster.
\n
\nCV: 0.825
| Model | \nCV | \n
|---|---|
| xcit_small without softlabel | \n0.875 | \n
| xcit_small withsoftlabel | \n0.895 | \n
| \n\n |
I use CutMix, MixUp, and RandomErasing. MixUp and CutMix contribute significantly to improving CV.
\nPadding method:
\nI discovered that the gesture performance segment only falls within the last 5 seconds (approximately 50 frames). However, when I cropped it to only the last 64 frames to make the model focus on the gesture segment, the accuracy decreased. This indicates that the transition segment also seems to play a role in providing additional useful information for the model.
\nAdditionally, when padding, I intended to maintain the time interval between frames by inserting zeros at the beginning of the sequence. However, that method seemed ineffective. Ultimately, I performed padding by duplicating the elements to fill in the gaps around that element.
Correct wrong subject:
\nThanks to the authors of these two notebooks, I used features from them.
\nOutput shape: (1, 24, 192)
\nNormalize: not used
Output shape: (192, 17)
\nNormalize: not used
Output shape: (192, 17)
\nNormalize: not used
Output shape: (1, 352, 192)
\nNormalize: IMU not use. THM + TOF use
\nThe special aspect here is that I treat the feature dimension as the height (h) of the model and the sequence length (seq len) as the width (w) of the model. When the model shifts the patch window, it slides along the time dimension. If this is reversed, the model's accuracy decreases significantly.
I use StratifiedGroupKFold to split data (group by subject, stratified by gesture_handedness_height_cm_bin)
\nI use the last epoch as advised by @theoviel https://www.kaggle.com/competitions/cmi-detect-behavior-with-sensor-data/discussion/596787
After training the k-fold models, I used the k models to predict the OOF (Out-Of-Fold) results. Then, I combined the predicted probabilities with the real labels at a ratio of 3:7 to obtain soft labels. Using these soft labels for training improved the CV score by 2.x%.
\nI used the ensemble technique one model at a time. Since I made improvements but didn't have time to retrain the old model, the CV score of the old model is lower than that of the new model.
\nResult: Public=0.874. Private=0.849
Good solution, nice work @quan0095
", "rawMarkdown": "Good solution, nice work @quan0095", "votes": null }, { "id": "3285914", "postDate": "09/08/2025 22:11:49", "content": "I really enjoyed reading your solution! The idea of reframing this problem as a computer vision task is super interesting. I noticed a few other teams also went in this direction, but CNN-based models seemed more common. For example:
\nI'm curious what led you to go with a transformer-based approach from the start? And did you also experiment with CNN-based models for comparison?
", "rawMarkdown": "I really enjoyed reading your solution! The idea of reframing this problem as a computer vision task is super interesting. I noticed a few other teams also went in this direction, but CNN-based models seemed more common. For example:\n\n* [7th Place Solution](https://www.kaggle.com/competitions/cmi-detect-behavior-with-sensor-data/writeups/7th-place-solution) --> EfficientNet & ConvNeXt \n* [3rd Place Solution](https://www.kaggle.com/competitions/cmi-detect-behavior-with-sensor-data/writeups/3rd-place-solution) --> ConvNeXt-v2 & MaxViT\n\nI'm curious what led you to go with a transformer-based approach from the start? And did you also experiment with CNN-based models for comparison?", "votes": null }, { "id": "3285954", "postDate": "09/09/2025 00:50:27", "content": "I have tried using CNNs like ConvNeXt, but the CV score is not as good as with ViT (less than 1% lower). Meanwhile, the CV score of ViT is lower than that of Xcit (less than 1% lower). The key point here is that the data is organized in an 8x8 matrix format, so the receptive field of the CNN or the patch size of the ViT should ideally be 8. I think the top teams had to fine-tune quite meticulously to get CNNs to work effectively. Additionally, when ensembling, training more diverse models generally helps to improve the ensemble score.
", "rawMarkdown": "I have tried using CNNs like ConvNeXt, but the CV score is not as good as with ViT (less than 1% lower). Meanwhile, the CV score of ViT is lower than that of Xcit (less than 1% lower). The key point here is that the data is organized in an 8x8 matrix format, so the receptive field of the CNN or the patch size of the ViT should ideally be 8. I think the top teams had to fine-tune quite meticulously to get CNNs to work effectively. Additionally, when ensembling, training more diverse models generally helps to improve the ensemble score.", "votes": null }, { "id": "3285957", "postDate": "09/09/2025 00:55:37", "content": "You will see the considerable potential of this solution when compared to the top teams' solutions. The top teams mainly rely on two tricks:
\nWhat I and the teams using computer vision models here want to demonstrate is that it is entirely possible to use computer vision models for processing sequence data, with effectiveness comparable to traditional solutions based on 1D CNN, LSTM, or Transformer (for sequences).
", "rawMarkdown": "What I and the teams using computer vision models here want to demonstrate is that it is entirely possible to use computer vision models for processing sequence data, with effectiveness comparable to traditional solutions based on 1D CNN, LSTM, or Transformer (for sequences).", "votes": null }, { "id": "3288891", "postDate": "09/15/2025 03:48:53", "content": "Good work indeed!
", "rawMarkdown": "Good work indeed!", "votes": null } ], "comments": [ { "id": 3281329, "author_name": "tom99763", "author_url": "", "post_date": "09/04/2025 04:23:40", "content": "Good solution, nice work @quan0095
", "votes": null, "replies": [] }, { "id": 3285914, "author_name": "nguyenhoa", "author_url": "", "post_date": "09/08/2025 22:11:49", "content": "I really enjoyed reading your solution! The idea of reframing this problem as a computer vision task is super interesting. I noticed a few other teams also went in this direction, but CNN-based models seemed more common. For example:
\nI'm curious what led you to go with a transformer-based approach from the start? And did you also experiment with CNN-based models for comparison?
", "votes": null, "replies": [ { "id": 3285954, "author_name": "quan0095", "author_url": "", "post_date": "09/09/2025 00:50:27", "content": "I have tried using CNNs like ConvNeXt, but the CV score is not as good as with ViT (less than 1% lower). Meanwhile, the CV score of ViT is lower than that of Xcit (less than 1% lower). The key point here is that the data is organized in an 8x8 matrix format, so the receptive field of the CNN or the patch size of the ViT should ideally be 8. I think the top teams had to fine-tune quite meticulously to get CNNs to work effectively. Additionally, when ensembling, training more diverse models generally helps to improve the ensemble score.
", "votes": null, "replies": [] }, { "id": 3285957, "author_name": "quan0095", "author_url": "", "post_date": "09/09/2025 00:55:37", "content": "You will see the considerable potential of this solution when compared to the top teams' solutions. The top teams mainly rely on two tricks:
\nWhat I and the teams using computer vision models here want to demonstrate is that it is entirely possible to use computer vision models for processing sequence data, with effectiveness comparable to traditional solutions based on 1D CNN, LSTM, or Transformer (for sequences).
", "votes": null, "replies": [] } ] }, { "id": 3288891, "author_name": "dungnv15", "author_url": "", "post_date": "09/15/2025 03:48:53", "content": "Good work indeed!
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "3281258": "The key difference in my solution is the utilization of computer vision to train the sequence model.\n# Importance factor\nI found that there are four key factors that significantly impact the final result:\n- Plus information: Post process using information from other sequence (Not used)\n- Dataset bias: The test dataset only have right subject and the private dataset contains subjects wearing the device incorrectly, in an upside-down position. (Not used)\n- Orientation: Use multi-task learning with orientation information. (Used)\n- Model: Switching to xcit resulted in a significant improvement in CV performance. (Used)\n\n# Model\nI used a computer vision model because of its fast training speed and quick convergence (22 epochs, each taking about 40 seconds). The pipeline using 1DCNN + GRU is only used for the IMU-only model (it requires 120 epochs to converge, with each epoch taking about 110 seconds). The computer vision model I chose is Dino patch 8. Why Dino? Because among the vision transformer models, Dino offers faster convergence and better final results (likely due to good pretraining). Why patch 8? Because the TOF features are organized in an 8x8 matrix. I tried Dinov2 patch 14, but the results were not as good as Dino patch 8 (the score difference was about 1%). In the Dino family (Dino, Dinov2, Dinov3), only two models meet the patch 8 condition: ViT-S/8 and xcit_small_12_p8. Among them, xcit small performs significantly better than ViT small, but the training speed of ViT small is 1.5 times faster. \n## IMU only model\n\nCV: 0.825\n## IMU + THM + TOF model\n| Model | CV |\n| --- | --- |\n| xcit_small without softlabel | 0.875 |\n| xcit_small withsoftlabel | 0.895 |\n\n# Data augmentation\nI use CutMix, MixUp, and RandomErasing. MixUp and CutMix contribute significantly to improving CV.\n# Dataset\nPadding method: \n- Training Augmentation: Randomly samples 80-100% of sequence length, maintaining temporal order\n- Length Constraint: Crops to last 192 frames if sequence exceeds this limit\n- Resampling: Interpolates sequences to exact sample_length (192 steps)\n\nI discovered that the gesture performance segment only falls within the last 5 seconds (approximately 50 frames). However, when I cropped it to only the last 64 frames to make the model focus on the gesture segment, the accuracy decreased. This indicates that the transition segment also seems to play a role in providing additional useful information for the model.\nAdditionally, when padding, I intended to maintain the time interval between frames by inserting zeros at the beginning of the sequence. However, that method seemed ineffective. Ultimately, I performed padding by duplicating the elements to fill in the gaps around that element.\n\nCorrect wrong subject:\n- SUBJ_019262: flip acc_y = - acc_y. rot_y = - rot_y.\n- SUBJ_045235: flip acc_x = -acc_x. acc_y = - acc_y. rot_x = - rot_x. rot_y = - rot_y.\n\n## Feature engineering \nThanks to the authors of these two notebooks, I used features from them.\n- https://www.kaggle.com/code/rktqwe/lb-0-77-linear-accel-tf-bilstm-gru-attention\n- https://www.kaggle.com/code/nksusth/lb-0-78-quaternions-tf-bilstm-gru-attention\nFeatures set:\n[\n 'acc_mag', 'acc_mag_jerk',\n 'rot_angle', 'rot_angle_vel',\n 'linear_acc_mag', 'linear_acc_mag_jerk',\n 'angular_vel_x', 'angular_vel_y', 'angular_vel_z', \n 'angular_vel_mag', 'angular_vel_mag_jerk',\n 'angular_distance',\n 'roll', 'pitch', 'yaw'\n]\nTo avoid having NaN elements between two elements in the sequence, I implemented the safe_diff_fillna_zero function and corrected the feature calculation method from the public notebook.\nI used a Kalman filter to smooth the sequence and eliminate outliers, but the score improvement was not significant.\n## IMU only model\n### Computer vision model\nOutput shape: (1, 24, 192)\nNormalize: not used\n### 1DCNN + GRU\nOutput shape: (192, 17)\nNormalize: not used\n### 1DCNN + Transformer\nOutput shape: (192, 17)\nNormalize: not used\n## IMU + THM + TOF model\nOutput shape: (1, 352, 192)\nNormalize: IMU not use. THM + TOF use\n*The special aspect here is that I treat the feature dimension as the height (h) of the model and the sequence length (seq len) as the width (w) of the model. When the model shifts the patch window, it slides along the time dimension. If this is reversed, the model's accuracy decreases significantly.*\n# Evaluation\nI use StratifiedGroupKFold to split data (group by subject, stratified by gesture_handedness_height_cm_bin)\nI use the last epoch as advised by @theoviel https://www.kaggle.com/competitions/cmi-detect-behavior-with-sensor-data/discussion/596787\n# Softlabel\nAfter training the k-fold models, I used the k models to predict the OOF (Out-Of-Fold) results. Then, I combined the predicted probabilities with the real labels at a ratio of 3:7 to obtain soft labels. Using these soft labels for training improved the CV score by 2.x%.\n# Ensemle\nI used the ensemble technique one model at a time. Since I made improvements but didn't have time to retrain the old model, the CV score of the old model is lower than that of the new model.\nResult: Public=0.874. Private=0.849", "3281329": "Good solution, nice work @quan0095", "3285914": "I really enjoyed reading your solution! The idea of reframing this problem as a computer vision task is super interesting. I noticed a few other teams also went in this direction, but CNN-based models seemed more common. For example:\n\n* [7th Place Solution](https://www.kaggle.com/competitions/cmi-detect-behavior-with-sensor-data/writeups/7th-place-solution) --> EfficientNet & ConvNeXt \n* [3rd Place Solution](https://www.kaggle.com/competitions/cmi-detect-behavior-with-sensor-data/writeups/3rd-place-solution) --> ConvNeXt-v2 & MaxViT\n\nI'm curious what led you to go with a transformer-based approach from the start? And did you also experiment with CNN-based models for comparison?", "3285954": "I have tried using CNNs like ConvNeXt, but the CV score is not as good as with ViT (less than 1% lower). Meanwhile, the CV score of ViT is lower than that of Xcit (less than 1% lower). The key point here is that the data is organized in an 8x8 matrix format, so the receptive field of the CNN or the patch size of the ViT should ideally be 8. I think the top teams had to fine-tune quite meticulously to get CNNs to work effectively. Additionally, when ensembling, training more diverse models generally helps to improve the ensemble score.", "3285957": "You will see the considerable potential of this solution when compared to the top teams' solutions. The top teams mainly rely on two tricks: \n- post-processing (since this is experimental data, the label creation follows a pattern, allowing the design of methods to overfit this pattern, which wouldn't be effective in real-world scenarios). \n- If the dataset distribution is known precisely, it is possible to overfit to that dataset using augmentation", "3285958": "What I and the teams using computer vision models here want to demonstrate is that it is entirely possible to use computer vision models for processing sequence data, with effectiveness comparable to traditional solutions based on 1D CNN, LSTM, or Transformer (for sequences).", "3288891": "Good work indeed!" }, "source": "existing_discussion", "complete": true, "competition_id": 102335, "competition_slug": "cmi-detect-behavior-with-sensor-data", "team_id": 13895838, "solution_url": "/competitions/cmi-detect-behavior-with-sensor-data/writeups/30th-place-solution-vision-transformer" }