Thanks Kaggle and organizer for hosting this interesting competition. A special thanks as well to the valuable public notebooks which helped me a lot:
\nhttps://www.kaggle.com/code/tarundirector/sensor-pulse-viz-eda-for-bfrb-detection
\nhttps://www.kaggle.com/code/jiazhuang/cmi-imu-only-lstm
\nhttps://www.kaggle.com/code/kaigaokaigao/lb-0-75-imu-only-multibranch-inference/notebook
\nI think their work deserves much more votes than rubbish ensemble notebooks.
Data Processing
\nFeature Engineering
\nModel
\nMulti-task Learning
\nAugmentation
\nOther Details
\nEnsemble
\nReproduce
\nI only share my best CV single model training and inference code to avoid publishing too much notebooks of similar content.
Thanks Kaggle and organizer for hosting this interesting competition. A special thanks as well to the valuable public notebooks which helped me a lot:
\nhttps://www.kaggle.com/code/tarundirector/sensor-pulse-viz-eda-for-bfrb-detection
\nhttps://www.kaggle.com/code/jiazhuang/cmi-imu-only-lstm
\nhttps://www.kaggle.com/code/kaigaokaigao/lb-0-75-imu-only-multibranch-inference/notebook
\nI think their work deserves much more votes than rubbish ensemble notebooks.
Data Processing
\nFeature Engineering
\nModel
\nMulti-task Learning
\nAugmentation
\nOther Details
\nEnsemble
\nReproduce
\nI only share my best CV single model training and inference code to avoid publishing too much notebooks of similar content.
Congrats with solo gold - it should be in top 5 in my opinion with such cv and scores.
\nMy advice to organizers - talk to such Kagglers that are not “hacking” metric and creating really good models.
", "rawMarkdown": "Congrats with solo gold - it should be in top 5 in my opinion with such cv and scores. \n\nMy advice to organizers - talk to such Kagglers that are not “hacking” metric and creating really good models.", "votes": null }, { "id": "3280729", "postDate": "09/03/2025 04:51:24", "content": "Congrats with another solo gold. Your model is really strong.
", "rawMarkdown": "Congrats with another solo gold. Your model is really strong.", "votes": null }, { "id": "3280756", "postDate": "09/03/2025 05:34:42", "content": "Thanks and Congratulations for your triple GM! I have learned a lot from your past sharing when I just started to learn machine learning.
", "rawMarkdown": "Thanks and Congratulations for your triple GM! I have learned a lot from your past sharing when I just started to learn machine learning.", "votes": null }, { "id": "3280777", "postDate": "09/03/2025 06:25:35", "content": "Some top modeling and feature engineering approaches deserve more attention and should not be overlooked in favor of only focusing on the top-ranked kagglers in the gold medal zone. I hope the host should be aware of that.
", "rawMarkdown": "Some top modeling and feature engineering approaches deserve more attention and should not be overlooked in favor of only focusing on the top-ranked kagglers in the gold medal zone. I hope the host should be aware of that.", "votes": null }, { "id": "3280810", "postDate": "09/03/2025 07:12:56", "content": "Congratulations on winning the gold medal. The complete solution and training code are perfect for beginners like me to learn. Thank you for sharing
", "rawMarkdown": "Congratulations on winning the gold medal. The complete solution and training code are perfect for beginners like me to learn. Thank you for sharing", "votes": null }, { "id": "3280830", "postDate": "09/03/2025 07:49:46", "content": "\n\nMAE on main loss (not sure if worked)
\n
Oh! This is familiar 🤣
\nFor me it only hurt local validation.
HaHaHa I first see that idea in your LEAP solution writeup.
\nI just added it to search space instead of testing it independently, and the final weight for this part is indeed low, usually <0.01.
Great solution Ruby!
\nIncredible skill in mixing different losses to improve performance.
\nDo you know how much the performance improved compared to using normal loss?
orientation classification gives 0.002~0.004 boost. dice loss impoves 0~0.002. the other two I tend to think they didn't work, CV is not stable enough to test changes with minor impact.
", "rawMarkdown": "orientation classification gives 0.002~0.004 boost. dice loss impoves 0~0.002. the other two I tend to think they didn't work, CV is not stable enough to test changes with minor impact.", "votes": null }, { "id": "3280897", "postDate": "09/03/2025 10:12:27", "content": "Congrats for the solo gold medal! I just want to remind you that your solution didn't showed on the LB.
", "rawMarkdown": "Congrats for the solo gold medal! I just want to remind you that your solution didn't showed on the LB.", "votes": null }, { "id": "3280902", "postDate": "09/03/2025 10:21:39", "content": "Thanks for your reminder. I post it as discussion as before, but seems solution writeup now has its own category. Haven't finger out how to convert it to solution writeup.
", "rawMarkdown": "Thanks for your reminder. I post it as discussion as before, but seems solution writeup now has its own category. Haven't finger out how to convert it to solution writeup.", "votes": null } ], "comments": [ { "id": 3280715, "author_name": "kyakovlev", "author_url": "", "post_date": "09/03/2025 04:09:34", "content": "Congrats with solo gold - it should be in top 5 in my opinion with such cv and scores.
\nMy advice to organizers - talk to such Kagglers that are not “hacking” metric and creating really good models.
", "votes": null, "replies": [ { "id": 3280756, "author_name": "w5833946", "author_url": "", "post_date": "09/03/2025 05:34:42", "content": "Thanks and Congratulations for your triple GM! I have learned a lot from your past sharing when I just started to learn machine learning.
", "votes": null, "replies": [] }, { "id": 3280777, "author_name": "sweetyheehee", "author_url": "", "post_date": "09/03/2025 06:25:35", "content": "Some top modeling and feature engineering approaches deserve more attention and should not be overlooked in favor of only focusing on the top-ranked kagglers in the gold medal zone. I hope the host should be aware of that.
", "votes": null, "replies": [] } ] }, { "id": 3280729, "author_name": "chenxin1991", "author_url": "", "post_date": "09/03/2025 04:51:24", "content": "Congrats with another solo gold. Your model is really strong.
", "votes": null, "replies": [] }, { "id": 3280810, "author_name": "aichangeworld", "author_url": "", "post_date": "09/03/2025 07:12:56", "content": "Congratulations on winning the gold medal. The complete solution and training code are perfect for beginners like me to learn. Thank you for sharing
", "votes": null, "replies": [] }, { "id": 3280830, "author_name": "shlomoron", "author_url": "", "post_date": "09/03/2025 07:49:46", "content": "\n\nMAE on main loss (not sure if worked)
\n
Oh! This is familiar 🤣
\nFor me it only hurt local validation.
HaHaHa I first see that idea in your LEAP solution writeup.
\nI just added it to search space instead of testing it independently, and the final weight for this part is indeed low, usually <0.01.
Great solution Ruby!
\nIncredible skill in mixing different losses to improve performance.
\nDo you know how much the performance improved compared to using normal loss?
orientation classification gives 0.002~0.004 boost. dice loss impoves 0~0.002. the other two I tend to think they didn't work, CV is not stable enough to test changes with minor impact.
", "votes": null, "replies": [] } ] }, { "id": 3280897, "author_name": "kurisew", "author_url": "", "post_date": "09/03/2025 10:12:27", "content": "Congrats for the solo gold medal! I just want to remind you that your solution didn't showed on the LB.
", "votes": null, "replies": [ { "id": 3280902, "author_name": "w5833946", "author_url": "", "post_date": "09/03/2025 10:21:39", "content": "Thanks for your reminder. I post it as discussion as before, but seems solution writeup now has its own category. Haven't finger out how to convert it to solution writeup.
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "3280712": "Thanks Kaggle and organizer for hosting this interesting competition. A special thanks as well to the valuable public notebooks which helped me a lot:\nhttps://www.kaggle.com/code/tarundirector/sensor-pulse-viz-eda-for-bfrb-detection\nhttps://www.kaggle.com/code/jiazhuang/cmi-imu-only-lstm\nhttps://www.kaggle.com/code/kaigaokaigao/lb-0-75-imu-only-multibranch-inference/notebook\nI think their work deserves much more votes than rubbish ensemble notebooks.\n\n**Data Processing**\n1. IMU\n - rot in this dataset is normalized to rot_w>0, recover sign by forcing dot_product(rot,rot[-1]) >=0 with sign flip\n - handedness flip: -acc_x, -rot_y, -rot_z\n - no normalization applied, using BN as first layer for convenience\n2. THM\n - set all sequence sensor values to nan if : mean>40 or mean<20 or std==0\n - handedness flip: switch sensor 3 and 5\n - normalization: (t-27)/3\n3. TOF\n - value <0 or >249 (if any) to nan\n - handedness flip: switch sensor 3 and 5, up-down flip for sensor 3 and 5, left-right flip for sensor 1,2,4\n - normalization: (250-x)/65\n - nan to 0 (250 before normalization), no interpolation\n\n**Feature Engineering**\n1. IMU\n - acc/ rot/ angular velocity (in device and world frame)/ acc with gravity remove (in both frame)/ angular velocity abs_diff/ gravity projection in device frame/ handedness (actually meta feature, too lazy to make it an independent branch)\n - most features come in group of size 3 and rot with size 4, I add zero columns to make feature matrix in format of bins with size 4. This is for later group convolution \n2. TOF\n - 0-1 nan indicator as extra feature\n\n**Model**\n1. IMU branch\n - two standard 1DResBlock with SCSE attention \n - using group conv such that interaction only happens within group (eg: within acc_x/y/z)\n2. THM branch\n - two standard 1DResBlock with SCSE attention\n3. TOF branch\n - three standard 3DResBlock with SCSE attention, kernel size (1,3,3)*2+(3,1,1)\n - group conv such that interaction only happens within same TOF sensor\n4. Fusion\n - two GRU + MHA (with ROPE) + AttentionPooling + MLP\n - I use other combination among GRU/ResBlock/MHA for ensemble\n - GRU: I modularize it with skip-connect and layernorm\n - MHA: I found extra feedforward layer in common transformer block useless in this dataset and I only keep the MHA part\n\n\n**Multi-task Learning**\n1. main loss: cross entropy + dice loss with cross entropy ratio decayed while training\n2. cross entropy on orientation classification\n3. MSE on ratio of gesture region (I didn’t properly normalize this part, not sure if worked)\n4. MAE on main loss (not sure if worked)\n\n**Augmentation**\n1.\trot sign flip\n2.\tworld frame x-y plane rotation (change facing direction)\n3.\ttime stretching\n4.\tsequence patch mask\n5.\tTOF pixel jitter\n6.\tTOF patch mask\n7.\tsensor mask\n\n**Other Details**\n1.\tonecycle lr, AdamW, EMA, mixup, dropout, silu, truncate sequence with tail region pre-padding by zero to length 120\n2.\tI run simple random search over params space and manually clip search space after analysis\n\n**Ensemble**\n1.\tclass rank average among Kfolds, voting with different models\n2.\tbased on local CV tuning ensemble weights or selecting from a large pool easily cause overfitting, so I restricted to models with good CV and only apply voting (better than average might due to some over confident models)\n3.\tgains\nsingle best (based on CV) IMU/All:0.8364/0.9008, public/private: 0.868/0.859\nensemble IMU/All:0.8458/0.906 public/private: 0.875/0.862\n\n**Reproduce**\nI only share my best CV single model training and inference code to avoid publishing too much notebooks of similar content.\n1.\tIMU only training: https://www.kaggle.com/code/w5833946/m21-cnn-gru-train\n2.\tAll feature training: https://www.kaggle.com/code/w5833946/all-m14-cnn-gru-train (trained offline with same code, download from google drive)\n3.\tInference: https://www.kaggle.com/code/w5833946/cmi3-submit-single", "3280715": "Congrats with solo gold - it should be in top 5 in my opinion with such cv and scores. \n\nMy advice to organizers - talk to such Kagglers that are not “hacking” metric and creating really good models.", "3280729": "Congrats with another solo gold. Your model is really strong.", "3280756": "Thanks and Congratulations for your triple GM! I have learned a lot from your past sharing when I just started to learn machine learning.", "3280777": "Some top modeling and feature engineering approaches deserve more attention and should not be overlooked in favor of only focusing on the top-ranked kagglers in the gold medal zone. I hope the host should be aware of that.", "3280810": "Congratulations on winning the gold medal. The complete solution and training code are perfect for beginners like me to learn. Thank you for sharing", "3280830": ">MAE on main loss (not sure if worked) \n\nOh! This is familiar 🤣\nFor me it only hurt local validation.", "3280834": "HaHaHa I first see that idea in your LEAP solution writeup.\nI just added it to search space instead of testing it independently, and the final weight for this part is indeed low, usually <0.01.", "3280890": "Great solution Ruby!\nIncredible skill in mixing different losses to improve performance.\nDo you know how much the performance improved compared to using normal loss?", "3280892": "orientation classification gives 0.002~0.004 boost. dice loss impoves 0~0.002. the other two I tend to think they didn't work, CV is not stable enough to test changes with minor impact.", "3280897": "Congrats for the solo gold medal! I just want to remind you that your solution didn't showed on the LB.", "3280902": "Thanks for your reminder. I post it as discussion as before, but seems solution writeup now has its own category. Haven't finger out how to convert it to solution writeup." }, "source": "meta" }