Hi, Kagglers!
\nIt’s a wrap! The CMI Detect Behavior with Sensor Data competition has officially come to an end, and we were excited to see such incredible engagement over the past 3+ months. On behalf of our entire organizing team, we want to say thank you to everyone who joined this challenge and made this competition a huge success. Your contributions in developing predictive models to distinguish BFRB-like from non-BFRB-like activity are helping advance the design of wearable devices for BFRB detection.
\nFirst, big congratulations to the six winners of this competition: the Devin | Ogurtsov | zyz Team (@devinanzelmo, @ogurtsov, @yuanzhezhou; 1st place), @daiwakun (2nd place), the ln | yu4u | Theo | minerppdy Team (@nazarov, @ren4yu, @theoviel, @minerppdy; 3rd place), @dott1718 (4th place), the have fun Team (@chenxin1991, @dc5e964768ef56302a32; 5th place), and @rsakata (6th place). We really enjoyed learning the details of your different approaches and solutions!
\nWe would also like to give a special shout-out to the top-ranked first-time Kagglers in this competition: @Himeno, who participated solo (90th place) and the Qingyang Jiao Team (@oliviachuhanzhang, @ztqq08, @qingyangjiao, and @pantianlan; 79th place). It’s amazing to see participants join their very first Kaggle competition motivated by our challenges and data!
\nWe also wanted to shout out some of our favorite discussions and things we learned from all of you throughout this competition! While our 6 winners achieved the top scores, it’s important to recognize the value that the incredible creativity and collaboration across the community brought to the competition. Many of you went beyond the quantitative challenge, by contributing thoughtful discussion posts, supporting fellow participants, sharing resources and ideas, and approaching the data from fresh and innovative perspectives. A few highlights include:
\nNow, let’s talk about the data! The competition metric we used averages two F1-scores: one measuring whether the model correctly distinguishes BFRB-like behaviors from non-BFRB-like activities, and another measuring how well it identifies the 8 different types of BFRB-like behaviors, giving equal weight to both aspects of performance. We are thrilled to report that the best submissions from the 6 winners achieved an average score of 0.878 on the private test set, demonstrating that these models can effectively detect and classify BFRB behaviors on data they have never encountered before.
\nOne of the most striking findings from the competition was the impact of sensor diversity: models performed better on sequences that included data from multiple sensors compared to IMU-only, with a substantial 0.104 improvement in the competition score. This strongly suggests that BFRB-detection devices would benefit from the inclusion of time-of-flight and thermopile sensors alongside traditional motion sensors!
\nThis sensor diversity advantage becomes even more interesting when we look at how models generalized to new data. On sequences with only IMU data, models showed classic overfitting behavior, with average F1-scores for identifying BFRBs dropping from 0.712 on public data to 0.641 on private data (-0.071). However, on sequences containing all sensor types (IMU plus thermopile and time-of-flight), models actually improved their scores from 0.801 to 0.858 on unseen data (+0.057!). This suggests that when models encounter richer multi-modal input—capturing movement, thermal signatures, and spatial relationships—they learn more generalizable patterns. In contrast, with IMU-only sequences, they appear to rely on less robust features that don't transfer well to new data. Interestingly, we saw this pattern for all of our top-6 models, suggesting that this finding is not just a spurious trend. You can explore these results and more in our interactive dashboard.
\nWe are extremely excited by these results, and want to thank you all for making this competition not just about the leaderboard, but about testing secondary hypotheses, sharing knowledge, supporting others, and exploring the data in inventive ways.
\nOne thing that is important to remember is that this is where the work truly begins. This competition helped us test a hypothesis about how to design devices for BFRB detection. Does that mean we can now take this device and use these models to predict repetitive behaviors in the real-world? No, of course not. There is an abundance of work that still needs to be done to test generalizability, including designing datasets that lend themselves to real-world inferences that have not yet taken place. This competition represents an important, but early, step towards that goal.
\nWe look forward to continuing exploring and learning about our data with you, as we collectively strive to advance our understanding of brain health!
\n-Greg, on behalf of the rest of the Child Mind Institute team
", "messages": [ { "id": 3302085, "postDate": "2025-10-15T01:23:30.433Z", "content": "Hi, Kagglers!
\nIt’s a wrap! The CMI Detect Behavior with Sensor Data competition has officially come to an end, and we were excited to see such incredible engagement over the past 3+ months. On behalf of our entire organizing team, we want to say thank you to everyone who joined this challenge and made this competition a huge success. Your contributions in developing predictive models to distinguish BFRB-like from non-BFRB-like activity are helping advance the design of wearable devices for BFRB detection.
\nFirst, big congratulations to the six winners of this competition: the Devin | Ogurtsov | zyz Team (@devinanzelmo, @ogurtsov, @yuanzhezhou; 1st place), @daiwakun (2nd place), the ln | yu4u | Theo | minerppdy Team (@nazarov, @ren4yu, @theoviel, @minerppdy; 3rd place), @dott1718 (4th place), the have fun Team (@chenxin1991, @dc5e964768ef56302a32; 5th place), and @rsakata (6th place). We really enjoyed learning the details of your different approaches and solutions!
\nWe would also like to give a special shout-out to the top-ranked first-time Kagglers in this competition: @Himeno, who participated solo (90th place) and the Qingyang Jiao Team (@oliviachuhanzhang, @ztqq08, @qingyangjiao, and @pantianlan; 79th place). It’s amazing to see participants join their very first Kaggle competition motivated by our challenges and data!
\nWe also wanted to shout out some of our favorite discussions and things we learned from all of you throughout this competition! While our 6 winners achieved the top scores, it’s important to recognize the value that the incredible creativity and collaboration across the community brought to the competition. Many of you went beyond the quantitative challenge, by contributing thoughtful discussion posts, supporting fellow participants, sharing resources and ideas, and approaching the data from fresh and innovative perspectives. A few highlights include:
\nNow, let’s talk about the data! The competition metric we used averages two F1-scores: one measuring whether the model correctly distinguishes BFRB-like behaviors from non-BFRB-like activities, and another measuring how well it identifies the 8 different types of BFRB-like behaviors, giving equal weight to both aspects of performance. We are thrilled to report that the best submissions from the 6 winners achieved an average score of 0.878 on the private test set, demonstrating that these models can effectively detect and classify BFRB behaviors on data they have never encountered before.
\nOne of the most striking findings from the competition was the impact of sensor diversity: models performed better on sequences that included data from multiple sensors compared to IMU-only, with a substantial 0.104 improvement in the competition score. This strongly suggests that BFRB-detection devices would benefit from the inclusion of time-of-flight and thermopile sensors alongside traditional motion sensors!
\nThis sensor diversity advantage becomes even more interesting when we look at how models generalized to new data. On sequences with only IMU data, models showed classic overfitting behavior, with average F1-scores for identifying BFRBs dropping from 0.712 on public data to 0.641 on private data (-0.071). However, on sequences containing all sensor types (IMU plus thermopile and time-of-flight), models actually improved their scores from 0.801 to 0.858 on unseen data (+0.057!). This suggests that when models encounter richer multi-modal input—capturing movement, thermal signatures, and spatial relationships—they learn more generalizable patterns. In contrast, with IMU-only sequences, they appear to rely on less robust features that don't transfer well to new data. Interestingly, we saw this pattern for all of our top-6 models, suggesting that this finding is not just a spurious trend. You can explore these results and more in our interactive dashboard.
\nWe are extremely excited by these results, and want to thank you all for making this competition not just about the leaderboard, but about testing secondary hypotheses, sharing knowledge, supporting others, and exploring the data in inventive ways.
\nOne thing that is important to remember is that this is where the work truly begins. This competition helped us test a hypothesis about how to design devices for BFRB detection. Does that mean we can now take this device and use these models to predict repetitive behaviors in the real-world? No, of course not. There is an abundance of work that still needs to be done to test generalizability, including designing datasets that lend themselves to real-world inferences that have not yet taken place. This competition represents an important, but early, step towards that goal.
\nWe look forward to continuing exploring and learning about our data with you, as we collectively strive to advance our understanding of brain health!
\n-Greg, on behalf of the rest of the Child Mind Institute team
", "rawMarkdown": "Hi, Kagglers!\n\nIt’s a wrap! The CMI ***Detect Behavior with Sensor*** Data competition has officially come to an end, and we were excited to see such incredible engagement over the past 3+ months. On behalf of our entire organizing team, we want to say thank you to everyone who joined this challenge and made this competition a huge success. Your contributions in developing predictive models to distinguish BFRB-like from non-BFRB-like activity are helping advance the design of wearable devices for BFRB detection. \n\nFirst, big congratulations to the **six winners** of this competition: the *Devin | Ogurtsov | zyz* Team (@devinanzelmo, @ogurtsov, @yuanzhezhou; 1st place), @daiwakun (2nd place), the *ln | yu4u | Theo | minerppdy* Team (@nazarov, @ren4yu, @theoviel, @minerppdy; 3rd place), @dott1718 (4th place), the *have fun* Team (@chenxin1991, @dc5e964768ef56302a32; 5th place), and @rsakata (6th place). We really enjoyed learning the details of your different approaches and solutions!\n\nWe would also like to give a special shout-out to the **top-ranked first-time Kagglers** in this competition: @Himeno, who participated solo (90th place) and the Qingyang Jiao Team (@oliviachuhanzhang, @ztqq08, @qingyangjiao, and @pantianlan; 79th place). It’s amazing to see participants join their very first Kaggle competition motivated by our challenges and data!\n\nWe also wanted to shout out some of our **favorite discussions** and things we learned from all of you throughout this competition! While our 6 winners achieved the top scores, it’s important to recognize the value that the incredible creativity and collaboration across the community brought to the competition. Many of you went beyond the quantitative challenge, by contributing thoughtful discussion posts, supporting fellow participants, sharing resources and ideas, and approaching the data from fresh and innovative perspectives. A few highlights include: \n* @phalanx’s engaging posts on agents, including [this one](https://www.kaggle.com/competitions/cmi-detect-behavior-with-sensor-data/discussion/583863), which became the most upvoted discussion of the competition and sparked great conversation. It’s very interesting to think about what the future of data science competitions may be with the recent advancements in agents 🤯\n* The team “ShoRI : Device is all you need” (@sakarush, @haruhisakimoto, @tanakashintaro, @nakaso, @kentatsukaue), who ingeniously built their own device to better study the behavior of the original hardware. Very cool hands-on approach that we never would have anticipated!\n* Extremely helpful contributions such as the [tips for robust validation](https://www.kaggle.com/competitions/cmi-detect-behavior-with-sensor-data/discussion/596787) by @theoveil, which laid a crystal clear roadmap for others to build generalizable models and get the most out of the competition. \n* The visualizations shared by @tatamikenn, from hand-drawn sketches to detailed plots (e.g., [here ](https://www.kaggle.com/competitions/cmi-detect-behavior-with-sensor-data/discussion/603566) and [here](https://www.kaggle.com/competitions/cmi-detect-behavior-with-sensor-data/discussion/603566)), illustrating how essential visualization is to data science. \n\nNow, let’s talk about **the data**! The competition metric we used averages two F1-scores: one measuring whether the model correctly distinguishes BFRB-like behaviors from non-BFRB-like activities, and another measuring how well it identifies the 8 different types of BFRB-like behaviors, giving equal weight to both aspects of performance. We are thrilled to report that the best submissions from the 6 winners achieved an average score of 0.878 on the private test set, demonstrating that these models can effectively detect and classify BFRB behaviors on data they have never encountered before. \n\nOne of the most striking findings from the competition was the impact of sensor diversity: models performed better on sequences that included data from multiple sensors compared to IMU-only, with **a substantial 0.104 improvement** in the competition score. This strongly suggests that BFRB-detection devices would benefit from the inclusion of time-of-flight and thermopile sensors alongside traditional motion sensors!\n\nThis sensor diversity advantage becomes even more interesting when we look at how models generalized to new data. On sequences with only IMU data, models showed classic overfitting behavior, with average F1-scores for identifying BFRBs dropping from 0.712 on public data to 0.641 on private data (**-0.071**). However, on sequences containing all sensor types (IMU plus thermopile and time-of-flight), models actually improved their scores from 0.801 to 0.858 on unseen data (**+0.057!**). This suggests that when models encounter richer multi-modal input—capturing movement, thermal signatures, and spatial relationships—they learn more generalizable patterns. In contrast, with IMU-only sequences, they appear to rely on less robust features that don't transfer well to new data. Interestingly, we saw this pattern for all of our top-6 models, suggesting that this finding is not just a spurious trend. You can explore these results and more in our [interactive dashboard](https://childmindresearch.github.io/helios_eval/apps/helios_dashboard.html).\n\nWe are extremely excited by these results, and want to thank you all for making this competition not just about the leaderboard, but about testing secondary hypotheses, sharing knowledge, supporting others, and exploring the data in inventive ways.\n\nOne thing that is important to remember is that this is where the work truly begins. This competition helped us test a hypothesis about how to design devices for BFRB detection. Does that mean we can now take this device and use these models to predict repetitive behaviors in the real-world? No, of course not. There is an abundance of work that still needs to be done to test generalizability, including designing datasets that lend themselves to real-world inferences that have not yet taken place. This competition represents an important, but early, step towards that goal.\n\nWe look forward to continuing exploring and learning about our data with you, as we collectively strive to advance our understanding of brain health! \n\n-Greg, on behalf of the rest of the Child Mind Institute team", "votes": 5 } ], "comments": [], "raw_markdown_by_id": { "3302085": "Hi, Kagglers!\n\nIt’s a wrap! The CMI ***Detect Behavior with Sensor*** Data competition has officially come to an end, and we were excited to see such incredible engagement over the past 3+ months. On behalf of our entire organizing team, we want to say thank you to everyone who joined this challenge and made this competition a huge success. Your contributions in developing predictive models to distinguish BFRB-like from non-BFRB-like activity are helping advance the design of wearable devices for BFRB detection. \n\nFirst, big congratulations to the **six winners** of this competition: the *Devin | Ogurtsov | zyz* Team (@devinanzelmo, @ogurtsov, @yuanzhezhou; 1st place), @daiwakun (2nd place), the *ln | yu4u | Theo | minerppdy* Team (@nazarov, @ren4yu, @theoviel, @minerppdy; 3rd place), @dott1718 (4th place), the *have fun* Team (@chenxin1991, @dc5e964768ef56302a32; 5th place), and @rsakata (6th place). We really enjoyed learning the details of your different approaches and solutions!\n\nWe would also like to give a special shout-out to the **top-ranked first-time Kagglers** in this competition: @Himeno, who participated solo (90th place) and the Qingyang Jiao Team (@oliviachuhanzhang, @ztqq08, @qingyangjiao, and @pantianlan; 79th place). It’s amazing to see participants join their very first Kaggle competition motivated by our challenges and data!\n\nWe also wanted to shout out some of our **favorite discussions** and things we learned from all of you throughout this competition! While our 6 winners achieved the top scores, it’s important to recognize the value that the incredible creativity and collaboration across the community brought to the competition. Many of you went beyond the quantitative challenge, by contributing thoughtful discussion posts, supporting fellow participants, sharing resources and ideas, and approaching the data from fresh and innovative perspectives. A few highlights include: \n* @phalanx’s engaging posts on agents, including [this one](https://www.kaggle.com/competitions/cmi-detect-behavior-with-sensor-data/discussion/583863), which became the most upvoted discussion of the competition and sparked great conversation. It’s very interesting to think about what the future of data science competitions may be with the recent advancements in agents 🤯\n* The team “ShoRI : Device is all you need” (@sakarush, @haruhisakimoto, @tanakashintaro, @nakaso, @kentatsukaue), who ingeniously built their own device to better study the behavior of the original hardware. Very cool hands-on approach that we never would have anticipated!\n* Extremely helpful contributions such as the [tips for robust validation](https://www.kaggle.com/competitions/cmi-detect-behavior-with-sensor-data/discussion/596787) by @theoveil, which laid a crystal clear roadmap for others to build generalizable models and get the most out of the competition. \n* The visualizations shared by @tatamikenn, from hand-drawn sketches to detailed plots (e.g., [here ](https://www.kaggle.com/competitions/cmi-detect-behavior-with-sensor-data/discussion/603566) and [here](https://www.kaggle.com/competitions/cmi-detect-behavior-with-sensor-data/discussion/603566)), illustrating how essential visualization is to data science. \n\nNow, let’s talk about **the data**! The competition metric we used averages two F1-scores: one measuring whether the model correctly distinguishes BFRB-like behaviors from non-BFRB-like activities, and another measuring how well it identifies the 8 different types of BFRB-like behaviors, giving equal weight to both aspects of performance. We are thrilled to report that the best submissions from the 6 winners achieved an average score of 0.878 on the private test set, demonstrating that these models can effectively detect and classify BFRB behaviors on data they have never encountered before. \n\nOne of the most striking findings from the competition was the impact of sensor diversity: models performed better on sequences that included data from multiple sensors compared to IMU-only, with **a substantial 0.104 improvement** in the competition score. This strongly suggests that BFRB-detection devices would benefit from the inclusion of time-of-flight and thermopile sensors alongside traditional motion sensors!\n\nThis sensor diversity advantage becomes even more interesting when we look at how models generalized to new data. On sequences with only IMU data, models showed classic overfitting behavior, with average F1-scores for identifying BFRBs dropping from 0.712 on public data to 0.641 on private data (**-0.071**). However, on sequences containing all sensor types (IMU plus thermopile and time-of-flight), models actually improved their scores from 0.801 to 0.858 on unseen data (**+0.057!**). This suggests that when models encounter richer multi-modal input—capturing movement, thermal signatures, and spatial relationships—they learn more generalizable patterns. In contrast, with IMU-only sequences, they appear to rely on less robust features that don't transfer well to new data. Interestingly, we saw this pattern for all of our top-6 models, suggesting that this finding is not just a spurious trend. You can explore these results and more in our [interactive dashboard](https://childmindresearch.github.io/helios_eval/apps/helios_dashboard.html).\n\nWe are extremely excited by these results, and want to thank you all for making this competition not just about the leaderboard, but about testing secondary hypotheses, sharing knowledge, supporting others, and exploring the data in inventive ways.\n\nOne thing that is important to remember is that this is where the work truly begins. This competition helped us test a hypothesis about how to design devices for BFRB detection. Does that mean we can now take this device and use these models to predict repetitive behaviors in the real-world? No, of course not. There is an abundance of work that still needs to be done to test generalizability, including designing datasets that lend themselves to real-world inferences that have not yet taken place. This competition represents an important, but early, step towards that goal.\n\nWe look forward to continuing exploring and learning about our data with you, as we collectively strive to advance our understanding of brain health! \n\n-Greg, on behalf of the rest of the Child Mind Institute team" } }