First of all, I would like to thank Kaggle for organizing this fascinating competition, and also acknowledge the contributors of the public code.
\n(1) We mainly developed boosting algorithms, including lightgbm and xgboost, added many features, and integrated them with the publicly available code catboost. The LightGBM model was configured with cat_smooth=20.0, which smooths the probability of samples belonging to each category to prevent the model from being overly confident.The integrated public lb is 0.78, and the private lb is approximately 0.76.
\nhttps://www.kaggle.com/code/jiaoyouzhang/cmi-2025-boosting-0721
\n(2) Base on this code, we adjusted the parameters MAX_SQ_LENGTH=720, n_folds=20, and increased CV and LB by approximately 0.01. We also adjusted the additive attention mechanism and others, but there was no significant improvement in CV.
\n(3) We have tried integration in public code, some of which have adjusted parameters and retrained, but we believe that the main methods to improve performance are integration and boosting feature engineering. We think that some public code is overfitting, and the alternative solution used is simple weighted integration, which has been proven to be true.
", "messages": [ { "id": 3280827, "postDate": "2025-09-03T07:43:01.913Z", "content": "First of all, I would like to thank Kaggle for organizing this fascinating competition, and also acknowledge the contributors of the public code.
\n(1) We mainly developed boosting algorithms, including lightgbm and xgboost, added many features, and integrated them with the publicly available code catboost. The LightGBM model was configured with cat_smooth=20.0, which smooths the probability of samples belonging to each category to prevent the model from being overly confident.The integrated public lb is 0.78, and the private lb is approximately 0.76.
\nhttps://www.kaggle.com/code/jiaoyouzhang/cmi-2025-boosting-0721
\n(2) Base on this code, we adjusted the parameters MAX_SQ_LENGTH=720, n_folds=20, and increased CV and LB by approximately 0.01. We also adjusted the additive attention mechanism and others, but there was no significant improvement in CV.
\n(3) We have tried integration in public code, some of which have adjusted parameters and retrained, but we believe that the main methods to improve performance are integration and boosting feature engineering. We think that some public code is overfitting, and the alternative solution used is simple weighted integration, which has been proven to be true.
", "rawMarkdown": "First of all, I would like to thank Kaggle for organizing this fascinating competition, and also acknowledge the contributors of the public code.\n\n(1) We mainly developed boosting algorithms, including lightgbm and xgboost, added many features, and integrated them with the publicly available code catboost. The LightGBM model was configured with cat_smooth=20.0, which smooths the probability of samples belonging to each category to prevent the model from being overly confident.The integrated public lb is 0.78, and the private lb is approximately 0.76. \n\nhttps://www.kaggle.com/code/jiaoyouzhang/cmi-2025-boosting-0721\n\n(2) Base on **[this code](https://www.kaggle.com/code/jiazhuang/cmi-imu-only-lstm)**, we adjusted the parameters MAX_SQ_LENGTH=720, n_folds=20, and increased CV and LB by approximately 0.01. We also adjusted the additive attention mechanism and others, but there was no significant improvement in CV.\n\n(3) We have tried integration in public code, some of which have adjusted parameters and retrained, but we believe that the main methods to improve performance are integration and boosting feature engineering. We think that some public code is overfitting, and the alternative solution used is simple weighted integration, which has been proven to be true.", "votes": 7 } ], "comments": [], "raw_markdown_by_id": { "3280827": "First of all, I would like to thank Kaggle for organizing this fascinating competition, and also acknowledge the contributors of the public code.\n\n(1) We mainly developed boosting algorithms, including lightgbm and xgboost, added many features, and integrated them with the publicly available code catboost. The LightGBM model was configured with cat_smooth=20.0, which smooths the probability of samples belonging to each category to prevent the model from being overly confident.The integrated public lb is 0.78, and the private lb is approximately 0.76. \n\nhttps://www.kaggle.com/code/jiaoyouzhang/cmi-2025-boosting-0721\n\n(2) Base on **[this code](https://www.kaggle.com/code/jiazhuang/cmi-imu-only-lstm)**, we adjusted the parameters MAX_SQ_LENGTH=720, n_folds=20, and increased CV and LB by approximately 0.01. We also adjusted the additive attention mechanism and others, but there was no significant improvement in CV.\n\n(3) We have tried integration in public code, some of which have adjusted parameters and retrained, but we believe that the main methods to improve performance are integration and boosting feature engineering. We think that some public code is overfitting, and the alternative solution used is simple weighted integration, which has been proven to be true." }, "complete": true, "source": "existing_discussion", "competition_id": 102335, "competition_slug": "cmi-detect-behavior-with-sensor-data", "team_id": 13895639, "solution_url": "/competitions/cmi-detect-behavior-with-sensor-data/writeups/cmi-2025-boosting-0721" }