{
  "id": 597001,
  "title": "Anyone Pushing GBDT (LightGBM / XGBoost / CatBoost) Past 0.80?",
  "url": "/competitions/cmi-detect-behavior-with-sensor-data/discussion/597001",
  "author_name": "",
  "post_date": "2025-08-06T03:02:05.245186800Z",
  "votes": 2,
  "comment_count": 11,
  "views": 0,
  "content": "<p>Hi everyone,</p>\n<p>I’ve been experimenting with pure GBDT approaches (LightGBM, XGBoost, CatBoost) for the CMI Detect Behavior with Sensor Data competition, but my best public LB is stuck around 0.71.</p>\n<p>I’m curious:</p>\n<p>Has anyone managed to hit 0.80+ using only GBDT models?<br>\nIf so, what’s the highest score you’ve seen with this family of models?</p>\n<p>I’m not asking for code and approach—just want to know whether breaking the 0.80 barrier is realistically achievable with tree-based methods alone, or if deep models are simply necessary here.</p>\n<p>Thanks a lot for sharing your experience!</p>",
  "messages": [
    {
      "id": "3264009",
      "postDate": "08/06/2025 03:02:05",
      "content": "<p>Hi everyone,</p>\n<p>I’ve been experimenting with pure GBDT approaches (LightGBM, XGBoost, CatBoost) for the CMI Detect Behavior with Sensor Data competition, but my best public LB is stuck around 0.71.</p>\n<p>I’m curious:</p>\n<p>Has anyone managed to hit 0.80+ using only GBDT models?<br>\nIf so, what’s the highest score you’ve seen with this family of models?</p>\n<p>I’m not asking for code and approach—just want to know whether breaking the 0.80 barrier is realistically achievable with tree-based methods alone, or if deep models are simply necessary here.</p>\n<p>Thanks a lot for sharing your experience!</p>",
      "rawMarkdown": "Hi everyone,\n\nI’ve been experimenting with pure GBDT approaches (LightGBM, XGBoost, CatBoost) for the CMI Detect Behavior with Sensor Data competition, but my best public LB is stuck around 0.71.\n\nI’m curious:\n\nHas anyone managed to hit 0.80+ using only GBDT models?\nIf so, what’s the highest score you’ve seen with this family of models?\n\nI’m not asking for code and approach—just want to know whether breaking the 0.80 barrier is realistically achievable with tree-based methods alone, or if deep models are simply necessary here.\n\nThanks a lot for sharing your experience!",
      "votes": null
    },
    {
      "id": "3264041",
      "postDate": "08/06/2025 04:32:31",
      "content": "<p>LB 0.75 , GBDT has a relatively small effect.</p>",
      "rawMarkdown": "LB 0.75 , GBDT has a relatively small effect.",
      "votes": null
    },
    {
      "id": "3264138",
      "postDate": "08/06/2025 08:08:33",
      "content": "<p>LB 0.80, GDBT for IMU and all data. (Switching model) 0.76 for IMU only 0.83 CV for all data.</p>",
      "rawMarkdown": "LB 0.80, GDBT for IMU and all data. (Switching model) 0.76 for IMU only 0.83 CV for all data.",
      "votes": null
    },
    {
      "id": "3265863",
      "postDate": "08/08/2025 03:41:47",
      "content": "<p>I worked with LGBM only for the first few weeks, only pushed to 0.804. </p>\n<p>Was not able to make any more progress for another week. </p>\n<p>My first naive CNN submit was able to beat 0.804, so i decided to divert efforts there.</p>",
      "rawMarkdown": "I worked with LGBM only for the first few weeks, only pushed to 0.804. \n\nWas not able to make any more progress for another week. \n\nMy first naive CNN submit was able to beat 0.804, so i decided to divert efforts there.",
      "votes": null
    },
    {
      "id": "3265869",
      "postDate": "08/08/2025 03:50:43",
      "content": "<p>A great job!</p>",
      "rawMarkdown": "A great job!",
      "votes": null
    },
    {
      "id": "3265910",
      "postDate": "08/08/2025 05:56:31",
      "content": "<p>only imu or all data, i wish your reply.</p>",
      "rawMarkdown": "only imu or all data, i wish your reply.",
      "votes": null
    },
    {
      "id": "3265929",
      "postDate": "08/08/2025 06:51:29",
      "content": "<p><a href=\"https://www.kaggle.com/aichangeworld\" target=\"_blank\">@aichangeworld</a> i worked on both</p>",
      "rawMarkdown": "aichangeworld i worked on both",
      "votes": null
    },
    {
      "id": "3266018",
      "postDate": "08/08/2025 08:45:10",
      "content": "<p>Did you use the aggregated features into your NN models?</p>",
      "rawMarkdown": "Did you use the aggregated features into your NN models?",
      "votes": null
    },
    {
      "id": "3266029",
      "postDate": "08/08/2025 09:13:44",
      "content": "<p>I did not.</p>\n<p>I tried adding \"simpler\" aggregated features, like rolling stats, segment stats, etc. Did not help. </p>\n<p>So i didn't bother to try the more \"complex\" features.</p>",
      "rawMarkdown": "I did not.\n\nI tried adding \"simpler\" aggregated features, like rolling stats, segment stats, etc. Did not help. \n\nSo i didn't bother to try the more \"complex\" features.",
      "votes": null
    },
    {
      "id": "3266034",
      "postDate": "08/08/2025 09:25:09",
      "content": "<p>Hmm okay. I'll take that into account when moving to NN.</p>",
      "rawMarkdown": "Hmm okay. I'll take that into account when moving to NN.",
      "votes": null
    },
    {
      "id": "3266036",
      "postDate": "08/08/2025 09:26:23",
      "content": "<p>Because looking at the public nb single models. I'm almost able to match the best single model by using gbm's.</p>",
      "rawMarkdown": "Because looking at the public nb single models. I'm almost able to match the best single model by using gbm's.",
      "votes": null
    },
    {
      "id": "3266505",
      "postDate": "08/09/2025 08:34:21",
      "content": "<p><a href=\"https://www.kaggle.com/code/jiaoyouzhang/cmi-2025-only-lightgbm-training\" target=\"_blank\">https://www.kaggle.com/code/jiaoyouzhang/cmi-2025-only-lightgbm-training</a></p>",
      "rawMarkdown": "https://www.kaggle.com/code/jiaoyouzhang/cmi-2025-only-lightgbm-training",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 3264041,
      "author_name": "jiaoyouzhang",
      "author_url": "",
      "post_date": "08/06/2025 04:32:31",
      "content": "<p>LB 0.75 , GBDT has a relatively small effect.</p>",
      "votes": null,
      "replies": [
        {
          "id": 3266505,
          "author_name": "jiaoyouzhang",
          "author_url": "",
          "post_date": "08/09/2025 08:34:21",
          "content": "<p><a href=\"https://www.kaggle.com/code/jiaoyouzhang/cmi-2025-only-lightgbm-training\" target=\"_blank\">https://www.kaggle.com/code/jiaoyouzhang/cmi-2025-only-lightgbm-training</a></p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 3264138,
      "author_name": "stefanoclss",
      "author_url": "",
      "post_date": "08/06/2025 08:08:33",
      "content": "<p>LB 0.80, GDBT for IMU and all data. (Switching model) 0.76 for IMU only 0.83 CV for all data.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 3265863,
      "author_name": "gerwynng",
      "author_url": "",
      "post_date": "08/08/2025 03:41:47",
      "content": "<p>I worked with LGBM only for the first few weeks, only pushed to 0.804. </p>\n<p>Was not able to make any more progress for another week. </p>\n<p>My first naive CNN submit was able to beat 0.804, so i decided to divert efforts there.</p>",
      "votes": null,
      "replies": [
        {
          "id": 3265869,
          "author_name": "jiaoyouzhang",
          "author_url": "",
          "post_date": "08/08/2025 03:50:43",
          "content": "<p>A great job!</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 3265910,
          "author_name": "aichangeworld",
          "author_url": "",
          "post_date": "08/08/2025 05:56:31",
          "content": "<p>only imu or all data, i wish your reply.</p>",
          "votes": null,
          "replies": [
            {
              "id": 3265929,
              "author_name": "gerwynng",
              "author_url": "",
              "post_date": "08/08/2025 06:51:29",
              "content": "<p><a href=\"https://www.kaggle.com/aichangeworld\" target=\"_blank\">@aichangeworld</a> i worked on both</p>",
              "votes": null,
              "replies": []
            }
          ]
        },
        {
          "id": 3266018,
          "author_name": "stefanoclss",
          "author_url": "",
          "post_date": "08/08/2025 08:45:10",
          "content": "<p>Did you use the aggregated features into your NN models?</p>",
          "votes": null,
          "replies": [
            {
              "id": 3266029,
              "author_name": "gerwynng",
              "author_url": "",
              "post_date": "08/08/2025 09:13:44",
              "content": "<p>I did not.</p>\n<p>I tried adding \"simpler\" aggregated features, like rolling stats, segment stats, etc. Did not help. </p>\n<p>So i didn't bother to try the more \"complex\" features.</p>",
              "votes": null,
              "replies": [
                {
                  "id": 3266034,
                  "author_name": "stefanoclss",
                  "author_url": "",
                  "post_date": "08/08/2025 09:25:09",
                  "content": "<p>Hmm okay. I'll take that into account when moving to NN.</p>",
                  "votes": null,
                  "replies": []
                },
                {
                  "id": 3266036,
                  "author_name": "stefanoclss",
                  "author_url": "",
                  "post_date": "08/08/2025 09:26:23",
                  "content": "<p>Because looking at the public nb single models. I'm almost able to match the best single model by using gbm's.</p>",
                  "votes": null,
                  "replies": []
                }
              ]
            }
          ]
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "3264009": "Hi everyone,\n\nI’ve been experimenting with pure GBDT approaches (LightGBM, XGBoost, CatBoost) for the CMI Detect Behavior with Sensor Data competition, but my best public LB is stuck around 0.71.\n\nI’m curious:\n\nHas anyone managed to hit 0.80+ using only GBDT models?\nIf so, what’s the highest score you’ve seen with this family of models?\n\nI’m not asking for code and approach—just want to know whether breaking the 0.80 barrier is realistically achievable with tree-based methods alone, or if deep models are simply necessary here.\n\nThanks a lot for sharing your experience!",
    "3264041": "LB 0.75 , GBDT has a relatively small effect.",
    "3264138": "LB 0.80, GDBT for IMU and all data. (Switching model) 0.76 for IMU only 0.83 CV for all data.",
    "3265863": "I worked with LGBM only for the first few weeks, only pushed to 0.804. \n\nWas not able to make any more progress for another week. \n\nMy first naive CNN submit was able to beat 0.804, so i decided to divert efforts there.",
    "3265869": "A great job!",
    "3265910": "only imu or all data, i wish your reply.",
    "3265929": "aichangeworld i worked on both",
    "3266018": "Did you use the aggregated features into your NN models?",
    "3266029": "I did not.\n\nI tried adding \"simpler\" aggregated features, like rolling stats, segment stats, etc. Did not help. \n\nSo i didn't bother to try the more \"complex\" features.",
    "3266034": "Hmm okay. I'll take that into account when moving to NN.",
    "3266036": "Because looking at the public nb single models. I'm almost able to match the best single model by using gbm's.",
    "3266505": "https://www.kaggle.com/code/jiaoyouzhang/cmi-2025-only-lightgbm-training"
  },
  "source": "meta"
}