{
  "id": 590815,
  "title": "What is your highest score with feature engineering and decision trees ?",
  "url": "/competitions/cmi-detect-behavior-with-sensor-data/discussion/590815",
  "author_name": "",
  "post_date": "2025-07-23T07:41:21.898204600Z",
  "votes": null,
  "comment_count": 4,
  "views": 0,
  "content": "<p>I've seen many notebooks using a deep learning approach. I was wondering what was the best score you were able to achieve only with feature engineering and xgboost (or any other decision trees model) ?</p>",
  "messages": [
    {
      "id": "3252683",
      "postDate": "07/23/2025 07:41:21",
      "content": "<p>I've seen many notebooks using a deep learning approach. I was wondering what was the best score you were able to achieve only with feature engineering and xgboost (or any other decision trees model) ?</p>",
      "rawMarkdown": "I've seen many notebooks using a deep learning approach. I was wondering what was the best score you were able to achieve only with feature engineering and xgboost (or any other decision trees model) ?",
      "votes": null
    },
    {
      "id": "3252723",
      "postDate": "07/23/2025 09:13:29",
      "content": "<p><a href=\"https://www.kaggle.com/nardii\" target=\"_blank\">@nardii</a> I achieved a score of <strong>0.73 lb</strong> using a simple Dual LightGBM model with extensive feature engineering. However, further improvements may not be worth the effort, as the model already had thousands of features makin training time-consuming. While it might reach up to <strong>~0.75 lb</strong> with ensemble methods and tuning, I wouldn't recommend it due to diminishing returns.  </p>\n<p>I also came across a CatBoost model that scored <strong>0.76 lb</strong>, but the training process wasn't shared, so it's hard to assess its reliability.  </p>",
      "rawMarkdown": "nardii I achieved a score of **0.73 lb** using a simple Dual LightGBM model with extensive feature engineering. However, further improvements may not be worth the effort, as the model already had thousands of features makin training time-consuming. While it might reach up to **~0.75 lb** with ensemble methods and tuning, I wouldn't recommend it due to diminishing returns.  \n\nI also came across a CatBoost model that scored **0.76 lb**, but the training process wasn't shared, so it's hard to assess its reliability.",
      "votes": null
    },
    {
      "id": "3252812",
      "postDate": "07/23/2025 13:26:57",
      "content": "<p>0.79lb,But subsequently, no matter what method I use, it's hard to achieve improvement</p>",
      "rawMarkdown": "0.79lb,But subsequently, no matter what method I use, it's hard to achieve improvement",
      "votes": null
    },
    {
      "id": "3252894",
      "postDate": "07/23/2025 16:23:45",
      "content": "<p>0.77 cv - 0.76 lb with single GBDT model using IMU only features</p>",
      "rawMarkdown": "0.77 cv - 0.76 lb with single GBDT model using IMU only features",
      "votes": null
    },
    {
      "id": "3252980",
      "postDate": "07/23/2025 19:55:02",
      "content": "<p>Indeed, the best I was able to achieve is 0.76 LB too, it seems to be like a plateau for me (even though I tried ensembling different models with different ensembling techniques)</p>",
      "rawMarkdown": "Indeed, the best I was able to achieve is 0.76 LB too, it seems to be like a plateau for me (even though I tried ensembling different models with different ensembling techniques)",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 3252723,
      "author_name": "abdulhanan677",
      "author_url": "",
      "post_date": "07/23/2025 09:13:29",
      "content": "<p><a href=\"https://www.kaggle.com/nardii\" target=\"_blank\">@nardii</a> I achieved a score of <strong>0.73 lb</strong> using a simple Dual LightGBM model with extensive feature engineering. However, further improvements may not be worth the effort, as the model already had thousands of features makin training time-consuming. While it might reach up to <strong>~0.75 lb</strong> with ensemble methods and tuning, I wouldn't recommend it due to diminishing returns.  </p>\n<p>I also came across a CatBoost model that scored <strong>0.76 lb</strong>, but the training process wasn't shared, so it's hard to assess its reliability.  </p>",
      "votes": null,
      "replies": [
        {
          "id": 3252980,
          "author_name": "nardii",
          "author_url": "",
          "post_date": "07/23/2025 19:55:02",
          "content": "<p>Indeed, the best I was able to achieve is 0.76 LB too, it seems to be like a plateau for me (even though I tried ensembling different models with different ensembling techniques)</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 3252812,
      "author_name": "aaayan",
      "author_url": "",
      "post_date": "07/23/2025 13:26:57",
      "content": "<p>0.79lb,But subsequently, no matter what method I use, it's hard to achieve improvement</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 3252894,
      "author_name": "l0glikelihood",
      "author_url": "",
      "post_date": "07/23/2025 16:23:45",
      "content": "<p>0.77 cv - 0.76 lb with single GBDT model using IMU only features</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "3252683": "I've seen many notebooks using a deep learning approach. I was wondering what was the best score you were able to achieve only with feature engineering and xgboost (or any other decision trees model) ?",
    "3252723": "nardii I achieved a score of **0.73 lb** using a simple Dual LightGBM model with extensive feature engineering. However, further improvements may not be worth the effort, as the model already had thousands of features makin training time-consuming. While it might reach up to **~0.75 lb** with ensemble methods and tuning, I wouldn't recommend it due to diminishing returns.  \n\nI also came across a CatBoost model that scored **0.76 lb**, but the training process wasn't shared, so it's hard to assess its reliability.",
    "3252812": "0.79lb,But subsequently, no matter what method I use, it's hard to achieve improvement",
    "3252894": "0.77 cv - 0.76 lb with single GBDT model using IMU only features",
    "3252980": "Indeed, the best I was able to achieve is 0.76 LB too, it seems to be like a plateau for me (even though I tried ensembling different models with different ensembling techniques)"
  },
  "source": "meta"
}