{
  "id": 590715,
  "title": "Traditional LGB, XGB vs. Deep model",
  "url": "/competitions/cmi-detect-behavior-with-sensor-data/discussion/590715",
  "author_name": "",
  "post_date": "2025-07-22T12:08:04.896688200Z",
  "votes": null,
  "comment_count": 1,
  "views": 0,
  "content": "<p>I've noticed that many scores using traditional tree models, such as LGB and XGB, are not as good as those using deep models. On previous structured data, the traditional tree model performed better. But why was the result not satisfactory in this competition? Is there anyone who can explain?</p>",
  "messages": [
    {
      "id": "3252300",
      "postDate": "07/22/2025 12:08:04",
      "content": "<p>I've noticed that many scores using traditional tree models, such as LGB and XGB, are not as good as those using deep models. On previous structured data, the traditional tree model performed better. But why was the result not satisfactory in this competition? Is there anyone who can explain?</p>",
      "rawMarkdown": "I've noticed that many scores using traditional tree models, such as LGB and XGB, are not as good as those using deep models. On previous structured data, the traditional tree model performed better. But why was the result not satisfactory in this competition? Is there anyone who can explain?",
      "votes": null
    },
    {
      "id": "3252732",
      "postDate": "07/23/2025 09:44:48",
      "content": "<p><a href=\"https://www.kaggle.com/fengqh\" target=\"_blank\">@fengqh</a><br>\nTree-based models like LightGBM and XGBoost are excellent for most tabular data problems  they're fast, interpretable, and usually give you solid results with minimal fuss. But when it comes to temporal data, they're fighting with one hand tied behind their back.<br>\nthese tree models make decisions by splitting on individual features at specific points in time. They can't naturally <em>remember</em> what happened 50 timesteps ago or understand that a gradual trend over the last hour might be more important than the current reading.   <br>\nIt's like trying to understand a conversation by only looking at individual words instead of following the flow of sentences.  <br>\nDeep learning models, on the other hand, are built for exactly this kind of sequential reasoning. An LSTM can literally maintain a \"memory\" of past sensor readings and learn which historical patterns matter most for prediction. Transformers can even look at the entire sequence simultaneously and figure out complex relationships between different time periods.   </p>\n<p>Dont get me wrong,<br>\nYou can still make Traditional Models perform decent by extensive feature engineering, but its not what they are designed for.</p>",
      "rawMarkdown": "fengqh\nTree-based models like LightGBM and XGBoost are excellent for most tabular data problems  they're fast, interpretable, and usually give you solid results with minimal fuss. But when it comes to temporal data, they're fighting with one hand tied behind their back.\n\n these tree models make decisions by splitting on individual features at specific points in time. They can't naturally *remember* what happened 50 timesteps ago or understand that a gradual trend over the last hour might be more important than the current reading.   \n\n\nIt's like trying to understand a conversation by only looking at individual words instead of following the flow of sentences.  \n\nDeep learning models, on the other hand, are built for exactly this kind of sequential reasoning. An LSTM can literally maintain a \"memory\" of past sensor readings and learn which historical patterns matter most for prediction. Transformers can even look at the entire sequence simultaneously and figure out complex relationships between different time periods.   \n  \nDont get me wrong,\nYou can still make Traditional Models perform decent by extensive feature engineering, but its not what they are designed for.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 3252732,
      "author_name": "abdulhanan677",
      "author_url": "",
      "post_date": "07/23/2025 09:44:48",
      "content": "<p><a href=\"https://www.kaggle.com/fengqh\" target=\"_blank\">@fengqh</a><br>\nTree-based models like LightGBM and XGBoost are excellent for most tabular data problems  they're fast, interpretable, and usually give you solid results with minimal fuss. But when it comes to temporal data, they're fighting with one hand tied behind their back.<br>\nthese tree models make decisions by splitting on individual features at specific points in time. They can't naturally <em>remember</em> what happened 50 timesteps ago or understand that a gradual trend over the last hour might be more important than the current reading.   <br>\nIt's like trying to understand a conversation by only looking at individual words instead of following the flow of sentences.  <br>\nDeep learning models, on the other hand, are built for exactly this kind of sequential reasoning. An LSTM can literally maintain a \"memory\" of past sensor readings and learn which historical patterns matter most for prediction. Transformers can even look at the entire sequence simultaneously and figure out complex relationships between different time periods.   </p>\n<p>Dont get me wrong,<br>\nYou can still make Traditional Models perform decent by extensive feature engineering, but its not what they are designed for.</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "3252300": "I've noticed that many scores using traditional tree models, such as LGB and XGB, are not as good as those using deep models. On previous structured data, the traditional tree model performed better. But why was the result not satisfactory in this competition? Is there anyone who can explain?",
    "3252732": "fengqh\nTree-based models like LightGBM and XGBoost are excellent for most tabular data problems  they're fast, interpretable, and usually give you solid results with minimal fuss. But when it comes to temporal data, they're fighting with one hand tied behind their back.\n\n these tree models make decisions by splitting on individual features at specific points in time. They can't naturally *remember* what happened 50 timesteps ago or understand that a gradual trend over the last hour might be more important than the current reading.   \n\n\nIt's like trying to understand a conversation by only looking at individual words instead of following the flow of sentences.  \n\nDeep learning models, on the other hand, are built for exactly this kind of sequential reasoning. An LSTM can literally maintain a \"memory\" of past sensor readings and learn which historical patterns matter most for prediction. Transformers can even look at the entire sequence simultaneously and figure out complex relationships between different time periods.   \n  \nDont get me wrong,\nYou can still make Traditional Models perform decent by extensive feature engineering, but its not what they are designed for."
  },
  "source": "meta"
}