{
  "id": 80677,
  "title": "Does LightGBM or XGBoost need to manually select features?",
  "url": "/competitions/elo-merchant-category-recommendation/discussion/80677",
  "author_name": "",
  "post_date": "2019-02-15T11:37:16.529614200Z",
  "votes": 2,
  "comment_count": 2,
  "views": 0,
  "content": "<p>1) I am a newbie and I am wondering if LightGBM or XGBoost need to manually select features? Doesn't the model automatically select the feature with the greatest gain when training?</p>\n\n<p>2) What if two features contain duplicate information or are identical? I will also try to test this.</p>\n\n<p>Thank you guys very much in advance.</p>",
  "messages": [
    {
      "id": "472115",
      "postDate": "02/15/2019 11:37:16",
      "content": "<p>1) I am a newbie and I am wondering if LightGBM or XGBoost need to manually select features? Doesn't the model automatically select the feature with the greatest gain when training?</p>\n\n<p>2) What if two features contain duplicate information or are identical? I will also try to test this.</p>\n\n<p>Thank you guys very much in advance.</p>",
      "rawMarkdown": "1) I am a newbie and I am wondering if LightGBM or XGBoost need to manually select features? Doesn't the model automatically select the feature with the greatest gain when training?\n\n2) What if two features contain duplicate information or are identical? I will also try to test this.\n\nThank you guys very much in advance.",
      "votes": null
    },
    {
      "id": "472277",
      "postDate": "02/15/2019 16:17:59",
      "content": "<ol>\n<li><p>If you add regularization parameters such as lambda and alpha to your model then it automatically selects features.  You can also figure out the variable importance of each model after you train.  However, automatic feature selection in my experience works better.</p></li>\n<li><p>If two features contain duplicate information, then any decision tree model will be immune from it.\nFrom this link <a href=\"https://datascience.stackexchange.com/questions/12554/does-xgboost-handle-multicollinearity-by-itself\">https://datascience.stackexchange.com/questions/12554/does-xgboost-handle-multicollinearity-by-itself</a>:\n\"Decision trees are by nature immune to multi-collinearity. For example, if you have 2 features which are 99% correlated, when deciding upon a split the tree will choose only one of them. Other models such as Logistic regression would use both the features.\"</p></li>\n</ol>",
      "rawMarkdown": "1. If you add regularization parameters such as lambda and alpha to your model then it automatically selects features.  You can also figure out the variable importance of each model after you train.  However, automatic feature selection in my experience works better.\n\n2. If two features contain duplicate information, then any decision tree model will be immune from it.\nFrom this link https://datascience.stackexchange.com/questions/12554/does-xgboost-handle-multicollinearity-by-itself:\n\"Decision trees are by nature immune to multi-collinearity. For example, if you have 2 features which are 99% correlated, when deciding upon a split the tree will choose only one of them. Other models such as Logistic regression would use both the features.\"",
      "votes": null
    },
    {
      "id": "472684",
      "postDate": "02/16/2019 13:19:12",
      "content": "<p>Even the the most important feature decided by LGB may be the one that causes the overfitting. </p>",
      "rawMarkdown": "Even the the most important feature decided by LGB may be the one that causes the overfitting.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 472277,
      "author_name": "petersk20",
      "author_url": "",
      "post_date": "02/15/2019 16:17:59",
      "content": "<ol>\n<li><p>If you add regularization parameters such as lambda and alpha to your model then it automatically selects features.  You can also figure out the variable importance of each model after you train.  However, automatic feature selection in my experience works better.</p></li>\n<li><p>If two features contain duplicate information, then any decision tree model will be immune from it.\nFrom this link <a href=\"https://datascience.stackexchange.com/questions/12554/does-xgboost-handle-multicollinearity-by-itself\">https://datascience.stackexchange.com/questions/12554/does-xgboost-handle-multicollinearity-by-itself</a>:\n\"Decision trees are by nature immune to multi-collinearity. For example, if you have 2 features which are 99% correlated, when deciding upon a split the tree will choose only one of them. Other models such as Logistic regression would use both the features.\"</p></li>\n</ol>",
      "votes": null,
      "replies": []
    },
    {
      "id": 472684,
      "author_name": "longyin2",
      "author_url": "",
      "post_date": "02/16/2019 13:19:12",
      "content": "<p>Even the the most important feature decided by LGB may be the one that causes the overfitting. </p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "472115": "1) I am a newbie and I am wondering if LightGBM or XGBoost need to manually select features? Doesn't the model automatically select the feature with the greatest gain when training?\n\n2) What if two features contain duplicate information or are identical? I will also try to test this.\n\nThank you guys very much in advance.",
    "472277": "1. If you add regularization parameters such as lambda and alpha to your model then it automatically selects features.  You can also figure out the variable importance of each model after you train.  However, automatic feature selection in my experience works better.\n\n2. If two features contain duplicate information, then any decision tree model will be immune from it.\nFrom this link https://datascience.stackexchange.com/questions/12554/does-xgboost-handle-multicollinearity-by-itself:\n\"Decision trees are by nature immune to multi-collinearity. For example, if you have 2 features which are 99% correlated, when deciding upon a split the tree will choose only one of them. Other models such as Logistic regression would use both the features.\"",
    "472684": "Even the the most important feature decided by LGB may be the one that causes the overfitting."
  },
  "source": "meta"
}