![\"feature](\"https://image.ibb.co/kb5gCq/Screen-Shot-2018-11-12-at-1-12-23-PM.png\")
\nMy question is how fullVisitorId and visitId can be the most important features?\nAm I overfitting ?Hi forks,\nI'm trying a dummy model with xgboost and look at its feature important\n\nMy question is how fullVisitorId and visitId can be the most important features?\nAm I overfitting ?
Edit : I found this blog post Interpretable Machine Learning with XGBoost that describe how feature important in XGBoost works.
", "messages": [ { "id": "419534", "postDate": "11/12/2018 06:15:12", "content": "Hi forks,\nI'm trying a dummy model with xgboost and look at its feature important\n\nMy question is how fullVisitorId and visitId can be the most important features?\nAm I overfitting ?
Edit : I found this blog post Interpretable Machine Learning with XGBoost that describe how feature important in XGBoost works.
", "rawMarkdown": "Hi forks,\nI'm trying a dummy model with xgboost and look at its feature important\n![feature important][1]\nMy question is how fullVisitorId and visitId can be the most important features?\nAm I overfitting ?\n\n---\nEdit : I found this blog post [Interpretable Machine Learning with XGBoost][2] that describe how feature important in XGBoost works.\n\n\n [1]: https://image.ibb.co/kb5gCq/Screen-Shot-2018-11-12-at-1-12-23-PM.png\n [2]: https://towardsdatascience.com/interpretable-machine-learning-with-xgboost-9ec80d148d27", "votes": null }, { "id": "419545", "postDate": "11/12/2018 06:59:12", "content": "fullVisitorId shouldn't be train features
", "rawMarkdown": "fullVisitorId shouldn't be train features", "votes": null }, { "id": "419563", "postDate": "11/12/2018 07:47:22", "content": "As most of the fVID and vID are unique in the data, the model can get the exact transaction revenue by them alone, so they seem to be important. But generally they shouldn't be accounted as features, though it seems that in the new competition, they ARE the features.
", "rawMarkdown": "As most of the fVID and vID are unique in the data, the model can get the exact transaction revenue by them alone, so they seem to be important. But generally they shouldn't be accounted as features, though it seems that in the new competition, they ARE the features.", "votes": null }, { "id": "420355", "postDate": "11/13/2018 13:42:08", "content": "Yes you are overfitting, Since these IDs are mostly unique, the model is able to map them with the revenue while training. I guess your model will perform poorly on validation data, since the IDs in validation or test set will not be same as training data. \nYou should remove these two features. ML model should be able to generalize well while training, By keeping these two features you are making it difficult for your model to generalize.
\n\nI think your model in current state will still have predictive capabilities on validation data because there are so many 0s in target variable, which is helping the model generalize by mistake. But it can do much better
", "rawMarkdown": "Yes you are overfitting, Since these IDs are mostly unique, the model is able to map them with the revenue while training. I guess your model will perform poorly on validation data, since the IDs in validation or test set will not be same as training data. \nYou should remove these two features. ML model should be able to generalize well while training, By keeping these two features you are making it difficult for your model to generalize. \n\nI think your model in current state will still have predictive capabilities on validation data because there are so many 0s in target variable, which is helping the model generalize by mistake. But it can do much better", "votes": null }, { "id": "420439", "postDate": "11/13/2018 16:19:38", "content": "The importance you are plotting is \"split\" importance. The more splits present in the model, the more important the feature. Since fullVisitorId and visitId are very large integer numbers, it's likely that there are a lot of splits on them since they have a very large range. This makes their split importance a lot higher. I recommend to use SHAP values for more realistic importance values.
fullVisitorId shouldn't be train features
", "votes": null, "replies": [] }, { "id": 419563, "author_name": "aryerez", "author_url": "", "post_date": "11/12/2018 07:47:22", "content": "As most of the fVID and vID are unique in the data, the model can get the exact transaction revenue by them alone, so they seem to be important. But generally they shouldn't be accounted as features, though it seems that in the new competition, they ARE the features.
", "votes": null, "replies": [] }, { "id": 420355, "author_name": "karangautam", "author_url": "", "post_date": "11/13/2018 13:42:08", "content": "Yes you are overfitting, Since these IDs are mostly unique, the model is able to map them with the revenue while training. I guess your model will perform poorly on validation data, since the IDs in validation or test set will not be same as training data. \nYou should remove these two features. ML model should be able to generalize well while training, By keeping these two features you are making it difficult for your model to generalize.
\n\nI think your model in current state will still have predictive capabilities on validation data because there are so many 0s in target variable, which is helping the model generalize by mistake. But it can do much better
", "votes": null, "replies": [] }, { "id": 420439, "author_name": "returnofsputnik", "author_url": "", "post_date": "11/13/2018 16:19:38", "content": "The importance you are plotting is \"split\" importance. The more splits present in the model, the more important the feature. Since fullVisitorId and visitId are very large integer numbers, it's likely that there are a lot of splits on them since they have a very large range. This makes their split importance a lot higher. I recommend to use SHAP values for more realistic importance values.