![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F2180993%2Fc4ba4efb3a543bbee55635dd2698d9f3%2Fcheatsheet.png?generation=1589816993312396&alt=media\")
Most data have categorical variable which are nominal and also have larger values. converting into one-hot encoding has made decision tree based ensemble models like Random Forest,Xgb performance even worse? \nThen how can we deal with such categorical variables for these mentioned models?How to encode a categorical variable with larger values(approx 600) for Decision Tree Based ensemble models?
\n\nI am struggling since very long , I will highly appreciate for valuable inputs and suggestions.
", "messages": [ { "id": "851506", "postDate": "05/17/2020 17:18:17", "content": "Most data have categorical variable which are nominal and also have larger values. converting into one-hot encoding has made decision tree based ensemble models like Random Forest,Xgb performance even worse? \nThen how can we deal with such categorical variables for these mentioned models?How to encode a categorical variable with larger values(approx 600) for Decision Tree Based ensemble models?
\n\nI am struggling since very long , I will highly appreciate for valuable inputs and suggestions.
", "rawMarkdown": "Most data have categorical variable which are nominal and also have larger values. converting into one-hot encoding has made decision tree based ensemble models like Random Forest,Xgb performance even worse? \nThen how can we deal with such categorical variables for these mentioned models?How to encode a categorical variable with larger values(approx 600) for Decision Tree Based ensemble models? \n\nI am struggling since very long , I will highly appreciate for valuable inputs and suggestions.", "votes": null }, { "id": "851840", "postDate": "05/18/2020 00:47:37", "content": "If by larger values you mean 1 categorical variable has 600 discrete values, than I guess for me the first thing that comes to mind would be performing some kind of binning to decrease the number of discrete values. Obviously if you are a domain expert or have access to one, this can make the process easier.
\n\nRelated to this, if you have some numeric features as well, maybe you could try \"unsupervised binning\", and what I mean by that is performing some kind of unsupervised clustering to figure out which discrete values are closer to each other (i.e. they are in the same cluster) and then bin them accordingly.
", "rawMarkdown": "If by larger values you mean 1 categorical variable has 600 discrete values, than I guess for me the first thing that comes to mind would be performing some kind of binning to decrease the number of discrete values. Obviously if you are a domain expert or have access to one, this can make the process easier. \n\nRelated to this, if you have some numeric features as well, maybe you could try \"unsupervised binning\", and what I mean by that is performing some kind of unsupervised clustering to figure out which discrete values are closer to each other (i.e. they are in the same cluster) and then bin them accordingly.", "votes": null }, { "id": "852441", "postDate": "05/18/2020 12:55:48", "content": "You can try with the following encoding techniques:\nMean encoding\nHash encoding \nFrequency encoding\nAttaching a git file for the demonstration.\nhope it will help.\nhttps://github.com/strquark/Encoding/blob/master/encode.py
", "rawMarkdown": "You can try with the following encoding techniques:\nMean encoding\nHash encoding \nFrequency encoding\nAttaching a git file for the demonstration.\nhope it will help.\nhttps://github.com/strquark/Encoding/blob/master/encode.py", "votes": null }, { "id": "852680", "postDate": "05/18/2020 15:53:28", "content": "There are several approaches that can be useful to you.\n 1) I can use raw features without encoding. For example, catboost can handle them automatically, just give him column numbers.\n 2) What is the frequency distribution of your variables? You can encode only the most popular labels and name others like 'rare'. \n 3) As sarang210 said in his answer Mean, Hash and Frequency encoding methods can be useful for you.\n You can find their realizations here). There are plenty of encoders there, fully compatible with sklearn.\nI also found an encoder selection cheat-sheet, maybe it will help you.
\n\n
Also, check this and this articles.
\n\nHope, it will help you.
", "rawMarkdown": "There are several approaches that can be useful to you.\n 1) I can use raw features without encoding. For example, catboost can [handle them automatically]((https://catboost.ai/docs/concepts/python-usages-examples.html#binary-classification)), just give him column numbers.\n 2) What is the frequency distribution of your variables? You can encode only the most popular labels and name others like 'rare'. \n 3) As sarang210 said in his answer Mean, Hash and Frequency encoding methods can be useful for you.\n You can find their realizations [here](http://contrib.scikit-learn.org/category_encoders/)). There are plenty of encoders there, fully compatible with sklearn.\nI also found an encoder selection cheat-sheet, maybe it will help you.\n\n\n\nAlso, check [this](https://towardsdatascience.com/smarter-ways-to-encode-categorical-data-for-machine-learning-part-1-of-3-6dca2f71b159) and [this](https://towardsdatascience.com/all-about-categorical-variable-encoding-305f3361fd02) articles.\n\nHope, it will help you.", "votes": null }, { "id": "854689", "postDate": "05/20/2020 08:30:11", "content": "Basically, you can try any method from http://contrib.scikit-learn.org/category_encoders/\nI see this competition as a test case for which method actually works. I can say the most common knowledge methods do not work empirically. Try it out for this competition and see if you can beat the public notebook which uses plain logistic regression.
", "rawMarkdown": "Basically, you can try any method from http://contrib.scikit-learn.org/category_encoders/\nI see this competition as a test case for which method actually works. I can say the most common knowledge methods do not work empirically. Try it out for this competition and see if you can beat the public notebook which uses plain logistic regression.", "votes": null } ], "comments": [ { "id": 851840, "author_name": "thesherpafromalabama", "author_url": "", "post_date": "05/18/2020 00:47:37", "content": "If by larger values you mean 1 categorical variable has 600 discrete values, than I guess for me the first thing that comes to mind would be performing some kind of binning to decrease the number of discrete values. Obviously if you are a domain expert or have access to one, this can make the process easier.
\n\nRelated to this, if you have some numeric features as well, maybe you could try \"unsupervised binning\", and what I mean by that is performing some kind of unsupervised clustering to figure out which discrete values are closer to each other (i.e. they are in the same cluster) and then bin them accordingly.
", "votes": null, "replies": [] }, { "id": 852441, "author_name": "sarang210", "author_url": "", "post_date": "05/18/2020 12:55:48", "content": "You can try with the following encoding techniques:\nMean encoding\nHash encoding \nFrequency encoding\nAttaching a git file for the demonstration.\nhope it will help.\nhttps://github.com/strquark/Encoding/blob/master/encode.py
", "votes": null, "replies": [] }, { "id": 852680, "author_name": "gooogr", "author_url": "", "post_date": "05/18/2020 15:53:28", "content": "There are several approaches that can be useful to you.\n 1) I can use raw features without encoding. For example, catboost can handle them automatically, just give him column numbers.\n 2) What is the frequency distribution of your variables? You can encode only the most popular labels and name others like 'rare'. \n 3) As sarang210 said in his answer Mean, Hash and Frequency encoding methods can be useful for you.\n You can find their realizations here). There are plenty of encoders there, fully compatible with sklearn.\nI also found an encoder selection cheat-sheet, maybe it will help you.
\n\n
Also, check this and this articles.
\n\nHope, it will help you.
", "votes": null, "replies": [] }, { "id": 854689, "author_name": "superant", "author_url": "", "post_date": "05/20/2020 08:30:11", "content": "Basically, you can try any method from http://contrib.scikit-learn.org/category_encoders/\nI see this competition as a test case for which method actually works. I can say the most common knowledge methods do not work empirically. Try it out for this competition and see if you can beat the public notebook which uses plain logistic regression.
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "851506": "Most data have categorical variable which are nominal and also have larger values. converting into one-hot encoding has made decision tree based ensemble models like Random Forest,Xgb performance even worse? \nThen how can we deal with such categorical variables for these mentioned models?How to encode a categorical variable with larger values(approx 600) for Decision Tree Based ensemble models? \n\nI am struggling since very long , I will highly appreciate for valuable inputs and suggestions.", "851840": "If by larger values you mean 1 categorical variable has 600 discrete values, than I guess for me the first thing that comes to mind would be performing some kind of binning to decrease the number of discrete values. Obviously if you are a domain expert or have access to one, this can make the process easier. \n\nRelated to this, if you have some numeric features as well, maybe you could try \"unsupervised binning\", and what I mean by that is performing some kind of unsupervised clustering to figure out which discrete values are closer to each other (i.e. they are in the same cluster) and then bin them accordingly.", "852441": "You can try with the following encoding techniques:\nMean encoding\nHash encoding \nFrequency encoding\nAttaching a git file for the demonstration.\nhope it will help.\nhttps://github.com/strquark/Encoding/blob/master/encode.py", "852680": "There are several approaches that can be useful to you.\n 1) I can use raw features without encoding. For example, catboost can [handle them automatically]((https://catboost.ai/docs/concepts/python-usages-examples.html#binary-classification)), just give him column numbers.\n 2) What is the frequency distribution of your variables? You can encode only the most popular labels and name others like 'rare'. \n 3) As sarang210 said in his answer Mean, Hash and Frequency encoding methods can be useful for you.\n You can find their realizations [here](http://contrib.scikit-learn.org/category_encoders/)). There are plenty of encoders there, fully compatible with sklearn.\nI also found an encoder selection cheat-sheet, maybe it will help you.\n\n\n\nAlso, check [this](https://towardsdatascience.com/smarter-ways-to-encode-categorical-data-for-machine-learning-part-1-of-3-6dca2f71b159) and [this](https://towardsdatascience.com/all-about-categorical-variable-encoding-305f3361fd02) articles.\n\nHope, it will help you.", "854689": "Basically, you can try any method from http://contrib.scikit-learn.org/category_encoders/\nI see this competition as a test case for which method actually works. I can say the most common knowledge methods do not work empirically. Try it out for this competition and see if you can beat the public notebook which uses plain logistic regression." }, "source": "meta" }