These might be newbie questions.\n1. What is target encoding?
\n\nWhy we want to use target encoding? Specifically, what are the advantages of using it compared to other encoding techniques?
Finally, when should we use the target encoding? are there any specific circumstances in the data where it makes sense to use target encoding?
Thanks for your answers :)
", "messages": [ { "id": "754855", "postDate": "02/24/2020 06:04:47", "content": "These might be newbie questions.\n1. What is target encoding?
\n\nWhy we want to use target encoding? Specifically, what are the advantages of using it compared to other encoding techniques?
Finally, when should we use the target encoding? are there any specific circumstances in the data where it makes sense to use target encoding?
Thanks for your answers :)
", "rawMarkdown": "These might be newbie questions.\n1. What is target encoding? \n\n2. Why we want to use target encoding? Specifically, what are the advantages of using it compared to other encoding techniques?\n\n3. Finally, when should we use the target encoding? are there any specific circumstances in the data where it makes sense to use target encoding?\n\nThanks for your answers :)", "votes": null }, { "id": "754940", "postDate": "02/24/2020 08:52:13", "content": "Link to h2o:\nTarget Encoding Example
", "rawMarkdown": "1. Target encoding is a technique to handle categorical features. In target encoding one calculates the relative frequency of a category regarding one characteristic of the target variable. For example: If there are categories a, b, c etc. and a target [0, 1] a might occur in 10% of the cases in target=1 b Mighty occur in 15% an der so forth. \n2 and 3. Target encoding reduces dimensionality compard to methods like one hot. Especially interesting when there are many categories. Due to the strong relation to the target it often leads to overfitting. Therefore, techniques to reduce overfitting are used like k-fold, mean smoothing, and/or adding random numbers.\n\nLink to h2o:\n[Target Encoding Example](http://docs.h2o.ai/h2o/latest-stable/h2o-docs/data-munging/target-encoding.html)", "votes": null }, { "id": "755067", "postDate": "02/24/2020 11:57:07", "content": "Method : Target encoding \nTarget-based encoding is numerization of categorical variables via target. In this method, we replace the categorical variable with just one new numerical variable and replace each category of the categorical variable with its corresponding probability of the target (if categorical) or average of the target (if numerical). The main drawbacks of this method are its dependency to the distribution of the target, and its lower predictability power compare to the binary encoding method.
\n\nfor example,
\n\nCountry Target\nIndia 1\nChina 0\nIndia 0\nChina 1\nIndia 1\nEncoding for India = [Number of true targets under the label India/ Total Number of targets under the label India] which is 2/3 = 0.66\nEncoding for India = [Number of true targets under the label India/ Total Number of targets under the label India] which is 2/3 = 0.66
\n\nCountry Target\nIndia 0.66\nChina 0.5
", "rawMarkdown": "Method : Target encoding \nTarget-based encoding is numerization of categorical variables via target. In this method, we replace the categorical variable with just one new numerical variable and replace each category of the categorical variable with its corresponding probability of the target (if categorical) or average of the target (if numerical). The main drawbacks of this method are its dependency to the distribution of the target, and its lower predictability power compare to the binary encoding method.\n\nfor example,\n\nCountry\tTarget\nIndia\t1\nChina\t0\nIndia\t0\nChina\t1\nIndia\t1\nEncoding for India = [Number of true targets under the label India/ Total Number of targets under the label India] which is 2/3 = 0.66\nEncoding for India = [Number of true targets under the label India/ Total Number of targets under the label India] which is 2/3 = 0.66\n\nCountry\tTarget\nIndia\t0.66\nChina\t0.5", "votes": null }, { "id": "755336", "postDate": "02/24/2020 17:15:01", "content": "Target encoding/Mean encoding is the process of encoding the target in a predictor variable not necessarily only categorical variables. Target encoding embodies the target variable in the predictor variable which is much more efficient than label encoding, which on the other hand has no relation to the target.
\n\nFor example if there is target : item_cnt_month and predictor variable: item_id. \nWe can create new target encoded variable: item_id_avg_item_cnt_month.\nThis new variable tells us the average value of item_cnt_month of an item_id over several months.
\n\nThe downside of target encoding as mentioned is the overfitting due to data leakage. This can be solved by regularization and averaged cross validation techniques
", "rawMarkdown": "Target encoding/Mean encoding is the process of encoding the target in a predictor variable not necessarily only categorical variables. Target encoding embodies the target variable in the predictor variable which is much more efficient than label encoding, which on the other hand has no relation to the target.\n\nFor example if there is target : item_cnt_month and predictor variable: item_id. \nWe can create new target encoded variable: item_id_avg_item_cnt_month.\nThis new variable tells us the average value of item_cnt_month of an item_id over several months.\n\nThe downside of target encoding as mentioned is the overfitting due to data leakage. This can be solved by regularization and averaged cross validation techniques", "votes": null }, { "id": "755362", "postDate": "02/24/2020 17:45:57", "content": "There are already good answers. I just want to add that maybe you can find the \"Intermediate Machine Learning\" and \"Feature Engineering\" micro-courses on Kaggle helpful if you are starting. In the third and second lesson respectively they talk about different enconding methods
", "rawMarkdown": "There are already good answers. I just want to add that maybe you can find the [\"Intermediate Machine Learning\"](https://www.kaggle.com/learn/intermediate-machine-learning) and [\"Feature Engineering\"](https://www.kaggle.com/learn/feature-engineering) micro-courses on Kaggle helpful if you are starting. In the third and second lesson respectively they talk about different enconding methods", "votes": null }, { "id": "755380", "postDate": "02/24/2020 18:14:51", "content": "\"Therefore, techniques to reduce overfitting are used like k-fold, mean smoothing, and/or adding random numbers.\"
\n\nSometimes Yes and sometimes No, No way... and this one is a really weird competition, so there is something else we have to invent, understand !
\n\nWe have really have to be more than creative !
", "rawMarkdown": "\"Therefore, techniques to reduce overfitting are used like k-fold, mean smoothing, and/or adding random numbers.\"\n\nSometimes Yes and sometimes No, No way... and this one is a really weird competition, so there is something else we have to invent, understand !\n\nWe have really have to be more than creative !", "votes": null }, { "id": "758425", "postDate": "02/27/2020 18:28:47", "content": "For each input category, you get a distribution of targets. You, then, calculate Cramér's V and Cholesky decompose its matrix. Then you retrieve a mean and a spread for each input category by resampling.
", "rawMarkdown": "For each input category, you get a distribution of targets. You, then, calculate Cramér's V and Cholesky decompose its matrix. Then you retrieve a mean and a spread for each input category by resampling.", "votes": null } ], "comments": [ { "id": 754940, "author_name": "frankmollard", "author_url": "", "post_date": "02/24/2020 08:52:13", "content": "Link to h2o:\nTarget Encoding Example
", "votes": null, "replies": [ { "id": 755380, "author_name": "pfichou", "author_url": "", "post_date": "02/24/2020 18:14:51", "content": "\"Therefore, techniques to reduce overfitting are used like k-fold, mean smoothing, and/or adding random numbers.\"
\n\nSometimes Yes and sometimes No, No way... and this one is a really weird competition, so there is something else we have to invent, understand !
\n\nWe have really have to be more than creative !
", "votes": null, "replies": [] } ] }, { "id": 755067, "author_name": "ganeshmundra", "author_url": "", "post_date": "02/24/2020 11:57:07", "content": "Method : Target encoding \nTarget-based encoding is numerization of categorical variables via target. In this method, we replace the categorical variable with just one new numerical variable and replace each category of the categorical variable with its corresponding probability of the target (if categorical) or average of the target (if numerical). The main drawbacks of this method are its dependency to the distribution of the target, and its lower predictability power compare to the binary encoding method.
\n\nfor example,
\n\nCountry Target\nIndia 1\nChina 0\nIndia 0\nChina 1\nIndia 1\nEncoding for India = [Number of true targets under the label India/ Total Number of targets under the label India] which is 2/3 = 0.66\nEncoding for India = [Number of true targets under the label India/ Total Number of targets under the label India] which is 2/3 = 0.66
\n\nCountry Target\nIndia 0.66\nChina 0.5
", "votes": null, "replies": [] }, { "id": 755336, "author_name": "vijay187", "author_url": "", "post_date": "02/24/2020 17:15:01", "content": "Target encoding/Mean encoding is the process of encoding the target in a predictor variable not necessarily only categorical variables. Target encoding embodies the target variable in the predictor variable which is much more efficient than label encoding, which on the other hand has no relation to the target.
\n\nFor example if there is target : item_cnt_month and predictor variable: item_id. \nWe can create new target encoded variable: item_id_avg_item_cnt_month.\nThis new variable tells us the average value of item_cnt_month of an item_id over several months.
\n\nThe downside of target encoding as mentioned is the overfitting due to data leakage. This can be solved by regularization and averaged cross validation techniques
", "votes": null, "replies": [] }, { "id": 755362, "author_name": "carlosdg", "author_url": "", "post_date": "02/24/2020 17:45:57", "content": "There are already good answers. I just want to add that maybe you can find the \"Intermediate Machine Learning\" and \"Feature Engineering\" micro-courses on Kaggle helpful if you are starting. In the third and second lesson respectively they talk about different enconding methods
", "votes": null, "replies": [] }, { "id": 758425, "author_name": "glimmung", "author_url": "", "post_date": "02/27/2020 18:28:47", "content": "For each input category, you get a distribution of targets. You, then, calculate Cramér's V and Cholesky decompose its matrix. Then you retrieve a mean and a spread for each input category by resampling.
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "754855": "These might be newbie questions.\n1. What is target encoding? \n\n2. Why we want to use target encoding? Specifically, what are the advantages of using it compared to other encoding techniques?\n\n3. Finally, when should we use the target encoding? are there any specific circumstances in the data where it makes sense to use target encoding?\n\nThanks for your answers :)", "754940": "1. Target encoding is a technique to handle categorical features. In target encoding one calculates the relative frequency of a category regarding one characteristic of the target variable. For example: If there are categories a, b, c etc. and a target [0, 1] a might occur in 10% of the cases in target=1 b Mighty occur in 15% an der so forth. \n2 and 3. Target encoding reduces dimensionality compard to methods like one hot. Especially interesting when there are many categories. Due to the strong relation to the target it often leads to overfitting. Therefore, techniques to reduce overfitting are used like k-fold, mean smoothing, and/or adding random numbers.\n\nLink to h2o:\n[Target Encoding Example](http://docs.h2o.ai/h2o/latest-stable/h2o-docs/data-munging/target-encoding.html)", "755067": "Method : Target encoding \nTarget-based encoding is numerization of categorical variables via target. In this method, we replace the categorical variable with just one new numerical variable and replace each category of the categorical variable with its corresponding probability of the target (if categorical) or average of the target (if numerical). The main drawbacks of this method are its dependency to the distribution of the target, and its lower predictability power compare to the binary encoding method.\n\nfor example,\n\nCountry\tTarget\nIndia\t1\nChina\t0\nIndia\t0\nChina\t1\nIndia\t1\nEncoding for India = [Number of true targets under the label India/ Total Number of targets under the label India] which is 2/3 = 0.66\nEncoding for India = [Number of true targets under the label India/ Total Number of targets under the label India] which is 2/3 = 0.66\n\nCountry\tTarget\nIndia\t0.66\nChina\t0.5", "755336": "Target encoding/Mean encoding is the process of encoding the target in a predictor variable not necessarily only categorical variables. Target encoding embodies the target variable in the predictor variable which is much more efficient than label encoding, which on the other hand has no relation to the target.\n\nFor example if there is target : item_cnt_month and predictor variable: item_id. \nWe can create new target encoded variable: item_id_avg_item_cnt_month.\nThis new variable tells us the average value of item_cnt_month of an item_id over several months.\n\nThe downside of target encoding as mentioned is the overfitting due to data leakage. This can be solved by regularization and averaged cross validation techniques", "755362": "There are already good answers. I just want to add that maybe you can find the [\"Intermediate Machine Learning\"](https://www.kaggle.com/learn/intermediate-machine-learning) and [\"Feature Engineering\"](https://www.kaggle.com/learn/feature-engineering) micro-courses on Kaggle helpful if you are starting. In the third and second lesson respectively they talk about different enconding methods", "755380": "\"Therefore, techniques to reduce overfitting are used like k-fold, mean smoothing, and/or adding random numbers.\"\n\nSometimes Yes and sometimes No, No way... and this one is a really weird competition, so there is something else we have to invent, understand !\n\nWe have really have to be more than creative !", "758425": "For each input category, you get a distribution of targets. You, then, calculate Cramér's V and Cholesky decompose its matrix. Then you retrieve a mean and a spread for each input category by resampling." }, "source": "meta" }