{"cells":[{"metadata":{},"cell_type":"markdown","source":"# *Categorical encoding is a process of converting categories to numbers.*\n\nIn many Machine-learning or Data Science activities, the data set might contain text or categorical values (basically non-numerical values). For example, color feature having values like red, orange, blue, white etc. Meal plan having values like breakfast, lunch, snacks, dinner, tea etc. Few algorithms such as CATBOAST, decision-trees can handle categorical values very well but most of the algorithms expect numerical values to achieve state-of-the-art results. A machine can only understand numbers so we need to convert categorical columns to numerical columns"},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"cell_type":"code","source":"import numpy as np\nimport pandas as pd \n\n\nimport os\nfor dirname, _, filenames in os.walk('/kaggle/input'):\n    for filename in filenames:\n        print(os.path.join(dirname, filename))","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","trusted":true},"cell_type":"code","source":"df = pd.read_csv('/kaggle/input/cat-in-the-dat-ii/train.csv')\ndf.head(10)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"for feature in df.columns:\n    print('The feature is {} and number of categories are {}'.format(feature,len(df[feature].unique())))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Techniques for Categorical Encoding\n\nThere are several techniques for categorical techniques:\n* Label Encoding or Ordinal Encoding\n* One hot Encoding\n* Dummy Encoding\n* Effect Encoding\n* Binary Encoding\n* BaseN Encoding\n* Hash Encoding\n* Target Encoding\n\nIn this notebook I will be going through the two most commonly encoding techniques: *Label Encoding and One-Hot Encoding*"},{"metadata":{"trusted":true},"cell_type":"code","source":"#Though there are many more columns in the dataset, to understand encoding, we will focus on one categorical column only.\nfeatures = ['bin_4','nom_0','nom_4','ord_2']\n\ndf1  = df[features]\n\nprint(df1.shape)\n\ndf1.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"df1.info()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Label Encoding Or Ordinal Encoding\n\nThis approach is very simple and it involves converting each value in a column to a number.In this case, retaining the order is important. Hence encoding should reflect the sequence.\n\n**In Label encoding, *each label is converted into an integer value*.** "},{"metadata":{"trusted":true},"cell_type":"code","source":"from sklearn.preprocessing import LabelEncoder\n\nlabel_encoder = LabelEncoder()\n\n#lets take the feature ord_2 here\n\ndf1['ord_2_en'] = label_encoder.fit_transform(df1['ord_2'].astype(str))\n\ndf1.head(10)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"That how label encoding is performed but did you notice a pattern??\n\nYES, there is a pattern!! In the above scenario, the temperatures do not have an order or rank. But, when label encoding is performed, the temperatures are **ranked based on the alphabets**. Due to this, there is a very high probability that the model captures the relationship between temperatures such as Boiling Hot < Cold < Freezing < Lava Hot < Warm.\n\nThis ordering issue is addressed in another commonly used technique: **One-Hot Encoding**."},{"metadata":{},"cell_type":"markdown","source":"# One-Hot Encoding\n\nIn this technique, **each category value is converted into a new column and assigned a 1 or 0**  (notation for true/false) value to the column. "},{"metadata":{"trusted":true},"cell_type":"code","source":"from sklearn.preprocessing import OneHotEncoder\n\nohe = OneHotEncoder()\n\n#lets take the feature bin_4 here.\ndf1 = df1.dropna()\n\nenc_df = pd.DataFrame(ohe.fit_transform(df1[['bin_4']]).toarray())\n\nfinal_df = df1.join(enc_df)\n\nfinal_df.head(15)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"The major issue with OneHotEncoder is that it creates dummy variables(categories as seperate features). This leads to the problem of multicollinearity. Multicollinearity occurs where there is a dependency between the independent features. Multicollinearity is a serious issue in machine learning models like Linear Regression and Logistic Regression.  "},{"metadata":{},"cell_type":"markdown","source":"# When to use a Label Encoding vs. One Hot Encoding?\n\n\nWe apply One-Hot Encoding when:\n\n* The categorical feature is **not ordinal** (like the bin_4 feature above)\n* The number of categorical features is less so one-hot encoding can be effectively applied\n\nWe apply Label Encoding when:\n\n* The categorical feature is **ordinal** (like warm, cold, freezing, etc)\n* The number of categories is quite large as one-hot encoding can lead to high memory consumption"},{"metadata":{"trusted":true},"cell_type":"code","source":"","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}