{"cells":[{"metadata":{},"cell_type":"markdown","source":"\n

Categorical Feature Encoding Challenge II

\n"},{"metadata":{},"cell_type":"markdown","source":"## Competition Description\nThis follow-up competition offers an even more challenging dataset so that you can continue to build your skills with the common machine learning task of encoding categorical variables. This challenge adds the additional complexity of feature interactions, as well as missing data.\n\nThis Playground competition will give you the opportunity to try different encoding schemes for different algorithms to compare how they perform. We encourage you to share what you find with the community."},{"metadata":{},"cell_type":"markdown","source":"
\n

Notebook Content!

\n Import Libraries1\n Load Data2\n Visualization of data3\n Separate continuous, categorical and label column names4 \n Train and test Split5\n Creating the Model6\n Model Evaluation7\n References8 "},{"metadata":{},"cell_type":"markdown","source":"\n## 1. Import Libraries"},{"metadata":{"trusted":true},"cell_type":"code","source":"# Operating system dependent\nimport os\n\n# linear algebra\nimport numpy as np\n\n# data processing, CSV file I/O (e.g. pd.read_csv)\nimport pandas as pd\n\n#Visualization\nimport matplotlib.pyplot as plt\nimport seaborn as sns\n\n# Display HTML\nfrom IPython.core.display import display, HTML\n\n#collection of machine learning algorithms\nfrom sklearn.model_selection import train_test_split\nfrom sklearn.metrics import classification_report\nfrom sklearn.linear_model import LinearRegression\nfrom sklearn.metrics import accuracy_score\nfrom sklearn.metrics import confusion_matrix\n\n#Common Model Helpers\nfrom sklearn.preprocessing import LabelEncoder\nfrom sklearn import metrics\nfrom sklearn import model_selection\nimport pylab as pl\nfrom sklearn.metrics import roc_curve\nfrom sklearn.preprocessing import Imputer\n\nimport plotly.graph_objects as go\n#Ignore warnings\nimport warnings\nwarnings.filterwarnings('ignore')\n\nimport cufflinks as cf\ncf.go_offline()\n\n#%matplotlib inline = show plots in Jupyter Notebook browser\n%matplotlib inline","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"\n# 2. Load Data"},{"metadata":{"trusted":true},"cell_type":"code","source":"train = pd.read_csv(\"/kaggle/input/cat-in-the-dat-ii/train.csv\")\ntest = pd.read_csv(\"/kaggle/input/cat-in-the-dat-ii/test.csv\")\nsubmission = pd.read_csv(\"/kaggle/input/cat-in-the-dat-ii/sample_submission.csv\")","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"train_size = str(round(os.path.getsize('/kaggle/input/cat-in-the-dat-ii/train.csv') / 1000000, 2)) + 'MB'\ntest_size = str(round(os.path.getsize('/kaggle/input/cat-in-the-dat-ii/test.csv') / 1000000, 2)) + 'MB'\nsample_submission_size = str(round(os.path.getsize('/kaggle/input/cat-in-the-dat-ii/sample_submission.csv') / 1000000, 2)) + 'MB'","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"display(HTML(f\"\"\"\n\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
Filenamesize
Train{train_size}
Test{test_size}
Sample_Submission{sample_submission_size}
\n \n \n\"\"\"))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"train.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"train.columns","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"\n"},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"display(HTML(f\"\"\"\n \n \n \n \"\"\"))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"train.info()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"\n# 3. Visualization of data"},{"metadata":{"trusted":true},"cell_type":"code","source":"train.target.value_counts().iplot(kind='bar',text=['0', '1'], title='Distribution Binary target column',color=['blue'])","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"counts_train = train.target.value_counts(sort=False)\nlabels = counts_train.index\nvalues_train = counts_train.values\n\ndata = go.Pie(labels=labels, values=values_train ,pull=[0.03, 0])\nlayout = go.Layout(title='Comparing Target is binary (1) or not (0) in %')\n\nfig = go.Figure(data=[data], layout=layout)\nfig.update_traces(hole=.3, hoverinfo=\"label+percent+value\")\nfig.update_layout(\n # Add annotations in the center of the donut pies.\n annotations=[dict(text='Train', x=0.5, y=0.5, font_size=20, showarrow=False)])\nfig.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"missing = train.isnull().sum() \nmissing[missing>0].sort_values().iplot(kind='bar',title='Null values present in train Dataset', color=['red'])","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"bin_ = [col for col in train.columns if 'bin_' in col]\nprint(bin_ )\nnom_ = [col for col in train.columns if 'nom_' in col]\nprint(nom_)\nord_ = [col for col in train.columns if 'ord_' in col]\nprint(ord_)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"fig, axes = plt.subplots(nrows=3, ncols=2, figsize=(12,10))\n\nfor ax, column in zip(axes.flatten(), bin_):\n sns.countplot(x = column, ax = ax, data = train)\n\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"fig, axes = plt.subplots(nrows=3, ncols=2, figsize=(12,10))\n\nfor ax, column in zip(axes.flatten(), ord_):\n sns.countplot(x = column, ax = ax, data = train)\n\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"fig, axes = plt.subplots(nrows=5, ncols=2, figsize=(14,12))\n\nfor ax, column in zip(axes.flatten(), nom_):\n sns.countplot(x = column, ax = ax, data = train)\n\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":" \n# 4. Separate continuous, categorical and label column names"},{"metadata":{"trusted":true},"cell_type":"code","source":"cat_cols = [ col for col, dt in train.dtypes.items() if dt == object]\ny_col = ['target']\ncont_cols = [col for col in train.columns if col not in cat_cols + y_col]\n\n\nprint(f'cat_cols has {len(cat_cols)} columns') \nprint(f'cont_cols has {len(cont_cols)} columns') ","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Taking care of missing data in continous\nfrom sklearn.preprocessing import Imputer\n\nimputer = Imputer(strategy = 'most_frequent')\nimputer = imputer.fit(train[cont_cols])\ntrain[cont_cols] = imputer.transform(train[cont_cols])","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"#now for test\nimputer = imputer.fit(test[cont_cols])\ntest[cont_cols] = imputer.transform(test[cont_cols])","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"for cat in cat_cols:\n if train[cat].isnull().sum() > 0:\n train[cat] = train[cat].fillna(train[cat].mode()[0])\n if test[cat].isnull().sum() > 0:\n test[cat] = test[cat].fillna(test[cat].mode()[0])","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# check again for Null values\nmissing = train.isnull().sum() \nmissing[missing>0].sort_values()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"\n## Categorify\nRemember that Pandas offers a category dtype for converting categorical values to numerical codes. Pandas replaces the column values with codes, and retains an index list of category values. In the steps ahead we'll call the categorical values \"names\" and the encodings \"codes\"."},{"metadata":{"trusted":true},"cell_type":"code","source":"# Convert our categorical columns to category dtypes.\nfor cat in cat_cols:\n train[cat] = train[cat].astype('category')\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"train.info()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"for cat in cat_cols:\n test[cat] = test[cat].astype('category')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"train.nom_3.unique()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# this is how cat.codes work, assign a numeric value to each categorie.\ntrain.nom_3.cat.codes.unique()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"pd.DataFrame(train.nom_3.cat.codes.unique(), train.nom_3.unique())\n# Check here Russia is change for 5 values and so on","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# lets make as encoder this train columns\nfor col in cat_cols:\n train[col] = train[col].cat.codes\n ","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"for col in cat_cols:\n test[col] = test[col].cat.codes","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"train.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Number of unique itemns by colum\ntrain.nunique()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"#check correlation in train dataset\ntrain.corr()['target'][:-1].sort_values().plot.bar(figsize=(10,10))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"\n# 5. Train Test Split"},{"metadata":{"trusted":true},"cell_type":"code","source":"\nX = train.drop(['id', 'target'],axis=1).values\ny = train['target'].values","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"from sklearn.model_selection import train_test_split","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"X_train, X_test, y_train, y_test = train_test_split(X,y,test_size=0.30,random_state=42)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"\n# Scaling Data"},{"metadata":{"trusted":true},"cell_type":"code","source":"from sklearn.preprocessing import MinMaxScaler\nscaler = MinMaxScaler()\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"scaler.fit(X_train)\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"X_train = scaler.transform(X_train)\nX_test = scaler.transform(X_test)\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"X_train.max()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"\n# 6. Creating the Model "},{"metadata":{"trusted":true},"cell_type":"code","source":"from sklearn.model_selection import GridSearchCV\nfrom sklearn.linear_model import LogisticRegression","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"#Grid Search\nlogreg = LogisticRegression(class_weight='balanced')\nparameters = {'C':[0.1,5,10]}\nclf_grid = GridSearchCV(logreg,parameters,scoring='roc_auc',refit=True,cv=5, n_jobs=-1)\nclf_grid.fit(X_train, y_train)\nprint('Best roc_auc: {:.4}, with best C: {}'.format(clf_grid.best_score_, clf_grid.best_params_))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Fitting LogisticRegression Classification to the Training set with paramns\n\nclassifier = LogisticRegression(C=5, solver='lbfgs', class_weight='balanced')\nclassifier.fit(X_train, y_train)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Predicting the Test set results\ny_pred = classifier.predict(X_test)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"\n# 7. Model evaluation"},{"metadata":{"trusted":true},"cell_type":"code","source":"from sklearn.metrics import confusion_matrix\nprint( confusion_matrix(y_test, y_pred))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"from sklearn.metrics import classification_report,confusion_matrix\nprint(classification_report(y_test,y_pred))","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"# Showing Confusion Matrix\ndef plot_cm(y_true, y_pred, title, figsize=(5,4)):\n cm = confusion_matrix(y_true, y_pred, labels=np.unique(y_true))\n cm_sum = np.sum(cm, axis=1, keepdims=True)\n cm_perc = cm / cm_sum.astype(float) * 100\n annot = np.empty_like(cm).astype(str)\n nrows, ncols = cm.shape\n for i in range(nrows):\n for j in range(ncols):\n c = cm[i, j]\n p = cm_perc[i, j]\n if i == j:\n s = cm_sum[i]\n annot[i, j] = '%.1f%%\\n%d/%d' % (p, c, s)\n elif c == 0:\n annot[i, j] = ''\n else:\n annot[i, j] = '%.1f%%\\n%d' % (p, c)\n cm = pd.DataFrame(cm, index=np.unique(y_true), columns=np.unique(y_true))\n cm.index.name = 'Actual'\n cm.columns.name = 'Predicted'\n fig, ax = plt.subplots(figsize=figsize)\n plt.title(title)\n sns.heatmap(cm, cmap= \"YlGnBu\", annot=annot, fmt='', ax=ax)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Showing Confusion Matrix\nplot_cm(y_test,y_pred, 'Confution matrix of Tweets', figsize=(5,5))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"test = test.drop(['id'],axis=1).values\nscalert = MinMaxScaler()\nscalert.fit(test)\nr_test = scalert.transform(test)\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"predictions = classifier.predict(r_test)\npredictions","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# sample of submission\nsubmission.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"submission['target'] = predictions ","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"submission.to_csv(\"submission.csv\", index=False, header=True)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"\n# 8. References"},{"metadata":{},"cell_type":"markdown","source":"[Competition](https://www.kaggle.com/c/cat-in-the-dat-ii/overview)"},{"metadata":{},"cell_type":"markdown","source":"

I hope this notebook Usefull for you!

"},{"metadata":{},"cell_type":"markdown","source":"\nGo to TOP"}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":1}