{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"

Random Forest

\n\n\nRandom forests or random decision forests are an ensemble learning method for classification, regression and other tasks that operates by constructing a multitude of decision trees at training time and outputting the class that is the mode of the classes (classification) or mean/average prediction (regression).\n\n\n","metadata":{}},{"cell_type":"markdown","source":"

Data Description

\n\n\n\nIn this competition, you will be predicting the probability [0, 1] of a binary target column.\n\nThe data contains binary features (bin_*), nominal features (nom_*), ordinal features (ord_*) as well as (potentially cyclical) day (of the week) and month features. The string ordinal features ord_{3-5} are lexically ordered according to string.ascii_letters.\n\nSince the purpose of this competition is to explore various encoding strategies. Unlike the first Categorical Feature Encoding Challenge, the data for this challenge has missing values and feature interactions.\n\n### Files\n* train.csv - the training set\n* test.csv - the test set; you must make predictions against this data\n* sample_submission.csv - a sample submission file in the correct format\n\n\n#### Dataset Link :\n[Here](https://www.kaggle.com/c/cat-in-the-dat-ii/data)\n","metadata":{}},{"cell_type":"markdown","source":"Load Required Libraries ","metadata":{}},{"cell_type":"code","source":"import pandas as pd\nimport numpy as np\nfrom sklearn.model_selection import train_test_split\nfrom sklearn.preprocessing import LabelEncoder\nimport matplotlib.pyplot as plt\n%matplotlib inline\nimport seaborn as sns","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Read Data","metadata":{}},{"cell_type":"code","source":"df_train = pd.read_csv('../input/cat-in-the-dat-ii/train.csv')\ndf_test = pd.read_csv('../input/cat-in-the-dat-ii/test.csv')","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_train.head()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_test.head()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_train.info()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_train.describe()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_train.isna().sum()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_test.isna().sum()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Change boolean value to int so as to encode","metadata":{}},{"cell_type":"code","source":"df_train['bin_3'] = df_train['bin_3'].apply(lambda x: 1 if x=='T' else 0)\ndf_train['bin_4'] = df_train['bin_4'].apply(lambda x:1 if x =='Y' else 0)\ndf_test['bin_3'] = df_test['bin_3'].apply(lambda x:1 if x=='T' else 0)\ndf_test['bin_4'] = df_test['bin_4'].apply(lambda x:1 if x == 'Y' else 0)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def replace_nan(data):\n for column in data.columns:\n if data[column].isna().sum() > 0:\n data[column] = data[column].fillna(data[column].mode()[0])\n\n\nreplace_nan(df_train)\nreplace_nan(df_test)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"features = []\n\nfor col in df_train.columns[:-1]:\n rd = LabelEncoder()\n rd.fit_transform( df_train[col].append( df_test[col] ) )\n df_train[col] = rd.transform( df_train[col] )\n df_test [col] = rd.transform( df_test [col] )\n features.append(col)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_train.head()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_train.info()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import pandas_profiling as pp\npp.ProfileReport(df_train)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Split into train and test sets","metadata":{}},{"cell_type":"code","source":"X = df_train.drop('target', axis=1)\nY = df_train['target']\nX_train, X_test, Y_train, Y_test = train_test_split(X, Y, test_size = 0.20, random_state = 0)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print(X_train.shape)\nprint(X_test.shape)\nprint(Y_train.shape)\nprint(Y_test.shape)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Feature Scaling","metadata":{}},{"cell_type":"code","source":"\nfrom sklearn.preprocessing import StandardScaler\nsc = StandardScaler()\nX_train = sc.fit_transform(X_train)\nX_test = sc.transform(X_test)\nX_train","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Random Forest Classifier","metadata":{}},{"cell_type":"code","source":"from sklearn.ensemble import RandomForestClassifier\nforest = RandomForestClassifier(random_state = 0)\nforest.fit(X_train, Y_train)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"from sklearn.metrics import confusion_matrix, accuracy_score\ny_pred = forest.predict(X_test)\ncm = confusion_matrix(Y_test, y_pred)\nprint(cm)\naccuracy_score(Y_test, y_pred)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"conf_matrix = confusion_matrix(y_pred, Y_test)\n\nprint(f'Confussion Matrix: \\n{conf_matrix}\\n')\n\nsns.heatmap(conf_matrix, annot=True)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Performance Measures","metadata":{}},{"cell_type":"code","source":"tn = conf_matrix[0,0]\nfp = conf_matrix[0,1]\ntp = conf_matrix[1,1]\nfn = conf_matrix[1,0]\n\ntotal = tn + fp + tp + fn\nreal_positive = tp + fn\nreal_negative = tn + fp","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"accuracy = (tp + tn) / total # Accuracy Rate\nprecision = tp / (tp + fp) # Positive Predictive Value\nrecall = tp / (tp + fn) # True Positive Rate\nf1score = 2 * precision * recall / (precision + recall)\nspecificity = tn / (tn + fp) # True Negative Rate\nerror_rate = (fp + fn) / total # Missclassification Rate\nprevalence = real_positive / total\nmiss_rate = fn / real_positive # False Negative Rate\nfall_out = fp / real_negative # False Positive Rate\n\nprint(f'Accuracy : {accuracy}')\nprint(f'Precision : {precision}')\nprint(f'Recall : {recall}')\nprint(f'F1 score : {f1score}')\nprint(f'Specificity : {specificity}')\nprint(f'Error Rate : {error_rate}')\nprint(f'Prevalence : {prevalence}')\nprint(f'Miss Rate : {miss_rate}')\nprint(f'Fall Out : {fall_out}')","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Classification Report","metadata":{}},{"cell_type":"code","source":"from sklearn.metrics import classification_report\nprint(classification_report(y_pred, Y_test))","metadata":{"trusted":true},"execution_count":null,"outputs":[]}]}