{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"## Decision Tree\n\n* Decision Tree is a CART model, means Classification and Regression tree. CART model works by recursive partitioning of data in resulting in a local region of defined model. Local region is depicted as a leaf of the decision tree where we have reached to the conclusion of partitioning of the data.\n* To explain the CART approach, a function is used to define the region and the weights which specifies the response value in each region. That function is adaptive basis function. Like in a classification model, a threshold is set for each node to decide the next level in the set of (yes,no). \n* In Regression model, a regression cost is defined, linear regression model is fitted to each node using the inputs as features chosen from the root node and the residual error is then measured. The goal of the tree is here to minimize the residual error. \n* In Classification model, a classification model is defined and in order to maximize the accuracy of the model, there are several common errors caused during the evaluation of the proposed partition. \n1. Entropy or deviance -  It is the measure of the impurity or disorder in the node. It tells that how much a predicted value is deviated from the observed value of a feature. The optimality is reached by minimizing the entropy. The minimizaton of the entropy is equal to the maximization of the information gain. Information gain is the criteria which decides whether a feature is used for splitting the node. The feature which have the highest value of the information gain is used as a feature for splitting the node.\n2. Gini Index - It measures the probability for a random instance being misclassified when chosen randomly. Alternatively, it measures the probability of misclassification rate for every feature. It chooses that feature to split the node whose probability of misclassification rate is minimum.\n* Difference between gini index and entropy - Gini index and entropy, both are used for splitting the node. But, here are some major differences:\n1. Gini index is calculated in the range [0,0.5], while entropy is calculated in the range [0,1].\n2. Entropy is calculated on the basis of logarithms of base 2, while gini index is calculated by subtracting the sum of the squares of the probability of misclassification rate from 1.\n3. Gini index has more clarity in understanding the tree and always predict more accurate than entropy.","metadata":{"papermill":{"duration":0.014958,"end_time":"2023-05-23T12:36:10.061599","exception":false,"start_time":"2023-05-23T12:36:10.046641","status":"completed"},"tags":[]}},{"cell_type":"code","source":"import numpy as np\nimport pandas as pd\nimport seaborn as sns\nimport matplotlib.pyplot as plt\nimport warnings\nwarnings.filterwarnings(\"ignore\")\n%matplotlib inline\n","metadata":{"papermill":{"duration":1.281766,"end_time":"2023-05-23T12:36:11.358545","exception":false,"start_time":"2023-05-23T12:36:10.076779","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-05-24T06:27:23.744045Z","iopub.execute_input":"2023-05-24T06:27:23.744571Z","iopub.status.idle":"2023-05-24T06:27:23.758250Z","shell.execute_reply.started":"2023-05-24T06:27:23.744531Z","shell.execute_reply":"2023-05-24T06:27:23.756016Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df = pd.read_csv(\"/kaggle/input/credit-score-dataset/credit_score_dataset.csv\")\ndf.head()","metadata":{"papermill":{"duration":0.915215,"end_time":"2023-05-23T12:36:12.288726","exception":false,"start_time":"2023-05-23T12:36:11.373511","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-05-24T06:27:23.762117Z","iopub.execute_input":"2023-05-24T06:27:23.762530Z","iopub.status.idle":"2023-05-24T06:27:24.755076Z","shell.execute_reply.started":"2023-05-24T06:27:23.762499Z","shell.execute_reply":"2023-05-24T06:27:24.753308Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Null Data","metadata":{"papermill":{"duration":0.013767,"end_time":"2023-05-23T12:36:12.316931","exception":false,"start_time":"2023-05-23T12:36:12.303164","status":"completed"},"tags":[]}},{"cell_type":"code","source":"df.isnull().sum()","metadata":{"papermill":{"duration":0.102018,"end_time":"2023-05-23T12:36:12.433114","exception":false,"start_time":"2023-05-23T12:36:12.331096","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-05-24T06:27:24.756836Z","iopub.execute_input":"2023-05-24T06:27:24.757279Z","iopub.status.idle":"2023-05-24T06:27:25.038013Z","shell.execute_reply.started":"2023-05-24T06:27:24.757243Z","shell.execute_reply":"2023-05-24T06:27:25.036228Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df.shape","metadata":{"papermill":{"duration":0.025847,"end_time":"2023-05-23T12:36:12.473604","exception":false,"start_time":"2023-05-23T12:36:12.447757","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-05-24T06:27:25.039695Z","iopub.execute_input":"2023-05-24T06:27:25.040677Z","iopub.status.idle":"2023-05-24T06:27:25.047201Z","shell.execute_reply.started":"2023-05-24T06:27:25.040644Z","shell.execute_reply":"2023-05-24T06:27:25.045633Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df.info()","metadata":{"papermill":{"duration":0.117967,"end_time":"2023-05-23T12:36:12.606266","exception":false,"start_time":"2023-05-23T12:36:12.488299","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-05-24T06:27:25.052786Z","iopub.execute_input":"2023-05-24T06:27:25.056663Z","iopub.status.idle":"2023-05-24T06:27:25.357194Z","shell.execute_reply.started":"2023-05-24T06:27:25.056590Z","shell.execute_reply":"2023-05-24T06:27:25.355386Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Selecting null columns, object columns and numerical columns","metadata":{"papermill":{"duration":0.014311,"end_time":"2023-05-23T12:36:12.635408","exception":false,"start_time":"2023-05-23T12:36:12.621097","status":"completed"},"tags":[]}},{"cell_type":"code","source":"nulldata = df.isnull().sum()\nnulldata = nulldata[nulldata>0]\nnullcol = list(nulldata.index)\nnullcol","metadata":{"papermill":{"duration":0.098093,"end_time":"2023-05-23T12:36:12.748178","exception":false,"start_time":"2023-05-23T12:36:12.650085","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-05-24T06:27:25.359088Z","iopub.execute_input":"2023-05-24T06:27:25.360474Z","iopub.status.idle":"2023-05-24T06:27:25.637938Z","shell.execute_reply.started":"2023-05-24T06:27:25.360424Z","shell.execute_reply":"2023-05-24T06:27:25.636987Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"objcol = []\nfor i in df.columns:\n    if df[i].dtype == 'object':\n        objcol.append(i)\n        \nobjcol        ","metadata":{"papermill":{"duration":0.02518,"end_time":"2023-05-23T12:36:12.788381","exception":false,"start_time":"2023-05-23T12:36:12.763201","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-05-24T06:27:25.639704Z","iopub.execute_input":"2023-05-24T06:27:25.640439Z","iopub.status.idle":"2023-05-24T06:27:25.653465Z","shell.execute_reply.started":"2023-05-24T06:27:25.640398Z","shell.execute_reply":"2023-05-24T06:27:25.650809Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"numcol = []\nfor i in df.columns:\n    if df[i].dtype != 'object':\n        numcol.append(i)\n        \nnumcol        ","metadata":{"papermill":{"duration":0.025563,"end_time":"2023-05-23T12:36:12.829111","exception":false,"start_time":"2023-05-23T12:36:12.803548","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-05-24T06:27:25.655713Z","iopub.execute_input":"2023-05-24T06:27:25.656261Z","iopub.status.idle":"2023-05-24T06:27:25.677186Z","shell.execute_reply.started":"2023-05-24T06:27:25.656217Z","shell.execute_reply":"2023-05-24T06:27:25.674455Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"objmatch = []\nfor i in nullcol:\n    for j in objcol:\n        if i==j:\n            objmatch.append(j)\n        \nobjmatch        ","metadata":{"papermill":{"duration":0.02691,"end_time":"2023-05-23T12:36:12.871213","exception":false,"start_time":"2023-05-23T12:36:12.844303","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-05-24T06:27:25.681889Z","iopub.execute_input":"2023-05-24T06:27:25.682503Z","iopub.status.idle":"2023-05-24T06:27:25.700000Z","shell.execute_reply.started":"2023-05-24T06:27:25.682463Z","shell.execute_reply":"2023-05-24T06:27:25.698489Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"nummatch = []\nfor i in nullcol:\n    for j in numcol:\n        if i==j:\n            nummatch.append(j)\n        \nnummatch        ","metadata":{"papermill":{"duration":0.025243,"end_time":"2023-05-23T12:36:12.911637","exception":false,"start_time":"2023-05-23T12:36:12.886394","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-05-24T06:27:25.701590Z","iopub.execute_input":"2023-05-24T06:27:25.702473Z","iopub.status.idle":"2023-05-24T06:27:25.714688Z","shell.execute_reply.started":"2023-05-24T06:27:25.702424Z","shell.execute_reply":"2023-05-24T06:27:25.713740Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Data Cleaning","metadata":{"papermill":{"duration":0.015066,"end_time":"2023-05-23T12:36:12.942068","exception":false,"start_time":"2023-05-23T12:36:12.927002","status":"completed"},"tags":[]}},{"cell_type":"code","source":"df['Num_of_Delayed_Payment'] = df['Num_of_Delayed_Payment'].str.replace('_','')\ndf['Num_of_Delayed_Payment'] = df['Num_of_Delayed_Payment'].astype(float)","metadata":{"papermill":{"duration":0.079583,"end_time":"2023-05-23T12:36:13.037268","exception":false,"start_time":"2023-05-23T12:36:12.957685","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-05-24T06:27:25.717954Z","iopub.execute_input":"2023-05-24T06:27:25.718654Z","iopub.status.idle":"2023-05-24T06:27:25.786578Z","shell.execute_reply.started":"2023-05-24T06:27:25.718626Z","shell.execute_reply":"2023-05-24T06:27:25.785127Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df['Amount_invested_monthly'] = df['Amount_invested_monthly'].str.replace('_','')\ndf['Amount_invested_monthly'] = df['Amount_invested_monthly'].astype(float)","metadata":{"papermill":{"duration":0.105267,"end_time":"2023-05-23T12:36:13.158736","exception":false,"start_time":"2023-05-23T12:36:13.053469","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-05-24T06:27:25.789969Z","iopub.execute_input":"2023-05-24T06:27:25.791672Z","iopub.status.idle":"2023-05-24T06:27:25.859643Z","shell.execute_reply.started":"2023-05-24T06:27:25.791605Z","shell.execute_reply":"2023-05-24T06:27:25.857635Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df['Monthly_Balance'] = df['Monthly_Balance'].str.replace('_','')\ndf['Monthly_Balance'] = df['Monthly_Balance'].astype(float)","metadata":{"papermill":{"duration":0.15191,"end_time":"2023-05-23T12:36:13.325857","exception":false,"start_time":"2023-05-23T12:36:13.173947","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-05-24T06:27:25.865680Z","iopub.execute_input":"2023-05-24T06:27:25.866435Z","iopub.status.idle":"2023-05-24T06:27:25.974774Z","shell.execute_reply.started":"2023-05-24T06:27:25.866376Z","shell.execute_reply":"2023-05-24T06:27:25.972302Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"objcol = set(objcol)-set(['Num_of_Delayed_Payment','Amount_invested_monthly','Monthly_Balance'])\nobjcol = list(objcol)\nobjcol","metadata":{"papermill":{"duration":0.027436,"end_time":"2023-05-23T12:36:13.368723","exception":false,"start_time":"2023-05-23T12:36:13.341287","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-05-24T06:27:25.976900Z","iopub.execute_input":"2023-05-24T06:27:25.977566Z","iopub.status.idle":"2023-05-24T06:27:25.984723Z","shell.execute_reply.started":"2023-05-24T06:27:25.977534Z","shell.execute_reply":"2023-05-24T06:27:25.983146Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"nulldata","metadata":{"papermill":{"duration":0.027756,"end_time":"2023-05-23T12:36:13.413383","exception":false,"start_time":"2023-05-23T12:36:13.385627","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-05-24T06:27:25.986069Z","iopub.execute_input":"2023-05-24T06:27:25.987131Z","iopub.status.idle":"2023-05-24T06:27:26.010274Z","shell.execute_reply.started":"2023-05-24T06:27:25.987066Z","shell.execute_reply":"2023-05-24T06:27:26.007660Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df = df.fillna(method='ffill')\ndf = df.fillna(method = 'bfill')\ndf.isnull().sum()","metadata":{"papermill":{"duration":0.311357,"end_time":"2023-05-23T12:36:13.741030","exception":false,"start_time":"2023-05-23T12:36:13.429673","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-05-24T06:27:26.012303Z","iopub.execute_input":"2023-05-24T06:27:26.012739Z","iopub.status.idle":"2023-05-24T06:27:26.856688Z","shell.execute_reply.started":"2023-05-24T06:27:26.012707Z","shell.execute_reply":"2023-05-24T06:27:26.854713Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df['Outstanding_Debt'] = df['Outstanding_Debt'].str.replace('_','')\ndf['Outstanding_Debt'] = df['Outstanding_Debt'].astype(float)","metadata":{"papermill":{"duration":0.074739,"end_time":"2023-05-23T12:36:13.831834","exception":false,"start_time":"2023-05-23T12:36:13.757095","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-05-24T06:27:26.858634Z","iopub.execute_input":"2023-05-24T06:27:26.859085Z","iopub.status.idle":"2023-05-24T06:27:26.909877Z","shell.execute_reply.started":"2023-05-24T06:27:26.859047Z","shell.execute_reply":"2023-05-24T06:27:26.908079Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df['Annual_Income'] = df['Annual_Income'].str.replace('_','')\ndf['Annual_Income'] = df['Annual_Income'].astype(float)","metadata":{"papermill":{"duration":0.075062,"end_time":"2023-05-23T12:36:13.923657","exception":false,"start_time":"2023-05-23T12:36:13.848595","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-05-24T06:27:26.911750Z","iopub.execute_input":"2023-05-24T06:27:26.912189Z","iopub.status.idle":"2023-05-24T06:27:26.963518Z","shell.execute_reply.started":"2023-05-24T06:27:26.912158Z","shell.execute_reply":"2023-05-24T06:27:26.961550Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df['Age'] = df['Age'].str.replace('_','')\ndf['Age'] = df['Age'].astype(int)","metadata":{"papermill":{"duration":0.074258,"end_time":"2023-05-23T12:36:14.014369","exception":false,"start_time":"2023-05-23T12:36:13.940111","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-05-24T06:27:26.966232Z","iopub.execute_input":"2023-05-24T06:27:26.966673Z","iopub.status.idle":"2023-05-24T06:27:27.018640Z","shell.execute_reply.started":"2023-05-24T06:27:26.966635Z","shell.execute_reply":"2023-05-24T06:27:27.016627Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df['Num_of_Loan'] = df['Num_of_Loan'].str.replace('_','')\ndf['Num_of_Loan'] = df['Num_of_Loan'].astype(int)","metadata":{"papermill":{"duration":0.073492,"end_time":"2023-05-23T12:36:14.104451","exception":false,"start_time":"2023-05-23T12:36:14.030959","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-05-24T06:27:27.020404Z","iopub.execute_input":"2023-05-24T06:27:27.020830Z","iopub.status.idle":"2023-05-24T06:27:27.075640Z","shell.execute_reply.started":"2023-05-24T06:27:27.020795Z","shell.execute_reply":"2023-05-24T06:27:27.073448Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df['Changed_Credit_Limit'] = df['Changed_Credit_Limit'].replace('_',np.nan)\ndf['Changed_Credit_Limit'] = df['Changed_Credit_Limit'].astype(float)","metadata":{"papermill":{"duration":0.039401,"end_time":"2023-05-23T12:36:14.159788","exception":false,"start_time":"2023-05-23T12:36:14.120387","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-05-24T06:27:27.077862Z","iopub.execute_input":"2023-05-24T06:27:27.079055Z","iopub.status.idle":"2023-05-24T06:27:27.100815Z","shell.execute_reply.started":"2023-05-24T06:27:27.078998Z","shell.execute_reply":"2023-05-24T06:27:27.099237Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"col_to_clean = ['Credit_Mix','Occupation','Payment_Behaviour','Payment_of_Min_Amount']\nfor i in col_to_clean:\n    print(f\"{i}\\n{df[i].value_counts()}\\n\\n\")","metadata":{"papermill":{"duration":0.050099,"end_time":"2023-05-23T12:36:14.225388","exception":false,"start_time":"2023-05-23T12:36:14.175289","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-05-24T06:27:27.103548Z","iopub.execute_input":"2023-05-24T06:27:27.103997Z","iopub.status.idle":"2023-05-24T06:27:27.146264Z","shell.execute_reply.started":"2023-05-24T06:27:27.103963Z","shell.execute_reply":"2023-05-24T06:27:27.143742Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df['Credit_Mix'] = df['Credit_Mix'].replace('_',np.nan)\ndf['Occupation'] = df['Occupation'].replace('_______',np.nan)\ndf['Payment_Behaviour'] = df['Payment_Behaviour'].replace('!@9#%8',np.nan)\ndf['Payment_of_Min_Amount'] = df['Payment_of_Min_Amount'].replace('NM','No')","metadata":{"papermill":{"duration":0.05645,"end_time":"2023-05-23T12:36:14.297395","exception":false,"start_time":"2023-05-23T12:36:14.240945","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-05-24T06:27:27.149076Z","iopub.execute_input":"2023-05-24T06:27:27.149757Z","iopub.status.idle":"2023-05-24T06:27:27.177467Z","shell.execute_reply.started":"2023-05-24T06:27:27.149700Z","shell.execute_reply":"2023-05-24T06:27:27.175311Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df.isnull().sum()","metadata":{"papermill":{"duration":0.079478,"end_time":"2023-05-23T12:36:14.392404","exception":false,"start_time":"2023-05-23T12:36:14.312926","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-05-24T06:27:27.179519Z","iopub.execute_input":"2023-05-24T06:27:27.179965Z","iopub.status.idle":"2023-05-24T06:27:27.356187Z","shell.execute_reply.started":"2023-05-24T06:27:27.179921Z","shell.execute_reply":"2023-05-24T06:27:27.355511Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df = df.fillna(method='ffill')\ndf = df.fillna(method = 'bfill')\ndf.isnull().sum()","metadata":{"papermill":{"duration":0.246438,"end_time":"2023-05-23T12:36:14.654927","exception":false,"start_time":"2023-05-23T12:36:14.408489","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-05-24T06:27:27.357234Z","iopub.execute_input":"2023-05-24T06:27:27.357737Z","iopub.status.idle":"2023-05-24T06:27:27.960240Z","shell.execute_reply.started":"2023-05-24T06:27:27.357697Z","shell.execute_reply":"2023-05-24T06:27:27.958357Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"objcol = set(objcol) - set(['ID','Customer_ID','Name','SSN','Credit_History_Age','Outstanding_Debt',\n                            'Annual_Income','Age','Num_of_Loan','Changed_Credit_Limit',\n                            'Credit_Score','Type_of_Loan'])\nobjcol = list(objcol)\nobjcol","metadata":{"papermill":{"duration":0.025671,"end_time":"2023-05-23T12:36:14.696665","exception":false,"start_time":"2023-05-23T12:36:14.670994","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-05-24T06:27:27.961918Z","iopub.execute_input":"2023-05-24T06:27:27.962304Z","iopub.status.idle":"2023-05-24T06:27:27.974941Z","shell.execute_reply.started":"2023-05-24T06:27:27.962273Z","shell.execute_reply":"2023-05-24T06:27:27.973284Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"for i in objcol:\n    print(f\"{i}\\n{df[i].value_counts()}\\n\\n\")","metadata":{"papermill":{"duration":0.055544,"end_time":"2023-05-23T12:36:14.768131","exception":false,"start_time":"2023-05-23T12:36:14.712587","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-05-24T06:27:27.976326Z","iopub.execute_input":"2023-05-24T06:27:27.976664Z","iopub.status.idle":"2023-05-24T06:27:28.025889Z","shell.execute_reply.started":"2023-05-24T06:27:27.976639Z","shell.execute_reply":"2023-05-24T06:27:28.024451Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Data Visualization","metadata":{"papermill":{"duration":0.015864,"end_time":"2023-05-23T12:36:14.800020","exception":false,"start_time":"2023-05-23T12:36:14.784156","status":"completed"},"tags":[]}},{"cell_type":"markdown","source":"#### Occupation","metadata":{"papermill":{"duration":0.015753,"end_time":"2023-05-23T12:36:14.831953","exception":false,"start_time":"2023-05-23T12:36:14.816200","status":"completed"},"tags":[]}},{"cell_type":"code","source":"plt.subplots(figsize = (20,12))\nplt.pie(x = df['Occupation'].value_counts().values,labels = df['Occupation'].value_counts().index, autopct = '%0.02f%%')\nplt.show()","metadata":{"papermill":{"duration":0.325788,"end_time":"2023-05-23T12:36:15.173917","exception":false,"start_time":"2023-05-23T12:36:14.848129","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-05-24T06:27:28.027342Z","iopub.execute_input":"2023-05-24T06:27:28.027771Z","iopub.status.idle":"2023-05-24T06:27:28.329163Z","shell.execute_reply.started":"2023-05-24T06:27:28.027735Z","shell.execute_reply":"2023-05-24T06:27:28.327174Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#### Visualizing the number of accounts with credit scores and credit mixes","metadata":{"papermill":{"duration":0.019551,"end_time":"2023-05-23T12:36:15.212048","exception":false,"start_time":"2023-05-23T12:36:15.192497","status":"completed"},"tags":[]}},{"cell_type":"code","source":"plt.subplots(figsize = (20,12))\nsns.barplot(df, x = df.Credit_Score,y = df.Num_Bank_Accounts, ci = None, hue = df.Credit_Mix)\nplt.show()","metadata":{"papermill":{"duration":0.429204,"end_time":"2023-05-23T12:36:15.659350","exception":false,"start_time":"2023-05-23T12:36:15.230146","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-05-24T06:27:28.331691Z","iopub.execute_input":"2023-05-24T06:27:28.332084Z","iopub.status.idle":"2023-05-24T06:27:28.735358Z","shell.execute_reply.started":"2023-05-24T06:27:28.332055Z","shell.execute_reply":"2023-05-24T06:27:28.734387Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#### Visualizing number of bank accounts have credit scores on the basis of Payment behaviours","metadata":{"papermill":{"duration":0.017757,"end_time":"2023-05-23T12:36:15.695667","exception":false,"start_time":"2023-05-23T12:36:15.677910","status":"completed"},"tags":[]}},{"cell_type":"code","source":"plt.subplots(figsize = (20,12))\nsns.barplot(df, x = df.Credit_Score,y = df.Num_Bank_Accounts, ci = None, hue = df.Payment_Behaviour)\nplt.show()","metadata":{"papermill":{"duration":0.483643,"end_time":"2023-05-23T12:36:16.197337","exception":false,"start_time":"2023-05-23T12:36:15.713694","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-05-24T06:27:28.737053Z","iopub.execute_input":"2023-05-24T06:27:28.737705Z","iopub.status.idle":"2023-05-24T06:27:29.212250Z","shell.execute_reply.started":"2023-05-24T06:27:28.737667Z","shell.execute_reply":"2023-05-24T06:27:29.211413Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#### Visualizing the number of credit cards with payment behaviour on the basis of payment of minimum amount","metadata":{"papermill":{"duration":0.018069,"end_time":"2023-05-23T12:36:16.234177","exception":false,"start_time":"2023-05-23T12:36:16.216108","status":"completed"},"tags":[]}},{"cell_type":"code","source":"plt.subplots(figsize = (20,12))\nsns.barplot(df, x = df.Payment_Behaviour,y = df.Num_Credit_Card, ci = None,hue = df.Payment_of_Min_Amount)\nplt.show()","metadata":{"papermill":{"duration":0.448236,"end_time":"2023-05-23T12:36:16.700843","exception":false,"start_time":"2023-05-23T12:36:16.252607","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-05-24T06:27:29.213644Z","iopub.execute_input":"2023-05-24T06:27:29.214141Z","iopub.status.idle":"2023-05-24T06:27:29.650550Z","shell.execute_reply.started":"2023-05-24T06:27:29.214113Z","shell.execute_reply":"2023-05-24T06:27:29.649720Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Label Encoding","metadata":{"papermill":{"duration":0.020218,"end_time":"2023-05-23T12:36:16.745676","exception":false,"start_time":"2023-05-23T12:36:16.725458","status":"completed"},"tags":[]}},{"cell_type":"code","source":"from sklearn.preprocessing import LabelEncoder\nle = LabelEncoder()\nfor i in objcol:\n    df[i] = le.fit_transform(df[i])\n    print(f\"{i}\\n{le.classes_}\\n\\n\")","metadata":{"papermill":{"duration":0.32834,"end_time":"2023-05-23T12:36:17.093964","exception":false,"start_time":"2023-05-23T12:36:16.765624","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-05-24T06:27:29.651758Z","iopub.execute_input":"2023-05-24T06:27:29.652190Z","iopub.status.idle":"2023-05-24T06:27:29.768575Z","shell.execute_reply.started":"2023-05-24T06:27:29.652165Z","shell.execute_reply":"2023-05-24T06:27:29.766955Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Outlier Detection","metadata":{"papermill":{"duration":0.019837,"end_time":"2023-05-23T12:36:17.135050","exception":false,"start_time":"2023-05-23T12:36:17.115213","status":"completed"},"tags":[]}},{"cell_type":"code","source":"df.plot(kind = 'box',subplots = True, layout = (7,3),figsize = (20,30))\nplt.show()","metadata":{"papermill":{"duration":2.413745,"end_time":"2023-05-23T12:36:19.569791","exception":false,"start_time":"2023-05-23T12:36:17.156046","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-05-24T06:27:29.770659Z","iopub.execute_input":"2023-05-24T06:27:29.771060Z","iopub.status.idle":"2023-05-24T06:27:32.347251Z","shell.execute_reply.started":"2023-05-24T06:27:29.771026Z","shell.execute_reply":"2023-05-24T06:27:32.344550Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plt.subplots(figsize = (20,12))\nsns.heatmap(df.corr(),annot = True)\nplt.show()","metadata":{"papermill":{"duration":1.830153,"end_time":"2023-05-23T12:36:21.422764","exception":false,"start_time":"2023-05-23T12:36:19.592611","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-05-24T06:27:32.349392Z","iopub.execute_input":"2023-05-24T06:27:32.349846Z","iopub.status.idle":"2023-05-24T06:27:33.728432Z","shell.execute_reply.started":"2023-05-24T06:27:32.349807Z","shell.execute_reply":"2023-05-24T06:27:33.726945Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"*  There are very less significant features with good correlations","metadata":{"papermill":{"duration":0.024845,"end_time":"2023-05-23T12:36:21.473608","exception":false,"start_time":"2023-05-23T12:36:21.448763","status":"completed"},"tags":[]}},{"cell_type":"markdown","source":"## Eliminating negative values from Age ","metadata":{"papermill":{"duration":0.024448,"end_time":"2023-05-23T12:36:21.522729","exception":false,"start_time":"2023-05-23T12:36:21.498281","status":"completed"},"tags":[]}},{"cell_type":"code","source":"negval = df[df['Age']<0].Age.index","metadata":{"papermill":{"duration":0.051796,"end_time":"2023-05-23T12:36:21.600565","exception":false,"start_time":"2023-05-23T12:36:21.548769","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-05-24T06:27:33.729778Z","iopub.execute_input":"2023-05-24T06:27:33.730951Z","iopub.status.idle":"2023-05-24T06:27:33.761415Z","shell.execute_reply.started":"2023-05-24T06:27:33.730846Z","shell.execute_reply":"2023-05-24T06:27:33.759999Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df = df.drop(negval,axis=0)","metadata":{"papermill":{"duration":0.046911,"end_time":"2023-05-23T12:36:21.672765","exception":false,"start_time":"2023-05-23T12:36:21.625854","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-05-24T06:27:33.763082Z","iopub.execute_input":"2023-05-24T06:27:33.763800Z","iopub.status.idle":"2023-05-24T06:27:33.786315Z","shell.execute_reply.started":"2023-05-24T06:27:33.763768Z","shell.execute_reply":"2023-05-24T06:27:33.783963Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"* There are many features, to find if all features are important, we will check by feature importance using variance inflation factor.","metadata":{"papermill":{"duration":0.025041,"end_time":"2023-05-23T12:36:21.722866","exception":false,"start_time":"2023-05-23T12:36:21.697825","status":"completed"},"tags":[]}},{"cell_type":"markdown","source":"## Trying Feature Importance","metadata":{"papermill":{"duration":0.024817,"end_time":"2023-05-23T12:36:21.772819","exception":false,"start_time":"2023-05-23T12:36:21.748002","status":"completed"},"tags":[]}},{"cell_type":"code","source":"from statsmodels.stats.outliers_influence import variance_inflation_factor\ncol_list = []\nfor col in df.columns:\n    if ((df[col].dtype != 'object') & (col != 'price')):\n        col_list.append(col)\n        \nX = df[col_list]\nvif_data = pd.DataFrame()\nvif_data['feature'] = X.columns\nvif_data['VIF'] = [variance_inflation_factor(X.values,i)\n                  for i in range(len(X.columns))]\n\nprint(vif_data)","metadata":{"papermill":{"duration":2.85928,"end_time":"2023-05-23T12:36:24.656919","exception":false,"start_time":"2023-05-23T12:36:21.797639","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-05-24T06:27:33.794498Z","iopub.execute_input":"2023-05-24T06:27:33.794913Z","iopub.status.idle":"2023-05-24T06:27:37.955064Z","shell.execute_reply.started":"2023-05-24T06:27:33.794877Z","shell.execute_reply":"2023-05-24T06:27:37.954152Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"* All features are important, so no deduction of features can be done in order to achieve more accuracy of the model.","metadata":{"papermill":{"duration":0.035598,"end_time":"2023-05-23T12:36:24.728503","exception":false,"start_time":"2023-05-23T12:36:24.692905","status":"completed"},"tags":[]}},{"cell_type":"code","source":"df.shape","metadata":{"papermill":{"duration":0.035959,"end_time":"2023-05-23T12:36:24.799829","exception":false,"start_time":"2023-05-23T12:36:24.763870","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-05-24T06:27:37.956733Z","iopub.execute_input":"2023-05-24T06:27:37.957341Z","iopub.status.idle":"2023-05-24T06:27:37.971912Z","shell.execute_reply.started":"2023-05-24T06:27:37.957303Z","shell.execute_reply":"2023-05-24T06:27:37.969231Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"X = df.drop(columns = ['ID','Customer_ID','Name','SSN','Credit_History_Age','Type_of_Loan','Credit_Score'])\ny = df[['Credit_Score']]","metadata":{"papermill":{"duration":0.040343,"end_time":"2023-05-23T12:36:24.865289","exception":false,"start_time":"2023-05-23T12:36:24.824946","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-05-24T06:27:37.974172Z","iopub.execute_input":"2023-05-24T06:27:37.975472Z","iopub.status.idle":"2023-05-24T06:27:37.992153Z","shell.execute_reply.started":"2023-05-24T06:27:37.975428Z","shell.execute_reply":"2023-05-24T06:27:37.989942Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"from sklearn.model_selection import train_test_split\nx_train,x_test,y_train,y_test = train_test_split(X,y,test_size = 0.2,random_state = 99)","metadata":{"papermill":{"duration":0.353238,"end_time":"2023-05-23T12:36:25.243848","exception":false,"start_time":"2023-05-23T12:36:24.890610","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-05-24T06:27:37.994676Z","iopub.execute_input":"2023-05-24T06:27:37.995519Z","iopub.status.idle":"2023-05-24T06:27:38.037541Z","shell.execute_reply.started":"2023-05-24T06:27:37.995460Z","shell.execute_reply":"2023-05-24T06:27:38.036671Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Simple Decision Tree Classification","metadata":{"papermill":{"duration":0.024914,"end_time":"2023-05-23T12:36:25.293974","exception":false,"start_time":"2023-05-23T12:36:25.269060","status":"completed"},"tags":[]}},{"cell_type":"code","source":"from sklearn.tree import DecisionTreeClassifier\nmodel = DecisionTreeClassifier()\nmodel.fit(x_train,y_train)\ny_pred = model.predict(x_test)","metadata":{"papermill":{"duration":2.588929,"end_time":"2023-05-23T12:36:27.908272","exception":false,"start_time":"2023-05-23T12:36:25.319343","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-05-24T06:27:38.039159Z","iopub.execute_input":"2023-05-24T06:27:38.039691Z","iopub.status.idle":"2023-05-24T06:27:40.640067Z","shell.execute_reply.started":"2023-05-24T06:27:38.039662Z","shell.execute_reply":"2023-05-24T06:27:40.639044Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"from sklearn import metrics","metadata":{"papermill":{"duration":0.035977,"end_time":"2023-05-23T12:36:27.969959","exception":false,"start_time":"2023-05-23T12:36:27.933982","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-05-24T06:27:40.642038Z","iopub.execute_input":"2023-05-24T06:27:40.642765Z","iopub.status.idle":"2023-05-24T06:27:40.651440Z","shell.execute_reply.started":"2023-05-24T06:27:40.642736Z","shell.execute_reply":"2023-05-24T06:27:40.648168Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Helper function for evaluation metrics","metadata":{"papermill":{"duration":0.02598,"end_time":"2023-05-23T12:36:28.021084","exception":false,"start_time":"2023-05-23T12:36:27.995104","status":"completed"},"tags":[]}},{"cell_type":"code","source":"def evaluation_metrics(model,y_test,y_pred):\n    print(f\"Test Score = {metrics.accuracy_score(y_test,y_pred)}\\n\")\n    print(f\"Train Score = {model.score(x_train,y_train)}\\n\")\n    print(f\"Classification Report = \\n{metrics.classification_report(y_test,y_pred)}\\n\")\n    cm = metrics.confusion_matrix(y_test,y_pred,labels = model.classes_)\n    disp = metrics.ConfusionMatrixDisplay(confusion_matrix = cm, display_labels = model.classes_)\n    print(\"Confusion Matrix:\\n\")\n    disp.plot()\n    plt.show()","metadata":{"papermill":{"duration":0.036226,"end_time":"2023-05-23T12:36:28.085262","exception":false,"start_time":"2023-05-23T12:36:28.049036","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-05-24T06:27:40.653263Z","iopub.execute_input":"2023-05-24T06:27:40.653643Z","iopub.status.idle":"2023-05-24T06:27:40.677406Z","shell.execute_reply.started":"2023-05-24T06:27:40.653611Z","shell.execute_reply":"2023-05-24T06:27:40.674525Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"evaluation_metrics(model,y_test,y_pred)","metadata":{"papermill":{"duration":0.822421,"end_time":"2023-05-23T12:36:28.934249","exception":false,"start_time":"2023-05-23T12:36:28.111828","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-05-24T06:27:40.679200Z","iopub.execute_input":"2023-05-24T06:27:40.680676Z","iopub.status.idle":"2023-05-24T06:27:41.683905Z","shell.execute_reply.started":"2023-05-24T06:27:40.680614Z","shell.execute_reply":"2023-05-24T06:27:41.682398Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Overfitting\n* The major limitation of using Decision Tree is overfitting which led due to data fragmentation, which means that after allowing multi-way splits, too little data fall in each subtree.\n* To prevent overfitting, we can stop growing the tree by pruning. Pruning is done by growing the full tree and then pruning the branches which gives the least increase in error. To determine the pruning level, evaluation is done by cross validating error on each subtree and pick the tree whose CV error is within 1 standard error of the minimum. This way a simple tree is made by the point with minimum CV error.","metadata":{"papermill":{"duration":0.026542,"end_time":"2023-05-23T12:36:28.987652","exception":false,"start_time":"2023-05-23T12:36:28.961110","status":"completed"},"tags":[]}},{"cell_type":"markdown","source":"## Hyperparameter Tuning\n* Pruning by max depth","metadata":{"papermill":{"duration":0.026812,"end_time":"2023-05-23T12:36:29.041880","exception":false,"start_time":"2023-05-23T12:36:29.015068","status":"completed"},"tags":[]}},{"cell_type":"code","source":"from sklearn.model_selection import KFold\nfrom sklearn.model_selection import GridSearchCV\n\nn_folds = 10\n\nparameters = {'max_depth':range(1,21),\n             'criterion':[\"entropy\",\"gini\"]}\ntree = DecisionTreeClassifier()\ngridSearch = GridSearchCV(estimator = tree, param_grid = parameters,\n                         cv = n_folds, verbose = 1)\ngridSearch.fit(x_train,y_train)","metadata":{"papermill":{"duration":491.98249,"end_time":"2023-05-23T12:44:41.052414","exception":false,"start_time":"2023-05-23T12:36:29.069924","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-05-24T06:27:41.685362Z","iopub.execute_input":"2023-05-24T06:27:41.685655Z","iopub.status.idle":"2023-05-24T06:36:52.481024Z","shell.execute_reply.started":"2023-05-24T06:27:41.685626Z","shell.execute_reply":"2023-05-24T06:36:52.480154Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"scores = gridSearch.cv_results_\nscores = pd.DataFrame(scores)\nscores.head()","metadata":{"papermill":{"duration":0.055885,"end_time":"2023-05-23T12:44:41.135182","exception":false,"start_time":"2023-05-23T12:44:41.079297","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-05-24T06:36:52.482191Z","iopub.execute_input":"2023-05-24T06:36:52.482508Z","iopub.status.idle":"2023-05-24T06:36:52.508594Z","shell.execute_reply.started":"2023-05-24T06:36:52.482481Z","shell.execute_reply":"2023-05-24T06:36:52.507105Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"gridSearch.best_score_","metadata":{"papermill":{"duration":0.036698,"end_time":"2023-05-23T12:44:41.199423","exception":false,"start_time":"2023-05-23T12:44:41.162725","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-05-24T06:36:52.510305Z","iopub.execute_input":"2023-05-24T06:36:52.510654Z","iopub.status.idle":"2023-05-24T06:36:52.519700Z","shell.execute_reply.started":"2023-05-24T06:36:52.510625Z","shell.execute_reply":"2023-05-24T06:36:52.517891Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"gridSearch.best_estimator_","metadata":{"papermill":{"duration":0.039018,"end_time":"2023-05-23T12:44:41.265254","exception":false,"start_time":"2023-05-23T12:44:41.226236","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-05-24T06:36:52.522175Z","iopub.execute_input":"2023-05-24T06:36:52.522564Z","iopub.status.idle":"2023-05-24T06:36:52.539667Z","shell.execute_reply.started":"2023-05-24T06:36:52.522532Z","shell.execute_reply":"2023-05-24T06:36:52.538184Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"best_score = scores[scores['mean_test_score'] == scores['mean_test_score'].max()]\n\nfor i in (best_score.iloc[:,5].values):\n    best_max_depth = i\n    print(best_max_depth)","metadata":{"papermill":{"duration":0.038904,"end_time":"2023-05-23T12:44:41.331495","exception":false,"start_time":"2023-05-23T12:44:41.292591","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-05-24T06:36:52.541374Z","iopub.execute_input":"2023-05-24T06:36:52.541787Z","iopub.status.idle":"2023-05-24T06:36:52.558888Z","shell.execute_reply.started":"2023-05-24T06:36:52.541752Z","shell.execute_reply":"2023-05-24T06:36:52.557424Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"tree_best = DecisionTreeClassifier(max_depth = best_max_depth,criterion = \"gini\")\ntree_best.fit(x_train,y_train)\ny_pred_best = tree_best.predict(x_test)","metadata":{"papermill":{"duration":1.795593,"end_time":"2023-05-23T12:44:43.154144","exception":false,"start_time":"2023-05-23T12:44:41.358551","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-05-24T06:36:52.561853Z","iopub.execute_input":"2023-05-24T06:36:52.562440Z","iopub.status.idle":"2023-05-24T06:36:54.164865Z","shell.execute_reply.started":"2023-05-24T06:36:52.562384Z","shell.execute_reply":"2023-05-24T06:36:54.162809Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"evaluation_metrics(tree_best,y_test,y_pred_best)","metadata":{"papermill":{"duration":0.796761,"end_time":"2023-05-23T12:44:43.978487","exception":false,"start_time":"2023-05-23T12:44:43.181726","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-05-24T06:36:54.167163Z","iopub.execute_input":"2023-05-24T06:36:54.167495Z","iopub.status.idle":"2023-05-24T06:36:55.048315Z","shell.execute_reply.started":"2023-05-24T06:36:54.167468Z","shell.execute_reply":"2023-05-24T06:36:55.047262Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"* Overfitting is eliminated, as there are very small differences between train score and test score as compared to the simple model where train score was much higher than test score","metadata":{"papermill":{"duration":0.029858,"end_time":"2023-05-23T12:44:44.036120","exception":false,"start_time":"2023-05-23T12:44:44.006262","status":"completed"},"tags":[]}},{"cell_type":"code","source":"plt.subplots(figsize = (20,12))\nfrom sklearn.tree import plot_tree\nplot_tree(tree_best,max_depth = 2, feature_names = list(x_train.columns), filled = True, rounded = True)\nplt.show()","metadata":{"papermill":{"duration":0.811446,"end_time":"2023-05-23T12:44:44.879975","exception":false,"start_time":"2023-05-23T12:44:44.068529","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-05-24T06:36:55.049908Z","iopub.execute_input":"2023-05-24T06:36:55.050480Z","iopub.status.idle":"2023-05-24T06:36:55.873658Z","shell.execute_reply.started":"2023-05-24T06:36:55.050444Z","shell.execute_reply":"2023-05-24T06:36:55.872131Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Trying Cost Complexity Pruning","metadata":{"papermill":{"duration":0.03218,"end_time":"2023-05-23T12:44:44.941939","exception":false,"start_time":"2023-05-23T12:44:44.909759","status":"completed"},"tags":[]}},{"cell_type":"code","source":"path = tree_best.cost_complexity_pruning_path(x_train, y_train)\nccp_alphas, impurities = path.ccp_alphas, path.impurities\nccp_alphas = ccp_alphas[:-1]","metadata":{"papermill":{"duration":1.835522,"end_time":"2023-05-23T12:44:46.806789","exception":false,"start_time":"2023-05-23T12:44:44.971267","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-05-24T06:36:55.875041Z","iopub.execute_input":"2023-05-24T06:36:55.875439Z","iopub.status.idle":"2023-05-24T06:36:57.593809Z","shell.execute_reply.started":"2023-05-24T06:36:55.875392Z","shell.execute_reply":"2023-05-24T06:36:57.591347Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"len(ccp_alphas)","metadata":{"papermill":{"duration":0.043931,"end_time":"2023-05-23T12:44:46.881587","exception":false,"start_time":"2023-05-23T12:44:46.837656","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-05-24T06:36:57.596709Z","iopub.execute_input":"2023-05-24T06:36:57.597184Z","iopub.status.idle":"2023-05-24T06:36:57.603403Z","shell.execute_reply.started":"2023-05-24T06:36:57.597149Z","shell.execute_reply":"2023-05-24T06:36:57.602610Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print(f\"\"\"The length of ccp_alphas are very high. \nFor post pruning, we have to find that ccp_alpha which gave us train \nand test score approximately equal or near.\nIt is very time taking to train {len(ccp_alphas)} models \nand then compare the scores of the model. So, Post pruning is not performed here.\"\"\")","metadata":{"papermill":{"duration":0.041952,"end_time":"2023-05-23T12:44:46.953396","exception":false,"start_time":"2023-05-23T12:44:46.911444","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-05-24T06:36:57.604204Z","iopub.execute_input":"2023-05-24T06:36:57.604485Z","iopub.status.idle":"2023-05-24T06:36:57.623243Z","shell.execute_reply.started":"2023-05-24T06:36:57.604460Z","shell.execute_reply":"2023-05-24T06:36:57.621890Z"},"trusted":true},"execution_count":null,"outputs":[]}]}