{"metadata":{"kaggle":{"accelerator":"none","dataSources":[{"sourceId":63054,"databundleVersionId":6872631,"sourceType":"competition"}],"dockerImageVersionId":30587,"isInternetEnabled":true,"language":"python","sourceType":"notebook","isGpuEnabled":false},"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"codemirror_mode":{"name":"ipython","version":3},"file_extension":".py","mimetype":"text/x-python","name":"python","nbconvert_exporter":"python","pygments_lexer":"ipython3","version":"3.10.12"},"papermill":{"default_parameters":{},"duration":6943.019165,"end_time":"2023-11-21T16:52:36.065928","environment_variables":{},"exception":null,"input_path":"__notebook__.ipynb","output_path":"__notebook__.ipynb","parameters":{},"start_time":"2023-11-21T14:56:53.046763","version":"2.4.0"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"<a id=\"table\"></a>\n<h1 style=\"background-color:#0F0F0F;font-family:newtimeroman;font-size:350%;text-align:center;border-radius: 15px 50px;color:#F0F0F0\">Table of Contents</h1>\n\n\n- [Libraries and Dependencies](#1)\n\n- [Data Description](#2)\n\n- [Data Loading and Preprocessing](#3)\n\n- [Exploratory Data Analysis (EDA)](#4)\n\n- [Cross Validation Set](#5)\n  - [Training](#5.1)\n    - [RandomForestClassifier](#5.1.1)\n    - [XGBoost](#5.1.2)\n    - [Light GBM](#5.1.3)\n    - [DecisionTreeClassifier](#5.1.4)\n    - [Logistic Regression](#5.1.5)\n  - [Scores](#5.2)\n  \n  \n- [Model Training](#6)\n  - [RandomForestClassifier](#6.1)\n  - [XGBoost](#6.2)\n  - [Light GBM](#6.3)\n  - [DecisionTreeClassifier](#6.4)\n  - [Logistic Regression](#6.5)\n  \n- [Predictions](#7)\n\n- [Submissions](#8)\n\n<a id=\"1\"></a>\n<h1 style=\"background-color:#0F0F0F;font-family:newtimeroman;font-size:350%;text-align:center;border-radius: 15px 50px;color:#F0F0F0\">Libraries and Dependencies</h1>","metadata":{"papermill":{"duration":0.016583,"end_time":"2023-11-21T14:56:56.435182","exception":false,"start_time":"2023-11-21T14:56:56.418599","status":"completed"},"tags":[]}},{"cell_type":"code","source":"import numpy as np  # linear algebra\n\n# Importing methods for preprocessing and data handling\nimport pandas as pd\nfrom imblearn.over_sampling import SMOTE\nfrom sklearn.metrics import confusion_matrix,accuracy_score,roc_auc_score\nfrom sklearn.model_selection import train_test_split\n\n# Importing libraries for data visualization\nimport seaborn as sns\nimport matplotlib.pyplot as plt\n\n# Importing Models\nimport lightgbm as lgb\nimport xgboost\nfrom sklearn.ensemble import RandomForestClassifier\nfrom sklearn.tree import DecisionTreeClassifier\nfrom sklearn.linear_model import LogisticRegression","metadata":{"papermill":{"duration":5.616048,"end_time":"2023-11-21T14:57:02.067318","exception":false,"start_time":"2023-11-21T14:56:56.451270","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-11-26T14:16:35.687182Z","iopub.execute_input":"2023-11-26T14:16:35.687506Z","iopub.status.idle":"2023-11-26T14:16:38.748627Z","shell.execute_reply.started":"2023-11-26T14:16:35.687479Z","shell.execute_reply":"2023-11-26T14:16:38.747203Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<a id=\"2\"></a>\n<h1 style=\"background-color:#0F0F0F;font-family:newtimeroman;font-size:350%;text-align:center;border-radius: 15px 50px;color:#F0F0F0\">Data Description</h1>\n","metadata":{}},{"cell_type":"markdown","source":"\nColumns\n\n* **id**: Unique identifier for each entry.\n* **trans_date_trans_time**: The date and time of the transaction.\n* **cc_num**: Credit card number.\n* **merchant**: Merchant receiving payment.\n* **category**: Area of merchant's business.\n* **amt**: Amount in American Dollars.\n* **first**: First name of the cardholder.\n* **last**: Last name of the cardholder.\n* **gender**: Gender of the cardholder (Male/Female).\n* **street**: Street of cardholder's residence.\n* **city**: City of cardholder's residence.\n* **state**: State of cardholder's residence.\n* **zip**: ZIP code of cardholder's residence.\n* **lat**: Latitude of the cardholder.\n* **long**: Longitude of the cardholder.\n* **city_pop**: Population of the city.\n* **job**: Occupation of the cardholder.\n* **dob**: Date of birth of the cardholder.\n* **trans_num**: Transaction ID.\n* **unix_time**: Unix time (time since 1970).\n* **merch_lat**: Latitude of the merchant.\n* **merch_long**: Longitude of the merchant.\n* **is_fraud**: Binary flag indicating whether the transaction is fraud (1) or not (0).","metadata":{"papermill":{"duration":0.016274,"end_time":"2023-11-21T14:57:02.101438","exception":false,"start_time":"2023-11-21T14:57:02.085164","status":"completed"},"tags":[]}},{"cell_type":"markdown","source":"<a id=\"3\"></a>\n<h1 style=\"background-color:#0F0F0F;font-family:newtimeroman;font-size:350%;text-align:center;border-radius: 15px 50px;color:#F0F0F0\">Data Loading and Preprocessing</h1>","metadata":{"papermill":{"duration":0.015966,"end_time":"2023-11-21T14:57:02.133646","exception":false,"start_time":"2023-11-21T14:57:02.117680","status":"completed"},"tags":[]}},{"cell_type":"markdown","source":"**Loading Data**","metadata":{"papermill":{"duration":0.016012,"end_time":"2023-11-21T14:57:02.165758","exception":false,"start_time":"2023-11-21T14:57:02.149746","status":"completed"},"tags":[]}},{"cell_type":"code","source":"df = pd.read_csv(\"/kaggle/input/anyas-gojo-revival-bondman/trainData.csv\")\ndftest=pd.read_csv(\"/kaggle/input/anyas-gojo-revival-bondman/testData.csv\")","metadata":{"_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","papermill":{"duration":10.113129,"end_time":"2023-11-21T14:57:12.295057","exception":false,"start_time":"2023-11-21T14:57:02.181928","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-11-26T14:16:38.750813Z","iopub.execute_input":"2023-11-26T14:16:38.751113Z","iopub.status.idle":"2023-11-26T14:16:46.912617Z","shell.execute_reply.started":"2023-11-26T14:16:38.751089Z","shell.execute_reply":"2023-11-26T14:16:46.911772Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**Preprocessing Training Data**","metadata":{"papermill":{"duration":0.015989,"end_time":"2023-11-21T14:57:12.327965","exception":false,"start_time":"2023-11-21T14:57:12.311976","status":"completed"},"tags":[]}},{"cell_type":"markdown","source":"We will make copy datasets to perform furthur operations","metadata":{"papermill":{"duration":0.015865,"end_time":"2023-11-21T14:57:12.359857","exception":false,"start_time":"2023-11-21T14:57:12.343992","status":"completed"},"tags":[]}},{"cell_type":"code","source":"dfm = df.copy()\ndftestf=dftest.copy()","metadata":{"papermill":{"duration":0.179032,"end_time":"2023-11-21T14:57:12.554963","exception":false,"start_time":"2023-11-21T14:57:12.375931","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-11-26T14:16:46.913547Z","iopub.execute_input":"2023-11-26T14:16:46.914078Z","iopub.status.idle":"2023-11-26T14:16:47.002302Z","shell.execute_reply.started":"2023-11-26T14:16:46.914050Z","shell.execute_reply":"2023-11-26T14:16:47.000763Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"dfm.shape","metadata":{"papermill":{"duration":0.031643,"end_time":"2023-11-21T14:57:12.608923","exception":false,"start_time":"2023-11-21T14:57:12.577280","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-11-26T14:16:47.004463Z","iopub.execute_input":"2023-11-26T14:16:47.005461Z","iopub.status.idle":"2023-11-26T14:16:47.012461Z","shell.execute_reply.started":"2023-11-26T14:16:47.005430Z","shell.execute_reply":"2023-11-26T14:16:47.010936Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"dfm.head()","metadata":{"papermill":{"duration":0.052395,"end_time":"2023-11-21T14:57:12.677920","exception":false,"start_time":"2023-11-21T14:57:12.625525","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-11-26T14:16:47.014667Z","iopub.execute_input":"2023-11-26T14:16:47.015215Z","iopub.status.idle":"2023-11-26T14:16:47.062071Z","shell.execute_reply.started":"2023-11-26T14:16:47.015175Z","shell.execute_reply":"2023-11-26T14:16:47.060653Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"dfm.info()","metadata":{"papermill":{"duration":0.634931,"end_time":"2023-11-21T14:57:13.337357","exception":false,"start_time":"2023-11-21T14:57:12.702426","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-11-26T14:16:47.063400Z","iopub.execute_input":"2023-11-26T14:16:47.063723Z","iopub.status.idle":"2023-11-26T14:16:47.280500Z","shell.execute_reply.started":"2023-11-26T14:16:47.063696Z","shell.execute_reply":"2023-11-26T14:16:47.278771Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"null_entries = dfm.isnull().sum()\nprint(\"Null entries in the entire DataFrame:\")\nprint(null_entries)","metadata":{"papermill":{"duration":0.614521,"end_time":"2023-11-21T14:57:13.977295","exception":false,"start_time":"2023-11-21T14:57:13.362774","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-11-26T14:16:47.282721Z","iopub.execute_input":"2023-11-26T14:16:47.284034Z","iopub.status.idle":"2023-11-26T14:16:47.477544Z","shell.execute_reply.started":"2023-11-26T14:16:47.283989Z","shell.execute_reply":"2023-11-26T14:16:47.475909Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"There are no null values in the train dataset\n","metadata":{"papermill":{"duration":0.018049,"end_time":"2023-11-21T14:57:14.013419","exception":false,"start_time":"2023-11-21T14:57:13.995370","status":"completed"},"tags":[]}},{"cell_type":"code","source":"dfm.describe()","metadata":{"papermill":{"duration":0.225441,"end_time":"2023-11-21T14:57:14.256481","exception":false,"start_time":"2023-11-21T14:57:14.031040","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-11-26T14:16:47.479181Z","iopub.execute_input":"2023-11-26T14:16:47.479636Z","iopub.status.idle":"2023-11-26T14:16:47.671200Z","shell.execute_reply.started":"2023-11-26T14:16:47.479599Z","shell.execute_reply":"2023-11-26T14:16:47.669682Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"dfm.columns","metadata":{"papermill":{"duration":0.026516,"end_time":"2023-11-21T14:57:14.301113","exception":false,"start_time":"2023-11-21T14:57:14.274597","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-11-26T14:16:47.672604Z","iopub.execute_input":"2023-11-26T14:16:47.673255Z","iopub.status.idle":"2023-11-26T14:16:47.681078Z","shell.execute_reply.started":"2023-11-26T14:16:47.673227Z","shell.execute_reply":"2023-11-26T14:16:47.678952Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<a id=\"4\"></a>\n<h1 style=\"background-color:#0F0F0F;font-family:newtimeroman;font-size:350%;text-align:center;border-radius: 15px 50px;color:#F0F0F0\">Exploratory Data Analysis</h1>","metadata":{"papermill":{"duration":0.017891,"end_time":"2023-11-21T14:57:14.336726","exception":false,"start_time":"2023-11-21T14:57:14.318835","status":"completed"},"tags":[]}},{"cell_type":"markdown","source":"Using a single 'name' column is more beneficial from a feature engineering perspective than having separate 'first name' and 'last name' columns.\n\n","metadata":{"papermill":{"duration":0.018361,"end_time":"2023-11-21T14:57:14.373957","exception":false,"start_time":"2023-11-21T14:57:14.355596","status":"completed"},"tags":[]}},{"cell_type":"code","source":"dfm['name'] = dfm['first'] + ' ' + dfm['last']","metadata":{"papermill":{"duration":0.157921,"end_time":"2023-11-21T14:57:14.549927","exception":false,"start_time":"2023-11-21T14:57:14.392006","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-11-26T14:16:47.686044Z","iopub.execute_input":"2023-11-26T14:16:47.686457Z","iopub.status.idle":"2023-11-26T14:16:47.776243Z","shell.execute_reply.started":"2023-11-26T14:16:47.686426Z","shell.execute_reply":"2023-11-26T14:16:47.774832Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"I am removing the 'trans_num' column from the dataset since it contains entirely unique values, making it an unsuitable parameter for training.","metadata":{"execution":{"iopub.execute_input":"2023-11-13T21:04:55.396038Z","iopub.status.busy":"2023-11-13T21:04:55.395642Z","iopub.status.idle":"2023-11-13T21:04:55.404384Z","shell.execute_reply":"2023-11-13T21:04:55.402358Z","shell.execute_reply.started":"2023-11-13T21:04:55.396008Z"},"papermill":{"duration":0.017767,"end_time":"2023-11-21T14:57:14.586265","exception":false,"start_time":"2023-11-21T14:57:14.568498","status":"completed"},"tags":[]}},{"cell_type":"markdown","source":"'first' & 'last' colums are dropped since I have included complete name in 'name' column.","metadata":{"papermill":{"duration":0.017738,"end_time":"2023-11-21T14:57:14.622653","exception":false,"start_time":"2023-11-21T14:57:14.604915","status":"completed"},"tags":[]}},{"cell_type":"code","source":"dfm = dfm.drop(['trans_num','first', 'last'], axis=1)","metadata":{"papermill":{"duration":0.092767,"end_time":"2023-11-21T14:57:14.733357","exception":false,"start_time":"2023-11-21T14:57:14.640590","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-11-26T14:16:47.777308Z","iopub.execute_input":"2023-11-26T14:16:47.777673Z","iopub.status.idle":"2023-11-26T14:16:47.851506Z","shell.execute_reply.started":"2023-11-26T14:16:47.777646Z","shell.execute_reply":"2023-11-26T14:16:47.850424Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"dfm['is_fraud'].value_counts(normalize=True)","metadata":{"papermill":{"duration":0.0356,"end_time":"2023-11-21T14:57:14.787857","exception":false,"start_time":"2023-11-21T14:57:14.752257","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-11-26T14:16:47.852746Z","iopub.execute_input":"2023-11-26T14:16:47.853182Z","iopub.status.idle":"2023-11-26T14:16:47.867265Z","shell.execute_reply.started":"2023-11-26T14:16:47.853155Z","shell.execute_reply":"2023-11-26T14:16:47.865660Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Approximately out of every 69 transactions 1 is fraud.\n\n","metadata":{"papermill":{"duration":0.018215,"end_time":"2023-11-21T14:57:14.824396","exception":false,"start_time":"2023-11-21T14:57:14.806181","status":"completed"},"tags":[]}},{"cell_type":"code","source":"dfm['amt'].value_counts()\n#amtcount=dfm['amt'].value_counts()\n#pd.set_option('display.max_rows', None)  \n#print(amtcount)","metadata":{"papermill":{"duration":0.045328,"end_time":"2023-11-21T14:57:14.888494","exception":false,"start_time":"2023-11-21T14:57:14.843166","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-11-26T14:16:47.869094Z","iopub.execute_input":"2023-11-26T14:16:47.869529Z","iopub.status.idle":"2023-11-26T14:16:47.908825Z","shell.execute_reply.started":"2023-11-26T14:16:47.869491Z","shell.execute_reply":"2023-11-26T14:16:47.906729Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"The 'amt' column seems skewed. We will furthur check it through distplot","metadata":{"papermill":{"duration":0.018458,"end_time":"2023-11-21T14:57:14.925694","exception":false,"start_time":"2023-11-21T14:57:14.907236","status":"completed"},"tags":[]}},{"cell_type":"code","source":"sns.distplot(dfm['amt'])\nplt.title(\"Distribution of 'amt' Column\")\nplt.show()","metadata":{"papermill":{"duration":2.430257,"end_time":"2023-11-21T14:57:17.374339","exception":false,"start_time":"2023-11-21T14:57:14.944082","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-11-26T14:16:47.910859Z","iopub.execute_input":"2023-11-26T14:16:47.911460Z","iopub.status.idle":"2023-11-26T14:16:50.790961Z","shell.execute_reply.started":"2023-11-26T14:16:47.911415Z","shell.execute_reply":"2023-11-26T14:16:50.789899Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"It is highly skewed so we need to tranform it","metadata":{"execution":{"iopub.execute_input":"2023-11-13T21:10:41.283927Z","iopub.status.busy":"2023-11-13T21:10:41.283528Z","iopub.status.idle":"2023-11-13T21:10:41.290841Z","shell.execute_reply":"2023-11-13T21:10:41.289467Z","shell.execute_reply.started":"2023-11-13T21:10:41.283897Z"},"papermill":{"duration":0.019895,"end_time":"2023-11-21T14:57:17.414950","exception":false,"start_time":"2023-11-21T14:57:17.395055","status":"completed"},"tags":[]}},{"cell_type":"code","source":"dfm['amt'] = np.log1p(dfm['amt'])","metadata":{"papermill":{"duration":0.028832,"end_time":"2023-11-21T14:57:17.462850","exception":false,"start_time":"2023-11-21T14:57:17.434018","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-11-26T14:16:50.791957Z","iopub.execute_input":"2023-11-26T14:16:50.792194Z","iopub.status.idle":"2023-11-26T14:16:50.805320Z","shell.execute_reply.started":"2023-11-26T14:16:50.792172Z","shell.execute_reply":"2023-11-26T14:16:50.803875Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"sns.distplot(dfm['amt'])\nplt.title(\"Distribution of 'amt' Column after log transform\")\nplt.show()","metadata":{"papermill":{"duration":2.34806,"end_time":"2023-11-21T14:57:19.830091","exception":false,"start_time":"2023-11-21T14:57:17.482031","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-11-26T14:16:50.806861Z","iopub.execute_input":"2023-11-26T14:16:50.807233Z","iopub.status.idle":"2023-11-26T14:16:53.030400Z","shell.execute_reply.started":"2023-11-26T14:16:50.807200Z","shell.execute_reply":"2023-11-26T14:16:53.029141Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"dfm[\"gender\"].value_counts(normalize=True) #normalize=true returns the relative frequencies of each unique value","metadata":{"papermill":{"duration":0.086385,"end_time":"2023-11-21T14:57:19.937052","exception":false,"start_time":"2023-11-21T14:57:19.850667","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-11-26T14:16:53.032079Z","iopub.execute_input":"2023-11-26T14:16:53.032390Z","iopub.status.idle":"2023-11-26T14:16:53.062027Z","shell.execute_reply.started":"2023-11-26T14:16:53.032363Z","shell.execute_reply":"2023-11-26T14:16:53.060525Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plt.figure(figsize=(5,5))\nax = sns.histplot(x='gender', data=dfm, hue='is_fraud', stat='percent', multiple='dodge', common_norm=False)\n#dodge helps us create distinct bars for each category within the 'gender' variable\nax.set_ylabel('Percentage')\nax.set_xlabel('Credit Card Holder Gender')\nplt.legend(title='Type', labels=['Not Fraud', 'Fraud'])\nplt.show()","metadata":{"papermill":{"duration":1.113707,"end_time":"2023-11-21T14:57:21.072256","exception":false,"start_time":"2023-11-21T14:57:19.958549","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-11-26T14:16:53.063693Z","iopub.execute_input":"2023-11-26T14:16:53.064633Z","iopub.status.idle":"2023-11-26T14:16:53.721379Z","shell.execute_reply.started":"2023-11-26T14:16:53.064566Z","shell.execute_reply":"2023-11-26T14:16:53.720025Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Since both male and female have good contribution in both fraud and not fraud cases so 'gender' is not a good feature for training data.","metadata":{"execution":{"iopub.execute_input":"2023-11-13T21:15:25.988256Z","iopub.status.busy":"2023-11-13T21:15:25.987663Z","iopub.status.idle":"2023-11-13T21:15:25.995224Z","shell.execute_reply":"2023-11-13T21:15:25.993896Z","shell.execute_reply.started":"2023-11-13T21:15:25.988223Z"},"papermill":{"duration":0.021616,"end_time":"2023-11-21T14:57:21.115429","exception":false,"start_time":"2023-11-21T14:57:21.093813","status":"completed"},"tags":[]}},{"cell_type":"code","source":"plt.hist(dfm[\"category\"], color=\"red\", edgecolor=\"black\")\nplt.xticks(rotation=60)\nplt.title(\"Category\")\nplt.show()","metadata":{"papermill":{"duration":0.452334,"end_time":"2023-11-21T14:57:21.588454","exception":false,"start_time":"2023-11-21T14:57:21.136120","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-11-26T14:16:53.727010Z","iopub.execute_input":"2023-11-26T14:16:53.728218Z","iopub.status.idle":"2023-11-26T14:16:54.106242Z","shell.execute_reply.started":"2023-11-26T14:16:53.728165Z","shell.execute_reply":"2023-11-26T14:16:54.104884Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plt.figure(figsize=(16, 8))\nplt.hist((dfm[\"state\"]), bins=100)\nplt.show()","metadata":{"papermill":{"duration":0.745933,"end_time":"2023-11-21T14:57:22.356497","exception":false,"start_time":"2023-11-21T14:57:21.610564","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-11-26T14:16:54.107603Z","iopub.execute_input":"2023-11-26T14:16:54.107895Z","iopub.status.idle":"2023-11-26T14:16:54.686484Z","shell.execute_reply.started":"2023-11-26T14:16:54.107868Z","shell.execute_reply":"2023-11-26T14:16:54.685626Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"dfm['trans_date_trans_time']=pd.to_datetime(dfm['trans_date_trans_time'])#converting it from object type to date-time\ndfm['trans_date']=dfm['trans_date_trans_time'].dt.strftime('%Y-%m-%d')#store date of transaction in a separate column\n","metadata":{"papermill":{"duration":0.717517,"end_time":"2023-11-21T14:57:23.096103","exception":false,"start_time":"2023-11-21T14:57:22.378586","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-11-26T14:16:54.689343Z","iopub.execute_input":"2023-11-26T14:16:54.689766Z","iopub.status.idle":"2023-11-26T14:16:55.160011Z","shell.execute_reply.started":"2023-11-26T14:16:54.689739Z","shell.execute_reply":"2023-11-26T14:16:55.158948Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Date of fraud transaction can be a important feature while training  so I extracted it from 'trans_date_trans_time' column and created a new column 'trans_date' to store date of transaction.","metadata":{"papermill":{"duration":0.021861,"end_time":"2023-11-21T14:57:23.141976","exception":false,"start_time":"2023-11-21T14:57:23.120115","status":"completed"},"tags":[]}},{"cell_type":"code","source":"dfm['dob']=pd.to_datetime(dfm['dob'])\ndfm['trans_date']=pd.to_datetime(dfm['trans_date'])#This conversion will help me to perform date-related operations easily.","metadata":{"papermill":{"duration":0.28905,"end_time":"2023-11-21T14:57:23.452895","exception":false,"start_time":"2023-11-21T14:57:23.163845","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-11-26T14:16:55.161672Z","iopub.execute_input":"2023-11-26T14:16:55.162379Z","iopub.status.idle":"2023-11-26T14:16:55.355957Z","shell.execute_reply.started":"2023-11-26T14:16:55.162337Z","shell.execute_reply":"2023-11-26T14:16:55.354942Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"dfm['age']=dfm['trans_date'].dt.year-dfm['dob'].dt.year","metadata":{"papermill":{"duration":0.073841,"end_time":"2023-11-21T14:57:23.549548","exception":false,"start_time":"2023-11-21T14:57:23.475707","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-11-26T14:16:55.356982Z","iopub.execute_input":"2023-11-26T14:16:55.357369Z","iopub.status.idle":"2023-11-26T14:16:55.416056Z","shell.execute_reply.started":"2023-11-26T14:16:55.357335Z","shell.execute_reply":"2023-11-26T14:16:55.415305Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"I created a new column 'age' where I am storing the age of the person while performing the transaction as this might be a good feature from feature engineering point of view.","metadata":{"papermill":{"duration":0.02555,"end_time":"2023-11-21T14:57:23.601399","exception":false,"start_time":"2023-11-21T14:57:23.575849","status":"completed"},"tags":[]}},{"cell_type":"code","source":"ageplot=sns.kdeplot(x='age',data=dfm, hue='is_fraud', common_norm=False)\nplt.title('Age of people doing transaction with hue on basis of \"is_fraud\" column')\nplt.legend(title='Fraud', labels=['Yes', 'No'])\nageplot.set_xlabel('Age while performing transaction')\nageplot.set_ylabel('Density')","metadata":{"papermill":{"duration":2.308828,"end_time":"2023-11-21T14:57:25.936334","exception":false,"start_time":"2023-11-21T14:57:23.627506","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-11-26T14:16:55.417690Z","iopub.execute_input":"2023-11-26T14:16:55.418072Z","iopub.status.idle":"2023-11-26T14:16:57.422691Z","shell.execute_reply.started":"2023-11-26T14:16:55.418040Z","shell.execute_reply":"2023-11-26T14:16:57.421718Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"dfm['hour']=dfm['trans_date_trans_time'].dt.hour #separate column for hour of transaction\nHourplot=sns.histplot(data=dfm, x=\"hour\", hue=\"is_fraud\", common_norm=False,stat='percent',multiple='dodge')\nplt.xticks(np.arange(0,24,1))\nplt.legend(title='Fraud', labels=['Yes', 'No'])\nHourplot.set_ylabel('Percentage')\nHourplot.set_xlabel('Hour')","metadata":{"papermill":{"duration":1.140378,"end_time":"2023-11-21T14:57:27.100550","exception":false,"start_time":"2023-11-21T14:57:25.960172","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-11-26T14:16:57.423995Z","iopub.execute_input":"2023-11-26T14:16:57.424253Z","iopub.status.idle":"2023-11-26T14:16:58.399353Z","shell.execute_reply.started":"2023-11-26T14:16:57.424230Z","shell.execute_reply":"2023-11-26T14:16:58.398228Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"We can see there is huge increase in fraud transaction from 2200hrs to 0300hrs.","metadata":{"papermill":{"duration":0.024699,"end_time":"2023-11-21T14:57:27.149495","exception":false,"start_time":"2023-11-21T14:57:27.124796","status":"completed"},"tags":[]}},{"cell_type":"code","source":"dfm['day']=dfm['trans_date_trans_time'].dt.dayofweek\ndayplot=sns.histplot(data=dfm, x=\"day\", hue=\"is_fraud\", common_norm=False,stat='percent',multiple='dodge')\ndayplot.set_xticklabels(['',\"Monday\",\"Tuesday\",\"Wednesday\",\"Thursday\",\"Friday\",\"Saturday\",\"Sunday\"])\ndayplot.set_ylabel('Percentage')\ndayplot.set_xlabel('Days of Week')\nplt.legend(title='Fraud', labels=['Yes', 'No'])","metadata":{"papermill":{"duration":0.901125,"end_time":"2023-11-21T14:57:28.074295","exception":false,"start_time":"2023-11-21T14:57:27.173170","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-11-26T14:16:58.403376Z","iopub.execute_input":"2023-11-26T14:16:58.403715Z","iopub.status.idle":"2023-11-26T14:16:59.065420Z","shell.execute_reply.started":"2023-11-26T14:16:58.403688Z","shell.execute_reply":"2023-11-26T14:16:59.063850Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Slight spike is seen in fraud transactions around weekends.","metadata":{"papermill":{"duration":0.024553,"end_time":"2023-11-21T14:57:28.123832","exception":false,"start_time":"2023-11-21T14:57:28.099279","status":"completed"},"tags":[]}},{"cell_type":"markdown","source":"Similar analysis I will do for month of fraud transaction.","metadata":{"papermill":{"duration":0.024456,"end_time":"2023-11-21T14:57:28.173742","exception":false,"start_time":"2023-11-21T14:57:28.149286","status":"completed"},"tags":[]}},{"cell_type":"code","source":"dfm['month']=dfm['trans_date_trans_time'].dt.month\nmplot=sns.histplot(data=dfm, x=\"month\", hue=\"is_fraud\", common_norm=False,stat='percent',multiple='dodge')\nmplot.set_xlabel('Month')\nplt.xticks(np.arange(1,13,1))\nmplot.set_ylabel('Percentage')\nmplot.set_xticklabels([\"Jan\",\"Feb\",\"Mar\",\"Apr\",\"May\",\"Jun\",\"Jul\",'Aug','Sep','Oct','Nov','Dec'])\nplt.legend(title='Fraud', labels=['Yes', 'No'])","metadata":{"papermill":{"duration":0.968124,"end_time":"2023-11-21T14:57:29.165945","exception":false,"start_time":"2023-11-21T14:57:28.197821","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-11-26T14:16:59.067370Z","iopub.execute_input":"2023-11-26T14:16:59.067877Z","iopub.status.idle":"2023-11-26T14:16:59.784306Z","shell.execute_reply.started":"2023-11-26T14:16:59.067834Z","shell.execute_reply":"2023-11-26T14:16:59.782825Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"I am selecting the features which I consider important for training my model","metadata":{"execution":{"iopub.execute_input":"2023-11-13T21:52:33.733544Z","iopub.status.busy":"2023-11-13T21:52:33.733141Z","iopub.status.idle":"2023-11-13T21:52:33.740524Z","shell.execute_reply":"2023-11-13T21:52:33.738791Z","shell.execute_reply.started":"2023-11-13T21:52:33.733513Z"},"papermill":{"duration":0.024752,"end_time":"2023-11-21T14:57:29.216340","exception":false,"start_time":"2023-11-21T14:57:29.191588","status":"completed"},"tags":[]}},{"cell_type":"code","source":"train = dfm[['category', 'amt', 'zip', 'lat', 'long', 'merch_lat', 'merch_long', 'age', 'hour', 'day', 'month', 'is_fraud']]\ncategoryints = pd.get_dummies(train['category'], prefix='category', drop_first=True)\ncategoryints = categoryints.astype(int)# Replace boolean values with 0s and 1s\ntrain = train.drop('category', axis=1)\ntrain = pd.concat([train, categoryints], axis=1)\ny_train = train['is_fraud']\nX_train = train.drop(\"is_fraud\", axis=1)","metadata":{"papermill":{"duration":0.243124,"end_time":"2023-11-21T14:57:29.484257","exception":false,"start_time":"2023-11-21T14:57:29.241133","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-11-26T14:16:59.786135Z","iopub.execute_input":"2023-11-26T14:16:59.786460Z","iopub.status.idle":"2023-11-26T14:16:59.963394Z","shell.execute_reply.started":"2023-11-26T14:16:59.786433Z","shell.execute_reply":"2023-11-26T14:16:59.961773Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Similar operations on test dataset","metadata":{"papermill":{"duration":0.024971,"end_time":"2023-11-21T14:57:29.534827","exception":false,"start_time":"2023-11-21T14:57:29.509856","status":"completed"},"tags":[]}},{"cell_type":"code","source":"dftestf['trans_date_trans_time']=pd.to_datetime(dftestf['trans_date_trans_time'])\ndftestf['trans_date']=dftestf['trans_date_trans_time'].dt.strftime('%Y-%m-%d')\ndftestf['trans_date']=pd.to_datetime(dftestf['trans_date'])\ndftestf['dob']=pd.to_datetime(dftestf['dob'])\ndftestf['amt'] = np.log1p(dftestf['amt'])\ndftestf['age']=dftestf['trans_date'].dt.year-dftestf['dob'].dt.year\ndftestf['hour']=dftestf['trans_date_trans_time'].dt.hour\ndftestf['day']=dftestf['trans_date_trans_time'].dt.dayofweek\ndftestf['month']=dftestf['trans_date_trans_time'].dt.month\ntestf = dftestf[['category', 'amt', 'zip', 'lat', 'long', 'merch_lat', 'merch_long', 'age', 'hour', 'day', 'month']]\ncategorytest = pd.get_dummies(testf['category'], prefix='category', drop_first=True)\ncategorytest = categorytest.astype(int)\ntestf = testf.drop('category', axis=1)\ntestf = pd.concat([testf, categorytest], axis=1)","metadata":{"papermill":{"duration":1.288547,"end_time":"2023-11-21T14:57:30.848253","exception":false,"start_time":"2023-11-21T14:57:29.559706","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-11-26T14:16:59.964466Z","iopub.execute_input":"2023-11-26T14:16:59.964821Z","iopub.status.idle":"2023-11-26T14:17:00.921098Z","shell.execute_reply.started":"2023-11-26T14:16:59.964791Z","shell.execute_reply":"2023-11-26T14:17:00.920079Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**Handling Imbalanced nature of Train Dataset** ***using SMOTE (Synthetic Minority Oversampling Technique)***","metadata":{"papermill":{"duration":0.024976,"end_time":"2023-11-21T14:57:30.898882","exception":false,"start_time":"2023-11-21T14:57:30.873906","status":"completed"},"tags":[]}},{"cell_type":"code","source":"sm = SMOTE(random_state=42)#since only 1 one out of 69 transaction is fraud. So I am doing oversampling\nX_train_new,y_train_new=sm.fit_resample(X_train,y_train)","metadata":{"papermill":{"duration":0.993104,"end_time":"2023-11-21T14:57:31.917126","exception":false,"start_time":"2023-11-21T14:57:30.924022","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-11-26T14:17:00.923792Z","iopub.execute_input":"2023-11-26T14:17:00.925039Z","iopub.status.idle":"2023-11-26T14:17:01.910240Z","shell.execute_reply.started":"2023-11-26T14:17:00.924985Z","shell.execute_reply":"2023-11-26T14:17:01.908968Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"I opted for oversampling in this case because, during the initial trial and testing, it yielded higher scores compared to undersampling.","metadata":{}},{"cell_type":"code","source":"y_train.value_counts()","metadata":{"papermill":{"duration":0.039447,"end_time":"2023-11-21T14:57:31.982674","exception":false,"start_time":"2023-11-21T14:57:31.943227","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-11-26T14:17:01.914307Z","iopub.execute_input":"2023-11-26T14:17:01.915768Z","iopub.status.idle":"2023-11-26T14:17:01.930882Z","shell.execute_reply.started":"2023-11-26T14:17:01.915712Z","shell.execute_reply":"2023-11-26T14:17:01.929653Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"y_train_new.value_counts()","metadata":{"papermill":{"duration":0.043292,"end_time":"2023-11-21T14:57:32.051426","exception":false,"start_time":"2023-11-21T14:57:32.008134","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-11-26T14:17:01.933047Z","iopub.execute_input":"2023-11-26T14:17:01.933508Z","iopub.status.idle":"2023-11-26T14:17:01.958018Z","shell.execute_reply.started":"2023-11-26T14:17:01.933473Z","shell.execute_reply":"2023-11-26T14:17:01.956875Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Now the train dataset is balanced","metadata":{"papermill":{"duration":0.025443,"end_time":"2023-11-21T14:57:32.102283","exception":false,"start_time":"2023-11-21T14:57:32.076840","status":"completed"},"tags":[]}},{"cell_type":"markdown","source":"<a id=\"5\"></a>\n<h1 style=\"background-color:#0F0F0F;font-family:newtimeroman;font-size:350%;text-align:center;border-radius: 15px 50px;color:#F0F0F0\">Cross Validation Set</h1>","metadata":{"papermill":{"duration":0.026541,"end_time":"2023-11-21T14:57:32.154369","exception":false,"start_time":"2023-11-21T14:57:32.127828","status":"completed"},"tags":[]}},{"cell_type":"code","source":"X_traincv,X_testcv,y_traincv,y_testcv = train_test_split(X_train_new,y_train_new,test_size = 0.3, random_state = 42) \n#cross validation","metadata":{"execution":{"iopub.execute_input":"2023-11-21T14:57:32.207137Z","iopub.status.busy":"2023-11-21T14:57:32.206845Z","iopub.status.idle":"2023-11-21T14:57:32.591980Z","shell.execute_reply":"2023-11-21T14:57:32.591161Z"},"papermill":{"duration":0.413988,"end_time":"2023-11-21T14:57:32.594333","exception":false,"start_time":"2023-11-21T14:57:32.180345","status":"completed"},"tags":[]},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<a id=\"5.1\"></a>\n<h1><i><b>Training</b></i></h1>","metadata":{"papermill":{"duration":0.025897,"end_time":"2023-11-21T14:57:32.647703","exception":false,"start_time":"2023-11-21T14:57:32.621806","status":"completed"},"tags":[]}},{"cell_type":"markdown","source":"<h4>The hyperparameters utilized in the implemented models have been determined through RandomizedSearchCV and Optuna.</h4>\nYou can find the code for hyperparameters on my GitHub.","metadata":{}},{"cell_type":"markdown","source":"<a id=\"5.1.1\"></a>\n<h3><u><b>RandomForestClassifier</b></u></h3>","metadata":{"papermill":{"duration":0.025391,"end_time":"2023-11-21T14:57:32.698504","exception":false,"start_time":"2023-11-21T14:57:32.673113","status":"completed"},"tags":[]}},{"cell_type":"code","source":"best_rfc_params={'n_estimators': 1350,\n 'min_samples_split': 5,\n 'min_samples_leaf': 1,\n 'max_features': 'log2',\n 'max_depth': 1022,\n 'criterion': 'entropy'}\nrfc=RandomForestClassifier(**best_rfc_params)\nrfc.fit(X_traincv,y_traincv)\ny_predcv=rfc.predict(X_testcv)\nrfc_roc=roc_auc_score(y_testcv,y_predcv)\nrfc_acc=accuracy_score(y_testcv,y_predcv)\nprint(\"Confusion Matrix: \",confusion_matrix(y_testcv,y_predcv))\nprint(\"Accuracy Score: \",rfc_acc)\nprint(\"ROC_AUC_Score: \",rfc_roc)","metadata":{"execution":{"iopub.execute_input":"2023-11-21T14:57:32.751181Z","iopub.status.busy":"2023-11-21T14:57:32.750822Z","iopub.status.idle":"2023-11-21T15:40:37.284923Z","shell.execute_reply":"2023-11-21T15:40:37.283868Z"},"papermill":{"duration":2584.59302,"end_time":"2023-11-21T15:40:37.316930","exception":false,"start_time":"2023-11-21T14:57:32.723910","status":"completed"},"tags":[]},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<a id=\"5.1.2\"></a>\n<h3><u><b>XGBoost</b></u></h3>","metadata":{"papermill":{"duration":0.02646,"end_time":"2023-11-21T15:40:37.369532","exception":false,"start_time":"2023-11-21T15:40:37.343072","status":"completed"},"tags":[]}},{"cell_type":"code","source":"best_xgb_params={'min_child_weight': 1,\n 'max_depth': 15,\n 'learning_rate': 0.25,\n 'gamma': 0.4,\n 'colsample_bytree': 0.5}\nxg=xgboost.XGBClassifier(**best_xgb_params)\nxg.fit(X_traincv,y_traincv)\ny_predcv=xg.predict(X_testcv)\nxgb_roc=roc_auc_score(y_testcv,y_predcv)\nxgb_acc=accuracy_score(y_testcv,y_predcv)\nprint(\"Confusion Matrix: \",confusion_matrix(y_testcv,y_predcv))\nprint(\"Accuracy Score: \",xgb_acc)\nprint(\"ROC_AUC_Score: \",xgb_roc)","metadata":{"execution":{"iopub.execute_input":"2023-11-21T15:40:37.422788Z","iopub.status.busy":"2023-11-21T15:40:37.422456Z","iopub.status.idle":"2023-11-21T15:40:48.030931Z","shell.execute_reply":"2023-11-21T15:40:48.029867Z"},"papermill":{"duration":10.639078,"end_time":"2023-11-21T15:40:48.034514","exception":false,"start_time":"2023-11-21T15:40:37.395436","status":"completed"},"tags":[]},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<a id=\"5.1.3\"></a>\n<h3><u><b>Light GBM</b></u></h3>","metadata":{"papermill":{"duration":0.026359,"end_time":"2023-11-21T15:40:48.087707","exception":false,"start_time":"2023-11-21T15:40:48.061348","status":"completed"},"tags":[]}},{"cell_type":"code","source":"best_lgb_params={'learning_rate': 0.045, 'n_estimators': 2000}\nlgbmcv=lgb.LGBMClassifier(**best_lgb_params)\nlgbmcv.fit(X_traincv,y_traincv)\ny_predcv=lgbmcv.predict(X_testcv)\nlgbm_roc=roc_auc_score(y_testcv,y_predcv)\nlgbm_acc=accuracy_score(y_testcv,y_predcv)\nprint(\"Confusion Matrix: \",confusion_matrix(y_testcv,y_predcv))\nprint(\"Accuracy Score: \",lgbm_acc)\nprint(\"ROC_AUC_Score: \",lgbm_roc)","metadata":{"execution":{"iopub.execute_input":"2023-11-21T15:40:48.141714Z","iopub.status.busy":"2023-11-21T15:40:48.141372Z","iopub.status.idle":"2023-11-21T15:41:57.741824Z","shell.execute_reply":"2023-11-21T15:41:57.740777Z"},"papermill":{"duration":69.663139,"end_time":"2023-11-21T15:41:57.777375","exception":false,"start_time":"2023-11-21T15:40:48.114236","status":"completed"},"tags":[]},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<a id=\"5.1.4\"></a>\n<h3><u><b>DecisionTreeClassifier</b></u></h3>","metadata":{"papermill":{"duration":0.030797,"end_time":"2023-11-21T15:41:57.842986","exception":false,"start_time":"2023-11-21T15:41:57.812189","status":"completed"},"tags":[]}},{"cell_type":"code","source":"best_dtc_params={'max_depth': 207, 'min_samples_split': 0.01,\n                 'criterion': 'entropy', 'max_features': None}\n\ndtc=DecisionTreeClassifier(**best_dtc_params)\ndtc.fit(X_traincv,y_traincv)\ny_predcv=dtc.predict(X_testcv)\ndtc_roc=roc_auc_score(y_testcv,y_predcv)\ndtc_acc=accuracy_score(y_testcv,y_predcv)\nprint(\"Confusion Matrix: \",confusion_matrix(y_testcv,y_predcv))\nprint(\"Accuracy Score: \",dtc_acc)\nprint(\"ROC_AUC_Score: \",dtc_roc)","metadata":{"execution":{"iopub.execute_input":"2023-11-21T15:41:57.902909Z","iopub.status.busy":"2023-11-21T15:41:57.902540Z","iopub.status.idle":"2023-11-21T15:42:07.831341Z","shell.execute_reply":"2023-11-21T15:42:07.830300Z"},"papermill":{"duration":9.96286,"end_time":"2023-11-21T15:42:07.834915","exception":false,"start_time":"2023-11-21T15:41:57.872055","status":"completed"},"tags":[]},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<a id=\"5.1.5\"></a>\n<h3><u><b>Logistic Regression</b></u></h3>","metadata":{"papermill":{"duration":0.026784,"end_time":"2023-11-21T15:42:07.889376","exception":false,"start_time":"2023-11-21T15:42:07.862592","status":"completed"},"tags":[]}},{"cell_type":"code","source":"best_lr_params={'C': 85539.17794889913, 'penalty': 'l1','solver':'liblinear'}\n\nlrc=LogisticRegression(**best_lr_params)\nlrc.fit(X_traincv,y_traincv)\ny_predcv=lrc.predict(X_testcv)\nlrc_roc=roc_auc_score(y_testcv,y_predcv)\nlrc_acc=accuracy_score(y_testcv,y_predcv)\nprint(\"Confusion Matrix: \",confusion_matrix(y_testcv,y_predcv))\nprint(\"Accuracy Score: \",lrc_acc)\nprint(\"ROC_AUC_Score: \",lrc_roc)","metadata":{"execution":{"iopub.execute_input":"2023-11-21T15:42:07.944444Z","iopub.status.busy":"2023-11-21T15:42:07.944131Z","iopub.status.idle":"2023-11-21T15:42:52.343993Z","shell.execute_reply":"2023-11-21T15:42:52.342901Z"},"papermill":{"duration":44.429663,"end_time":"2023-11-21T15:42:52.345978","exception":false,"start_time":"2023-11-21T15:42:07.916315","status":"completed"},"tags":[]},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<a id=\"5.2\"></a>\n<h1><i><b>Scores</b></i></h1>","metadata":{"papermill":{"duration":0.027133,"end_time":"2023-11-21T15:42:52.400586","exception":false,"start_time":"2023-11-21T15:42:52.373453","status":"completed"},"tags":[]}},{"cell_type":"code","source":"\nmodels = [('Random Forest Classifier', rfc_acc, rfc_roc),\n          ('XGBoost Classifier', xgb_acc, xgb_roc),\n          ('LightGBM', lgbm_acc, lgbm_roc),\n          ('Decision Tree Classifier', dtc_acc, dtc_roc),\n          ('Logistic Regression', lrc_acc, lrc_roc)\n         ]\n\nmodelsdf = pd.DataFrame(data=models, columns=['Model', 'Accuracy', 'ROC_AUC Score']) \\\n                    .sort_values(by='ROC_AUC Score', ascending=False)\n\nprint(modelsdf)\n","metadata":{"execution":{"iopub.execute_input":"2023-11-21T15:42:52.456720Z","iopub.status.busy":"2023-11-21T15:42:52.456347Z","iopub.status.idle":"2023-11-21T15:42:52.466385Z","shell.execute_reply":"2023-11-21T15:42:52.465265Z"},"papermill":{"duration":0.040292,"end_time":"2023-11-21T15:42:52.468300","exception":false,"start_time":"2023-11-21T15:42:52.428008","status":"completed"},"tags":[]},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"The output above is sorted in DESCENDING order of ROC_AUC Score","metadata":{"papermill":{"duration":0.027277,"end_time":"2023-11-21T15:42:52.523148","exception":false,"start_time":"2023-11-21T15:42:52.495871","status":"completed"},"tags":[]}},{"cell_type":"code","source":"plt.figure(figsize=(10, 6))\nplt.bar(modelsdf['Model'], modelsdf['ROC_AUC Score'], color='red')\nplt.xlabel('Model')\nplt.ylabel('ROC_AUC Score')\nplt.title('ROC_AUC Score for Different Models')\nplt.xticks(rotation=45, ha='right')#ha='right' specifies horizontal alignment of the tick labels, aligning them to the right\nplt.tight_layout()#adjusts subplot parameters to prevent overlapping of labels, titles, and other elements within the figure\nplt.show()","metadata":{"execution":{"iopub.execute_input":"2023-11-21T15:42:52.579154Z","iopub.status.busy":"2023-11-21T15:42:52.578873Z","iopub.status.idle":"2023-11-21T15:42:52.890269Z","shell.execute_reply":"2023-11-21T15:42:52.889390Z"},"papermill":{"duration":0.341894,"end_time":"2023-11-21T15:42:52.892482","exception":false,"start_time":"2023-11-21T15:42:52.550588","status":"completed"},"tags":[]},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<a id = \"6\"></a>\n<h1 style=\"background-color:#0F0F0F;font-family:newtimeroman;font-size:350%;text-align:center;border-radius: 15px 50px;color:#F0F0F0\">Model Training</h1>","metadata":{"papermill":{"duration":0.028235,"end_time":"2023-11-21T15:42:52.949389","exception":false,"start_time":"2023-11-21T15:42:52.921154","status":"completed"},"tags":[]}},{"cell_type":"markdown","source":"<a id=\"6.1\"></a>\n<h5><u><b>RandomForestClassifier</b></u></h5>","metadata":{"papermill":{"duration":0.028206,"end_time":"2023-11-21T15:42:53.005775","exception":false,"start_time":"2023-11-21T15:42:52.977569","status":"completed"},"tags":[]}},{"cell_type":"code","source":"rfcf=RandomForestClassifier(**best_rfc_params)\nrfcf.fit(X_train_new,y_train_new)","metadata":{"execution":{"iopub.execute_input":"2023-11-21T15:42:53.063737Z","iopub.status.busy":"2023-11-21T15:42:53.063391Z","iopub.status.idle":"2023-11-21T16:47:30.216718Z","shell.execute_reply":"2023-11-21T16:47:30.215779Z"},"papermill":{"duration":3877.214916,"end_time":"2023-11-21T16:47:30.248843","exception":false,"start_time":"2023-11-21T15:42:53.033927","status":"completed"},"tags":[]},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<a id=\"6.2\"></a>\n<h5><u><b>XGBoost</b></u></h5>","metadata":{"papermill":{"duration":0.028828,"end_time":"2023-11-21T16:47:30.307280","exception":false,"start_time":"2023-11-21T16:47:30.278452","status":"completed"},"tags":[]}},{"cell_type":"code","source":"xgf=xgboost.XGBClassifier(**best_xgb_params)\nxgf.fit(X_train_new,y_train_new)","metadata":{"execution":{"iopub.execute_input":"2023-11-21T16:47:30.367309Z","iopub.status.busy":"2023-11-21T16:47:30.366540Z","iopub.status.idle":"2023-11-21T16:47:43.817822Z","shell.execute_reply":"2023-11-21T16:47:43.816761Z"},"papermill":{"duration":13.483474,"end_time":"2023-11-21T16:47:43.819999","exception":false,"start_time":"2023-11-21T16:47:30.336525","status":"completed"},"tags":[]},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<a id=\"6.3\"></a>\n<h5><u><b>Light GBM</b></u></h5>","metadata":{"papermill":{"duration":0.029491,"end_time":"2023-11-21T16:47:43.879910","exception":false,"start_time":"2023-11-21T16:47:43.850419","status":"completed"},"tags":[]}},{"cell_type":"code","source":"lgbmf=lgb.LGBMClassifier(**best_lgb_params)\nlgbmf.fit(X_train_new,y_train_new)","metadata":{"execution":{"iopub.execute_input":"2023-11-21T16:47:43.941397Z","iopub.status.busy":"2023-11-21T16:47:43.940639Z","iopub.status.idle":"2023-11-21T16:49:01.233835Z","shell.execute_reply":"2023-11-21T16:49:01.232761Z"},"papermill":{"duration":77.356126,"end_time":"2023-11-21T16:49:01.266246","exception":false,"start_time":"2023-11-21T16:47:43.910120","status":"completed"},"tags":[]},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<a id=\"6.4\"></a>\n<h5><u><b>DecisionTreeClassifier</b></u></h5>","metadata":{"papermill":{"duration":0.030191,"end_time":"2023-11-21T16:49:01.327143","exception":false,"start_time":"2023-11-21T16:49:01.296952","status":"completed"},"tags":[]}},{"cell_type":"code","source":"dtcf=DecisionTreeClassifier(**best_dtc_params)\ndtcf.fit(X_train_new,y_train_new)","metadata":{"execution":{"iopub.execute_input":"2023-11-21T16:49:01.389103Z","iopub.status.busy":"2023-11-21T16:49:01.388739Z","iopub.status.idle":"2023-11-21T16:49:16.208937Z","shell.execute_reply":"2023-11-21T16:49:16.207947Z"},"papermill":{"duration":14.853495,"end_time":"2023-11-21T16:49:16.211120","exception":false,"start_time":"2023-11-21T16:49:01.357625","status":"completed"},"tags":[]},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<a id=\"6.5\"></a>\n<h5><u><b>Logistic Regression</b></u></h5>","metadata":{"papermill":{"duration":0.030005,"end_time":"2023-11-21T16:49:16.272219","exception":false,"start_time":"2023-11-21T16:49:16.242214","status":"completed"},"tags":[]}},{"cell_type":"code","source":"lrcf=LogisticRegression(**best_lr_params)\nlrcf.fit(X_train_new,y_train_new)","metadata":{"execution":{"iopub.execute_input":"2023-11-21T16:49:16.334943Z","iopub.status.busy":"2023-11-21T16:49:16.333960Z","iopub.status.idle":"2023-11-21T16:50:18.756338Z","shell.execute_reply":"2023-11-21T16:50:18.755291Z"},"papermill":{"duration":62.487226,"end_time":"2023-11-21T16:50:18.789336","exception":false,"start_time":"2023-11-21T16:49:16.302110","status":"completed"},"tags":[]},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<a id = \"7\"></a>\n<h1 style=\"background-color:#0F0F0F;font-family:newtimeroman;font-size:350%;text-align:center;border-radius: 15px 50px;color:#F0F0F0\">Predictions</h1>","metadata":{"papermill":{"duration":0.030255,"end_time":"2023-11-21T16:50:18.850003","exception":false,"start_time":"2023-11-21T16:50:18.819748","status":"completed"},"tags":[]}},{"cell_type":"markdown","source":"Making predictions on the test set using Random Forest, XGBoost, LightGBM, Decision Tree and Logistic Regression.","metadata":{"papermill":{"duration":0.030564,"end_time":"2023-11-21T16:50:18.910780","exception":false,"start_time":"2023-11-21T16:50:18.880216","status":"completed"},"tags":[]}},{"cell_type":"code","source":"y_testfrfcf=rfcf.predict(testf)\ny_testfxgf=xgf.predict(testf)\ny_testflgbmf=lgbmf.predict(testf)\ny_testfdtcf=dtcf.predict(testf)\ny_testflrcf=lrcf.predict(testf)","metadata":{"execution":{"iopub.execute_input":"2023-11-21T16:50:18.975286Z","iopub.status.busy":"2023-11-21T16:50:18.974520Z","iopub.status.idle":"2023-11-21T16:52:28.734546Z","shell.execute_reply":"2023-11-21T16:52:28.733120Z"},"papermill":{"duration":129.868205,"end_time":"2023-11-21T16:52:28.809670","exception":false,"start_time":"2023-11-21T16:50:18.941465","status":"completed"},"tags":[]},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<a id = \"8\"></a>\n<h1 style=\"background-color:#0F0F0F;font-family:newtimeroman;font-size:350%;text-align:center;border-radius: 15px 50px;color:#F0F0F0\">Submissions</h1>","metadata":{"papermill":{"duration":0.030751,"end_time":"2023-11-21T16:52:28.895927","exception":false,"start_time":"2023-11-21T16:52:28.865176","status":"completed"},"tags":[]}},{"cell_type":"markdown","source":"Creating submission DataFrames for each model and saving predictions in CSV files ","metadata":{"papermill":{"duration":0.030421,"end_time":"2023-11-21T16:52:28.956951","exception":false,"start_time":"2023-11-21T16:52:28.926530","status":"completed"},"tags":[]}},{"cell_type":"code","source":"submissionrfcf=pd.DataFrame(data={'id': range(1, len(testf)+1), 'is_fraud': y_testfrfcf})\nsubmissionrfcf.to_csv('rfc.csv', index=False)#index=false means not including row numbers in the CSV file.\nsubmissionxgf=pd.DataFrame(data={'id': range(1, len(testf)+1), 'is_fraud': y_testfxgf})\nsubmissionxgf.to_csv('xg.csv', index=False)\nsubmissionlgbmf=pd.DataFrame(data={'id': range(1, len(testf)+1), 'is_fraud': y_testflgbmf})\nsubmissionlgbmf.to_csv('lgbm.csv', index=False)\nsubmissiondtcf=pd.DataFrame(data={'id': range(1, len(testf)+1), 'is_fraud': y_testfdtcf})\nsubmissiondtcf.to_csv('dtc.csv', index=False)\nsubmissionlrf=pd.DataFrame(data={'id': range(1, len(testf)+1), 'is_fraud': y_testflrcf})\nsubmissionlrf.to_csv('lr.csv', index=False)","metadata":{"execution":{"iopub.execute_input":"2023-11-21T16:52:29.020803Z","iopub.status.busy":"2023-11-21T16:52:29.020038Z","iopub.status.idle":"2023-11-21T16:52:34.395200Z","shell.execute_reply":"2023-11-21T16:52:34.394408Z"},"papermill":{"duration":5.409996,"end_time":"2023-11-21T16:52:34.397705","exception":false,"start_time":"2023-11-21T16:52:28.987709","status":"completed"},"tags":[]},"execution_count":null,"outputs":[]}]}