{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"code","source":" #Regular EDA and plotting libraries\nimport numpy as np # np is short for numpy\nimport pandas as pd # pandas is so commonly used, it's shortened to pd\nimport matplotlib.pyplot as plt\nimport seaborn as sns # seaborn gets shortened to sns\n\n# We want our plots to appear in the notebook\n%matplotlib inline \n\n## Models\nfrom sklearn.linear_model import LogisticRegression\nfrom sklearn.neighbors import KNeighborsClassifier\nfrom sklearn.ensemble import RandomForestClassifier\n\n## Model evaluators\nfrom sklearn.model_selection import train_test_split, cross_val_score\nfrom sklearn.model_selection import RandomizedSearchCV, GridSearchCV\nfrom sklearn.metrics import confusion_matrix, classification_report\nfrom sklearn.metrics import precision_score, recall_score, f1_score\nfrom sklearn.metrics import plot_roc_curve\n\ndf = pd.read_csv(\"../input/bigdatadebry/nyra_2019_complete.csv\");","metadata":{"id":"xHDAtp47lNsQ","execution":{"iopub.status.busy":"2022-10-03T18:27:22.802973Z","iopub.execute_input":"2022-10-03T18:27:22.804318Z","iopub.status.idle":"2022-10-03T18:27:41.304461Z","shell.execute_reply.started":"2022-10-03T18:27:22.804212Z","shell.execute_reply":"2022-10-03T18:27:41.303290Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Columns\n\n\n1.     track_id - 3 character id for the track the race took place at. AQU -Aqueduct, BEL - Belmont, SAR - Saratoga.\n1.     race_date - date the race took place. YYYY-MM-DD.\n1.     race_number - Number of the race. Passed as 3 characters but can be cast or converted to int for this data set.\n1.     program_number - Program number of the horse in the race passed as 3 characters. Should remain 3 characters as it isn't limited to just numbers. Is essentially the unique       identifier of the horse in the race.\n1.     weight_carried - An integer of the weight carried by the horse in the race.\n1.     jockey - Name of the jockey on the horse in the race. 50 character max.\n1.     odds - Odds to win the race passed as an integer. Divide by 100 to derive the odds to 1. Example - 1280 would be 12.8-1.\n1.     distance_id - Distance of the race in furlongs passed as an integer. Example - 600 would be 6 furlongs.\n1.     course_type - The course the race was run over passed as one character. M - Hurdle, D - Dirt, O - Outer turf, I - Inner turf, T - turf.\n1.     track_condition - The condition of the course the race was run on passed as three characters. YL - Yielding, FM - Firm, SY - Sloppy, GD - Good, FT - Fast, MY - Muddy, SF -     Soft.\n1.     run_up_distance - Distance in feet of the gate to the start of the race passed as an integer.\n1.     race_type - The classification of the race passed as as five characters. STK - Stakes, WCL - Waiver Claiming, WMC - Waiver Maiden Claiming, SST - Starter Stakes, SHP -         Starter Handicap, CLM - Claiming, STR - Starter Allowance, AOC - Allowance Optionl Claimer, SOC - Starter Optional Claimer, MCL - Maiden Claiming, ALW - Allowance, MSW -       Maiden Special Weight.\n1.     purse - Purse in US dollars of the race passed as an money with two decimal places.\n1.     post_time - Time of day the race began passed as 5 character. Example - 01220 would be 12:20.\n1.     trakus_index - The common collection of point of the lat / long of the horse in the race passed as an integer. From what we can tell, it's collected every 0.25 seconds.\n1.     latitude - The latitude of the horse in the race passed as a float.\n1.     longitude - The longitude of the horse in the race passed as a float.\n","metadata":{"id":"6sFg3NXRlNsk"}},{"cell_type":"markdown","source":"# Exploratory data analysis (EDA)","metadata":{"execution":{"iopub.status.busy":"2022-09-05T18:19:53.447605Z","iopub.execute_input":"2022-09-05T18:19:53.448098Z","iopub.status.idle":"2022-09-05T18:19:53.453289Z","shell.execute_reply.started":"2022-09-05T18:19:53.448060Z","shell.execute_reply":"2022-09-05T18:19:53.451913Z"},"id":"bgAsesuBlNsp"}},{"cell_type":"code","source":"df.head()","metadata":{"id":"MhDvCrUTlNst","execution":{"iopub.status.busy":"2022-10-03T18:27:50.417031Z","iopub.execute_input":"2022-10-03T18:27:50.417494Z","iopub.status.idle":"2022-10-03T18:27:50.452838Z","shell.execute_reply.started":"2022-10-03T18:27:50.417459Z","shell.execute_reply":"2022-10-03T18:27:50.451879Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Transfer the initial data set to the temporary data set.","metadata":{"id":"uykGXYF-lNsz"}},{"cell_type":"code","source":"df_tmp = df.copy()","metadata":{"id":"xy8_AUnqlNs0","execution":{"iopub.status.busy":"2022-10-03T18:27:59.059134Z","iopub.execute_input":"2022-10-03T18:27:59.059581Z","iopub.status.idle":"2022-10-03T18:27:59.809232Z","shell.execute_reply.started":"2022-10-03T18:27:59.059546Z","shell.execute_reply":"2022-10-03T18:27:59.808114Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_tmp.head()","metadata":{"id":"F4F3i7G-lNs3","execution":{"iopub.status.busy":"2022-10-03T18:28:02.889524Z","iopub.execute_input":"2022-10-03T18:28:02.889978Z","iopub.status.idle":"2022-10-03T18:28:02.911984Z","shell.execute_reply.started":"2022-10-03T18:28:02.889945Z","shell.execute_reply":"2022-10-03T18:28:02.910650Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Odds\n## looking for features that have a greater impact on winning chances","metadata":{"id":"rQYmWRFGV5Sy"}},{"cell_type":"markdown","source":"## I created the numerical feature subset just in case.","metadata":{}},{"cell_type":"code","source":"df_tmp_features_numerical = df_tmp.drop(['track_id','course_type','track_condition','race_type','jockey','odds'],axis = 1)\ndf_tmp_features_numerical.head()","metadata":{"id":"R65yGVoSlNs7","execution":{"iopub.status.busy":"2022-10-03T18:30:05.708828Z","iopub.execute_input":"2022-10-03T18:30:05.709300Z","iopub.status.idle":"2022-10-03T18:30:06.027550Z","shell.execute_reply.started":"2022-10-03T18:30:05.709264Z","shell.execute_reply":"2022-10-03T18:30:06.026125Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Provide a precise time and date.","metadata":{}},{"cell_type":"code","source":"df_tmp_features_numerical['race_date'] = pd.to_datetime(df_tmp_features_numerical['race_date'], errors='coerce')\ndf_tmp_features_numerical[\"DateMonth\"] = df_tmp_features_numerical.race_date.dt.month\ndf_tmp_features_numerical[\"DateDay\"] = df_tmp_features_numerical.race_date.dt.day\ndf_tmp_features_numerical.drop(\"race_date\", axis=1, inplace=True)","metadata":{"id":"_kWZasBElNtB","execution":{"iopub.status.busy":"2022-10-03T18:30:15.509686Z","iopub.execute_input":"2022-10-03T18:30:15.510195Z","iopub.status.idle":"2022-10-03T18:30:18.085678Z","shell.execute_reply.started":"2022-10-03T18:30:15.510157Z","shell.execute_reply":"2022-10-03T18:30:18.084389Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# To avoid another problem caused by having too many NULL values in the Program_number feature, I removed it for now. I intend to use this feature for the next analysis and compare them together. ","metadata":{"id":"66w5VHaWWUAh"}},{"cell_type":"code","source":"df_tmp_features_numerical.drop([\"program_number\"], axis=1, inplace=True)","metadata":{"id":"PgqiePpdlNtD","execution":{"iopub.status.busy":"2022-10-03T18:34:22.323287Z","iopub.execute_input":"2022-10-03T18:34:22.324611Z","iopub.status.idle":"2022-10-03T18:34:22.618725Z","shell.execute_reply.started":"2022-10-03T18:34:22.324567Z","shell.execute_reply":"2022-10-03T18:34:22.617092Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Numerical Features","metadata":{}},{"cell_type":"code","source":"df_tmp_features_numerical.head()","metadata":{"id":"SJmyLUSNlNtF","execution":{"iopub.status.busy":"2022-10-03T18:34:27.460290Z","iopub.execute_input":"2022-10-03T18:34:27.460757Z","iopub.status.idle":"2022-10-03T18:34:27.481370Z","shell.execute_reply.started":"2022-10-03T18:34:27.460721Z","shell.execute_reply":"2022-10-03T18:34:27.480337Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Dummy Variables\n## A dummy variable is a numerical variable that is used in regression analysis to represent subgroups of your study's sample in the conversion of categorical features to numbers.","metadata":{"id":"aJhDqxw0lNtH"}},{"cell_type":"code","source":"df_classify = df_tmp.select_dtypes(include=[object])\ndf_classify.drop([\"program_number\",\"race_date\"], axis=1, inplace=True)","metadata":{"id":"UR-0WTzElNtI","execution":{"iopub.status.busy":"2022-10-03T18:37:54.984637Z","iopub.execute_input":"2022-10-03T18:37:54.985174Z","iopub.status.idle":"2022-10-03T18:37:56.381254Z","shell.execute_reply.started":"2022-10-03T18:37:54.985137Z","shell.execute_reply":"2022-10-03T18:37:56.380146Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_classify.head()","metadata":{"id":"jet3DIk3lNtJ","execution":{"iopub.status.busy":"2022-10-03T18:38:02.080909Z","iopub.execute_input":"2022-10-03T18:38:02.081607Z","iopub.status.idle":"2022-10-03T18:38:02.096808Z","shell.execute_reply.started":"2022-10-03T18:38:02.081552Z","shell.execute_reply":"2022-10-03T18:38:02.095842Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"from sklearn import preprocessing","metadata":{"execution":{"iopub.status.busy":"2022-10-03T18:38:09.590432Z","iopub.execute_input":"2022-10-03T18:38:09.590920Z","iopub.status.idle":"2022-10-03T18:38:09.597681Z","shell.execute_reply.started":"2022-10-03T18:38:09.590883Z","shell.execute_reply":"2022-10-03T18:38:09.596132Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_classify.columns","metadata":{"execution":{"iopub.status.busy":"2022-10-03T18:38:13.905223Z","iopub.execute_input":"2022-10-03T18:38:13.905713Z","iopub.status.idle":"2022-10-03T18:38:13.914228Z","shell.execute_reply.started":"2022-10-03T18:38:13.905676Z","shell.execute_reply":"2022-10-03T18:38:13.913223Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"le = preprocessing.LabelEncoder()\nX_2 = df_classify.apply(le.fit_transform)\nX_2.head()","metadata":{"execution":{"iopub.status.busy":"2022-10-03T18:38:23.444578Z","iopub.execute_input":"2022-10-03T18:38:23.445085Z","iopub.status.idle":"2022-10-03T18:38:29.630182Z","shell.execute_reply.started":"2022-10-03T18:38:23.445047Z","shell.execute_reply":"2022-10-03T18:38:29.628804Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# TODO: create a OneHotEncoder object, and fit it to all of X\n\n# 1. INSTANTIATE\nenc = preprocessing.OneHotEncoder()\n\n# 2. FIT\nenc.fit(X_2)\n\n# 3. Transform\nonehotlabels = enc.transform(X_2)\nonehotlabels.shape\n","metadata":{"execution":{"iopub.status.busy":"2022-10-03T18:38:40.506096Z","iopub.execute_input":"2022-10-03T18:38:40.506548Z","iopub.status.idle":"2022-10-03T18:38:42.692089Z","shell.execute_reply.started":"2022-10-03T18:38:40.506502Z","shell.execute_reply":"2022-10-03T18:38:42.690506Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"from sklearn.feature_selection import mutual_info_regression\nthreshold = 5  # the number of most relevant features\nhigh_score_features = []\nfeature_scores = mutual_info_regression(df_tmp_features_numerical, df_tmp.odds, random_state=0)\nfor score, f_name in sorted(zip(feature_scores, df_tmp_features_numerical.columns), reverse=True)[:threshold]:\n        print(f_name, score)\n        high_score_features.append(f_name)\ndf_wine_norm_mic = df_tmp_features_numerical[high_score_features]\nprint(df_wine_norm_mic.columns)","metadata":{"id":"E10V5g8ub3Fd","execution":{"iopub.status.busy":"2022-09-29T20:37:09.063337Z","iopub.execute_input":"2022-09-29T20:37:09.063896Z","iopub.status.idle":"2022-09-29T20:50:34.635806Z","shell.execute_reply.started":"2022-09-29T20:37:09.063861Z","shell.execute_reply":"2022-09-29T20:50:34.634448Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Five of the most significant df_tmp_numerical features, as determined by the mutual_info_regression result, are as follows:post_time,run_up_distance,purse,DateDay,weight_carried.\n","metadata":{"id":"vxobz0JLoaEz"}},{"cell_type":"markdown","source":"### Note:From the df_tmp_features_numerical, it could be seen that the post time or the Derby time of day is the most important feature.","metadata":{"id":"Oy8vOCoFpL74"}},{"cell_type":"code","source":"from sklearn.feature_selection import mutual_info_classif\nthreshold = 5  # the number of most relevant features\nhigh_score_features = []\nfeature_scores = mutual_info_classif(X_2, df_tmp.odds, random_state=0)\nfor score, f_name in sorted(zip(feature_scores, X_2.columns), reverse=True)[:threshold]:\n        print(f_name, score)\n        high_score_features.append(f_name)\ndf_wine_norm_mic = X_2[high_score_features]\nprint(df_wine_norm_mic.columns)","metadata":{"id":"bsbIi34IpcFq","execution":{"iopub.status.busy":"2022-10-03T18:59:19.183965Z","iopub.execute_input":"2022-10-03T18:59:19.184546Z","iopub.status.idle":"2022-10-03T19:05:54.146913Z","shell.execute_reply.started":"2022-10-03T18:59:19.184505Z","shell.execute_reply":"2022-10-03T19:05:54.145828Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## The most effective categorical features, as determined by the outcome, are:jockey,race_type,track_condition","metadata":{"id":"A_8IRkEYvXTa"}}]}