{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"\n![](https://upload.wikimedia.org/wikipedia/commons/thumb/4/48/GGF_Race5.jpg/1200px-GGF_Race5.jpg)\n\n## File descriptions\n\n* nyra_start_table.csv - horse/jockey race data\n* nyra_race_table.csv - racetrack race data\n* nyra_tracking_table.csv - tracking data\n* nyra_2019_complete.csv - combined table of three above files\n\n### Columns\n### nyra_start_table.csv\n\n* track_id - 3 character id for the track the race took place at. AQU -Aqueduct, BEL - Belmont, SAR - Saratoga.\n* race_date - date the race took place. YYYY-MM-DD.\n* race_number - Number of the race. Passed as 3 characters but can be cast or converted to int for this data set.\n* program_number - Program number of the horse in the race passed as 3 characters. Should remain 3 characters as it isn't limited to just numbers. Is essentially the unique identifier of the horse in the race.\n* weight_carried - An integer of the weight carried by the horse in the race.\n* jockey - Name of the jockey on the horse in the race. 50 character max.\n* odds - Odds to win the race passed as an integer. Divide by 100 to derive the odds to 1. Example - 1280 would be 12.8-1.\n\n### nyra_race_table.csv\n\n* `track_id` - 3 character id for the track the race took place at. AQU -Aqueduct, BEL - Belmont, SAR - Saratoga.\n* `race_date` - date the race took place. YYYY-MM-DD.\n* `race_number` - Number of the race. Passed as 3 characters but can be cast or converted to int for this data set.\n* `distance_id` - Distance of the race in furlongs passed as an integer. Example - 600 would be 6 furlongs.\n* `course_type` - The course the race was run over passed as one character. M - Hurdle, D - Dirt, O - Outer turf, I - Inner turf, T - turf.\n* `track_condition` - The condition of the course the race was run on passed as three characters. YL - Yielding, FM - Firm, SY - Sloppy, GD - Good, FT - Fast, MY - Muddy, SF - Soft.\n* `run_up_distance` - Distance in feet of the gate to the start of the race passed as an integer.\n* `race_type` - The classification of the race passed as as five characters. STK - Stakes, WCL - Waiver Claiming, WMC - Waiver Maiden Claiming, SST - Starter Stakes, SHP - Starter Handicap, CLM - Claiming, STR - Starter Allowance, AOC - Allowance Optionl Claimer, SOC - Starter Optional Claimer, MCL - Maiden Claiming, ALW - Allowance, MSW - Maiden Special Weight.\n* `purse` - Purse in US dollars of the race passed as an money with two decimal places.\n* post_time - Time of day the race began passed as 5 character. Example - 01220 would be 12:20.\n\n### nyra_tracking_table.csv\n\n* track_id - 3 character id for the track the race took place at. AQU -Aqueduct, BEL - Belmont, SAR - Saratoga.\n* race_date - date the race took place. YYYY-MM-DD.\n* race_number - Number of the race. Passed as 3 characters but can be cast or converted to int for this data set.\n* program_number - Program number of the horse in the race passed as 3 characters. Should remain 3 characters as it isn't limited to just numbers. Is essentially the unique identifier of the horse in the race.\n* trakus_index - The common collection of point of the lat / long of the horse in the race passed as an integer. From what we can tell, it's collected every 0.25 seconds.\n* latitude - The latitude of the horse in the race passed as a float.\n* longitude - The longitude of the horse in the race passed as a float.\n\n### nyra_2019_complete.csv -\n\nThis file is the combined 3 files into one table. The keys to join them trakus with race - track_id, race_date, race_number. To join trakus with start - track_id, race_date, race_number, program_number.\n\n* track_id - char(3)\n* race_date - date\n* race_number - char(3)\n* program_number - char(3)\n* trakus_index - int\n* latitude - float\n* longitude - float\n* distance_id - int\n* course_type - char(1)\n* track_condition - char(3)\n* run_up_distance - int\n* race_type - char(5)\n* post_time - char(5)\n* weight_carried - int\n* jockey - char(50)\n* odds - int","metadata":{}},{"cell_type":"code","source":"import warnings\nwarnings.filterwarnings(\"ignore\", category=DeprecationWarning)\nwarnings.filterwarnings(\"ignore\", category=UserWarning)\nwarnings.filterwarnings(\"ignore\", category=FutureWarning)\n\nimport numpy as np\nimport pandas as pd \nimport plotly.express as px\nimport matplotlib.pyplot as plt\nimport seaborn as sns\n%matplotlib inline\nplt.rcParams['font.size'] = 12\nplt.rcParams['figure.figsize'] = (15, 10)\n\nfrom sklearn.preprocessing import OneHotEncoder\nfrom sklearn.cluster import KMeans\nfrom imblearn.over_sampling  import RandomOverSampler\nfrom sklearn.preprocessing import StandardScaler\nfrom sklearn.model_selection import train_test_split\nfrom sklearn.metrics import confusion_matrix\nfrom sklearn.metrics import classification_report\nfrom sklearn.metrics import recall_score,precision_score,f1_score\n\nfrom sklearn.linear_model import LogisticRegression\nfrom sklearn.neighbors import KNeighborsClassifier\nfrom sklearn.svm import SVC\nimport xgboost as xgb\nfrom xgboost import XGBClassifier\nfrom sklearn.ensemble import RandomForestClassifier\n\nimport tensorflow\nfrom tensorflow import keras\nfrom tensorflow.keras import Sequential\nfrom tensorflow.keras.callbacks import EarlyStopping\nfrom tensorflow.keras.layers import Dense, Dropout\n\nimport os\nfor dirname, _, filenames in os.walk('/kaggle/input'):\n    for filename in filenames:\n        print(os.path.join(dirname, filename))","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","execution":{"iopub.status.busy":"2022-09-01T11:22:50.221109Z","iopub.execute_input":"2022-09-01T11:22:50.221560Z","iopub.status.idle":"2022-09-01T11:22:53.567009Z","shell.execute_reply.started":"2022-09-01T11:22:50.221472Z","shell.execute_reply":"2022-09-01T11:22:53.565928Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Keeping Memory usage in mind, we have taken the table that contains all the data and have used only 100k rows for our purposes.","metadata":{}},{"cell_type":"code","source":"complete_2019 = pd.read_csv('../input/big-data-derby-2022/nyra_2019_complete.csv', nrows= 100000)\ncomplete_2019.head()","metadata":{"execution":{"iopub.status.busy":"2022-09-01T11:22:53.568879Z","iopub.execute_input":"2022-09-01T11:22:53.569452Z","iopub.status.idle":"2022-09-01T11:22:53.752565Z","shell.execute_reply.started":"2022-09-01T11:22:53.569423Z","shell.execute_reply":"2022-09-01T11:22:53.751548Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"complete_2019.describe()","metadata":{"execution":{"iopub.status.busy":"2022-09-01T11:22:53.754025Z","iopub.execute_input":"2022-09-01T11:22:53.755007Z","iopub.status.idle":"2022-09-01T11:22:53.810857Z","shell.execute_reply.started":"2022-09-01T11:22:53.754970Z","shell.execute_reply":"2022-09-01T11:22:53.809785Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"complete_2019.info()","metadata":{"execution":{"iopub.status.busy":"2022-09-01T11:22:53.813545Z","iopub.execute_input":"2022-09-01T11:22:53.813922Z","iopub.status.idle":"2022-09-01T11:22:53.856443Z","shell.execute_reply.started":"2022-09-01T11:22:53.813881Z","shell.execute_reply":"2022-09-01T11:22:53.855404Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"fig = px.scatter(complete_2019, \n                   x='course_type',\n                   y='odds',\n                   color='odds',\n                   template='seaborn',\n                   labels={\"course_type\": \"Course Type\", \"odds\":\"Odds\"},\n                   title='Winning Odds and Course Type ')\n\nfig.update_layout(xaxis_title=\"Course Type \",yaxis_title=\"Odds\")\nfig.update_layout({'plot_bgcolor': 'rgba(0, 0, 0, 0)','paper_bgcolor': 'rgba(0, 0, 0, 0)'},)\n\nfig.show()","metadata":{"execution":{"iopub.status.busy":"2022-09-01T11:22:53.858477Z","iopub.execute_input":"2022-09-01T11:22:53.858917Z","iopub.status.idle":"2022-09-01T11:22:54.456122Z","shell.execute_reply.started":"2022-09-01T11:22:53.858875Z","shell.execute_reply":"2022-09-01T11:22:54.454834Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"From the plots above, the odds of winning a race are highest in Course Type D and T. If a jockey wants to compete he or she should enter a race with either of the two Course Type.","metadata":{}},{"cell_type":"code","source":"fig = px.scatter(complete_2019, \n                   y='purse',\n                   x='race_number',\n                 color='purse',\n                   template='ggplot2',\n                   labels={\"purse\": \"Purse\", 'race_number': \"Race Number\"},\n                   title='Purse and Race Number')\n\nfig.update_layout(yaxis_title=\"Purse ($)\",xaxis_title=\"Race Number\")\nfig.update_layout({'plot_bgcolor': 'rgba(0, 0, 0, 0)','paper_bgcolor': 'rgba(0, 0, 0, 0)'},)\n\nfig.show()","metadata":{"execution":{"iopub.status.busy":"2022-09-01T11:22:54.457390Z","iopub.execute_input":"2022-09-01T11:22:54.457719Z","iopub.status.idle":"2022-09-01T11:22:54.567412Z","shell.execute_reply.started":"2022-09-01T11:22:54.457688Z","shell.execute_reply":"2022-09-01T11:22:54.566574Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Purse may refer to the total amount of money paid out to the owners of horses racing at a particular track over a given period of time, or to the percentages of a race's total purse that are awarded to each of the highest finishers.\n\nFrom the chart above, its visible that Purse is the highest in race number 9 followed by race number 13 which are the best best for any horse owners to be mindful of while selecting a race.","metadata":{}},{"cell_type":"code","source":"fig = px.scatter(complete_2019, \n                   y='odds',\n                   x='race_number',\n                 color='odds',\n                   template='plotly_white',\n                   labels={\"odds\": \"Odds\", 'race_number': \"Race Number\"},\n                   title='Winning Odds and Race Number')\n\nfig.update_layout(yaxis_title=\"Odds\",xaxis_title=\"Race Number\")\nfig.update_layout({'plot_bgcolor': 'rgba(0, 0, 0, 0)','paper_bgcolor': 'rgba(0, 0, 0, 0)'},)\n\nfig.show()","metadata":{"execution":{"iopub.status.busy":"2022-09-01T11:22:54.568678Z","iopub.execute_input":"2022-09-01T11:22:54.569191Z","iopub.status.idle":"2022-09-01T11:22:54.660204Z","shell.execute_reply.started":"2022-09-01T11:22:54.569149Z","shell.execute_reply":"2022-09-01T11:22:54.658987Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"The odds of winning a race is the highest in the race number 3 and 6. A horse owner or a jockey are most likely to win in these races if they compete.","metadata":{}},{"cell_type":"code","source":"fig = px.scatter(complete_2019, \n                   y='odds',\n                   x='race_date',\n                 color='odds',\n                   template='ygridoff',\n                   labels={\"odds\": \"Odds\", 'race_date': \"Race Date\"},\n                   title='Winning Odds and Race Date')\n\nfig.update_layout(yaxis_title=\"Odds\",xaxis_title=\"Race Date\")\nfig.update_layout({'plot_bgcolor': 'rgba(0, 0, 0, 0)','paper_bgcolor': 'rgba(0, 0, 0, 0)'},)\n\nfig.show()","metadata":{"execution":{"iopub.status.busy":"2022-09-01T11:22:54.661918Z","iopub.execute_input":"2022-09-01T11:22:54.662264Z","iopub.status.idle":"2022-09-01T11:22:55.053075Z","shell.execute_reply.started":"2022-09-01T11:22:54.662231Z","shell.execute_reply":"2022-09-01T11:22:55.051962Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"The odds of winning a race is highest in March and November month of the year.","metadata":{}},{"cell_type":"code","source":"fig = px.scatter(complete_2019, \n                   x='race_number',\n                   y='run_up_distance',\n                 color='run_up_distance',\n                   template='ggplot2',\n                   labels={\"race_number\": \"Race Number\", 'run_up_distance': \"Run Up Distance\"},\n                   title='Run Up Distance and Race Number')\n\nfig.update_layout(yaxis_title=\"Run Up Distance\",xaxis_title=\"Race Number\")\nfig.update_layout({'plot_bgcolor': 'rgba(0, 0, 0, 0)','paper_bgcolor': 'rgba(0, 0, 0, 0)'},)\n\nfig.show()","metadata":{"execution":{"iopub.status.busy":"2022-09-01T11:22:55.054608Z","iopub.execute_input":"2022-09-01T11:22:55.055173Z","iopub.status.idle":"2022-09-01T11:22:55.163892Z","shell.execute_reply.started":"2022-09-01T11:22:55.055136Z","shell.execute_reply":"2022-09-01T11:22:55.163021Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Run Up Distance is the distance in feet of the gate to the start of the race. In race number 10 and 8, its the highest. The lowest the run up distance, the better.","metadata":{}},{"cell_type":"code","source":"fig = px.histogram(complete_2019, \n                   y='weight_carried',\n                   x=\"track_condition\",\n                   color='track_condition',\n                   log_y=True,\n                   text_auto=True,\n                   template='simple_white',\n                   labels={\"weight_carried\": \"Weight Carried\", \"track_condition\":\"Track Condition\"},\n                   title='Weight Carried and Track Condition')\n\nfig.update_layout(yaxis_title=\"Weight Carried \",xaxis_title=\"Track Condition\")\n\nfig.update_xaxes(categoryorder='total descending')\n                 \nfig.show()","metadata":{"execution":{"iopub.status.busy":"2022-09-01T11:22:55.169132Z","iopub.execute_input":"2022-09-01T11:22:55.169800Z","iopub.status.idle":"2022-09-01T11:22:55.606620Z","shell.execute_reply.started":"2022-09-01T11:22:55.169762Z","shell.execute_reply":"2022-09-01T11:22:55.605719Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Weight Carried by the horse is the highest in the FT track\n\n* YL - Yielding\n* FM - Firm\n* SY - Sloppy\n* GD- Good\n* FT -  Fast\n* MY - Muddy\n* SF - Soft","metadata":{}},{"cell_type":"code","source":"print('The Average Weight Carried is',np.around(np.mean(complete_2019['weight_carried']),2))","metadata":{"execution":{"iopub.status.busy":"2022-09-01T11:22:55.607990Z","iopub.execute_input":"2022-09-01T11:22:55.608988Z","iopub.status.idle":"2022-09-01T11:22:55.615909Z","shell.execute_reply.started":"2022-09-01T11:22:55.608951Z","shell.execute_reply":"2022-09-01T11:22:55.614693Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"fig = px.scatter(complete_2019, \n                   x='race_type',\n                   y=\"weight_carried\",\n                   color='race_type',\n                   template='simple_white',\n                   labels={\"race_type\": \"Race Type\", \"weight_carried\":\"Weight Carried\"},\n                   title='Race Type and Weight Carried')\n\nfig.update_layout(xaxis_title=\"Race Type \",yaxis_title=\"Weight Carried\")\n\nfig.show()","metadata":{"execution":{"iopub.status.busy":"2022-09-01T11:22:55.617401Z","iopub.execute_input":"2022-09-01T11:22:55.618786Z","iopub.status.idle":"2022-09-01T11:22:56.053496Z","shell.execute_reply.started":"2022-09-01T11:22:55.618732Z","shell.execute_reply":"2022-09-01T11:22:56.052664Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Weight Carried by the horse is the highest in the STK race type\n \n- STK - Stakes\n- WCL - Waiver Claiming\n- WMC - Waiver Maiden Claiming\n- SST - Starter Stakes\n- SHP - Starter Handicap\n- CLM - Claiming\n- STR - Starter Allowance\n- AOC - Allowance Optionl Claimer\n- SOC - Starter Optional Claimer\n- MCL - Maiden Claiming\n- ALW - Allowance\n- MSW - Maiden Special Weight","metadata":{}},{"cell_type":"code","source":"fig = px.histogram(complete_2019, \n                   y='run_up_distance',\n                   x='track_condition',\n                   color='track_condition',\n                   log_y=True,\n                   text_auto=True,\n                   template='simple_white',\n                   labels={\"run_up_distance\": \"Run Up Distance\", \"track_condition\":\"Track Condition\"},\n                   title='Run Up Distance and Track Condition')\n\nfig.update_layout(xaxis_title=\"Track Condition \",yaxis_title=\"Run Up Distance\", bargap=0.1)\n\nfig.show()","metadata":{"execution":{"iopub.status.busy":"2022-09-01T11:22:56.054933Z","iopub.execute_input":"2022-09-01T11:22:56.055501Z","iopub.status.idle":"2022-09-01T11:22:56.495027Z","shell.execute_reply.started":"2022-09-01T11:22:56.055466Z","shell.execute_reply":"2022-09-01T11:22:56.493977Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"sns.pairplot(complete_2019, corner=True)\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-09-01T11:22:56.496667Z","iopub.execute_input":"2022-09-01T11:22:56.497021Z","iopub.status.idle":"2022-09-01T11:23:18.115648Z","shell.execute_reply.started":"2022-09-01T11:22:56.496988Z","shell.execute_reply":"2022-09-01T11:23:18.114793Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"for i in complete_2019:\n    fig = px.histogram(complete_2019, \n                   x=i,\n                y=\"run_up_distance\",\n                    color_discrete_sequence=[\"indianred\"],\n                   text_auto=True,\n                   template='simple_white')\n    fig.update_layout(xaxis_title=i,yaxis_title=\"Run Up Distance\", bargap=0.1)\n    fig.show()","metadata":{"execution":{"iopub.status.busy":"2022-09-01T11:23:18.117184Z","iopub.execute_input":"2022-09-01T11:23:18.117844Z","iopub.status.idle":"2022-09-01T11:23:21.607799Z","shell.execute_reply.started":"2022-09-01T11:23:18.117800Z","shell.execute_reply":"2022-09-01T11:23:21.606824Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Above is the comparison between Run Up Distance and the other columns in the dataframe.","metadata":{}},{"cell_type":"markdown","source":"## Data Preprocessing","metadata":{}},{"cell_type":"code","source":"complete_2019['race_date'] = pd.to_datetime(complete_2019['race_date'])\ncomplete_2019.drop('jockey', axis=1, inplace=True)","metadata":{"execution":{"iopub.status.busy":"2022-09-01T11:23:21.609354Z","iopub.execute_input":"2022-09-01T11:23:21.609971Z","iopub.status.idle":"2022-09-01T11:23:21.650099Z","shell.execute_reply.started":"2022-09-01T11:23:21.609935Z","shell.execute_reply":"2022-09-01T11:23:21.649000Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"complete_2019 = complete_2019.merge(pd.get_dummies(complete_2019['program_number']), \n                                    how= 'inner', \n                                    left_index=True, \n                                    right_index=True)\ncomplete_2019.drop('program_number', axis=1, inplace=True)\ncomplete_2019.rename(columns={\n                              '10 ': \"program_number10\", '11 ': \"program_number11\",'12 ':'program_number12', \n                              '13 ': 'program_number13', '1A ': 'program_number1A','2B ': 'program_number2B', \n                              '2  ': 'program_number2',  '3  ': 'program_number3', '4  ': 'program_number4', \n                              '5  ':'program_number5', '6  ': 'program_number6','7  ':'program_number7', \n                              '8  ': 'program_number8', '9  ':'program_number9', '1  ': 'program_number1',\n                             '1 ': \"program_number1\", '2 ': \"program_number_2\",'12 ':'program_number12', \n                              '4 ': 'program_number4', '12 ': 'program_number12'\n                             }, inplace=True)","metadata":{"execution":{"iopub.status.busy":"2022-09-01T11:23:21.651897Z","iopub.execute_input":"2022-09-01T11:23:21.652567Z","iopub.status.idle":"2022-09-01T11:23:21.711254Z","shell.execute_reply.started":"2022-09-01T11:23:21.652529Z","shell.execute_reply":"2022-09-01T11:23:21.710251Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"input_cols = []\nfor i in complete_2019:\n    if i != 'Target':\n        input_cols.append(i)","metadata":{"execution":{"iopub.status.busy":"2022-09-01T11:23:21.713742Z","iopub.execute_input":"2022-09-01T11:23:21.714469Z","iopub.status.idle":"2022-09-01T11:23:21.719891Z","shell.execute_reply.started":"2022-09-01T11:23:21.714429Z","shell.execute_reply":"2022-09-01T11:23:21.718913Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"categorical_cols = complete_2019[input_cols].select_dtypes(include='object').columns.tolist()\ncategorical_cols","metadata":{"execution":{"iopub.status.busy":"2022-09-01T11:23:21.721776Z","iopub.execute_input":"2022-09-01T11:23:21.722220Z","iopub.status.idle":"2022-09-01T11:23:21.745349Z","shell.execute_reply.started":"2022-09-01T11:23:21.722109Z","shell.execute_reply":"2022-09-01T11:23:21.744444Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"encoder = OneHotEncoder(sparse=False, handle_unknown='ignore').fit(complete_2019[categorical_cols])","metadata":{"execution":{"iopub.status.busy":"2022-09-01T11:23:21.746495Z","iopub.execute_input":"2022-09-01T11:23:21.746795Z","iopub.status.idle":"2022-09-01T11:23:21.768941Z","shell.execute_reply.started":"2022-09-01T11:23:21.746760Z","shell.execute_reply":"2022-09-01T11:23:21.768078Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"encoded_cols = list(encoder.get_feature_names(categorical_cols))\ncomplete_2019[encoded_cols] = encoder.transform(complete_2019[categorical_cols])\ncomplete_2019.drop(categorical_cols, axis=1, inplace=True)\ncomplete_2019.head()","metadata":{"execution":{"iopub.status.busy":"2022-09-01T11:23:21.771720Z","iopub.execute_input":"2022-09-01T11:23:21.772000Z","iopub.status.idle":"2022-09-01T11:23:21.946974Z","shell.execute_reply.started":"2022-09-01T11:23:21.771976Z","shell.execute_reply":"2022-09-01T11:23:21.945821Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def split_date(df):\n    df['Year'] = df.race_date.dt.year\n    df['Month'] = df.race_date.dt.month\n    df['Day'] = df.race_date.dt.day","metadata":{"execution":{"iopub.status.busy":"2022-09-01T11:23:21.948627Z","iopub.execute_input":"2022-09-01T11:23:21.949291Z","iopub.status.idle":"2022-09-01T11:23:21.955592Z","shell.execute_reply.started":"2022-09-01T11:23:21.949247Z","shell.execute_reply":"2022-09-01T11:23:21.954584Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"split_date(complete_2019)\ncomplete_2019.drop(['race_date'], axis=1, inplace=True)","metadata":{"execution":{"iopub.status.busy":"2022-09-01T11:23:21.957264Z","iopub.execute_input":"2022-09-01T11:23:21.957658Z","iopub.status.idle":"2022-09-01T11:23:22.007467Z","shell.execute_reply.started":"2022-09-01T11:23:21.957622Z","shell.execute_reply":"2022-09-01T11:23:22.006458Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"complete_2019.head()","metadata":{"execution":{"iopub.status.busy":"2022-09-01T11:23:22.009162Z","iopub.execute_input":"2022-09-01T11:23:22.009558Z","iopub.status.idle":"2022-09-01T11:23:22.033471Z","shell.execute_reply.started":"2022-09-01T11:23:22.009522Z","shell.execute_reply":"2022-09-01T11:23:22.032165Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Clustering","metadata":{}},{"cell_type":"code","source":"wcss = []\n\nfor i in range(1,11):\n    km = KMeans(n_clusters=i)\n    km.fit_predict(complete_2019)\n    wcss.append(km.inertia_)\n    print('For', i,'Cluster,', np.around(km.inertia_,2),'is the WCSS.')","metadata":{"execution":{"iopub.status.busy":"2022-09-01T11:23:22.035288Z","iopub.execute_input":"2022-09-01T11:23:22.035886Z","iopub.status.idle":"2022-09-01T11:23:39.262224Z","shell.execute_reply.started":"2022-09-01T11:23:22.035853Z","shell.execute_reply":"2022-09-01T11:23:39.261114Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"fig = px.line(x = range(1,11),\n              y=wcss, \n              template='simple_white',\n              title = 'Elbow Plot', markers=True,\n             color_discrete_sequence=[\"skyblue\"])\n\nfig.update_layout(xaxis_title=\"Clusters\",yaxis_title=\" WCSS\")\n\nfig.show()","metadata":{"execution":{"iopub.status.busy":"2022-09-01T11:23:39.265215Z","iopub.execute_input":"2022-09-01T11:23:39.265544Z","iopub.status.idle":"2022-09-01T11:23:39.327006Z","shell.execute_reply.started":"2022-09-01T11:23:39.265514Z","shell.execute_reply":"2022-09-01T11:23:39.326084Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"km = KMeans(n_clusters=6)\ny_means = km.fit_predict(complete_2019)\ny_means","metadata":{"execution":{"iopub.status.busy":"2022-09-01T11:23:39.328503Z","iopub.execute_input":"2022-09-01T11:23:39.329102Z","iopub.status.idle":"2022-09-01T11:23:41.241545Z","shell.execute_reply.started":"2022-09-01T11:23:39.329063Z","shell.execute_reply":"2022-09-01T11:23:41.240769Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"We are now creating a target variable to see if there will be a horse injury or not. 0 here is no injury and 5 is highly likely to be injured.","metadata":{}},{"cell_type":"code","source":"complete_2019['Target'] = y_means\nX = complete_2019[complete_2019.columns[:-1]].values\ny = complete_2019[complete_2019.columns[-1]].values","metadata":{"execution":{"iopub.status.busy":"2022-09-01T11:23:41.248367Z","iopub.execute_input":"2022-09-01T11:23:41.248677Z","iopub.status.idle":"2022-09-01T11:23:41.297627Z","shell.execute_reply.started":"2022-09-01T11:23:41.248648Z","shell.execute_reply":"2022-09-01T11:23:41.296293Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Scaling","metadata":{}},{"cell_type":"code","source":"scaler = StandardScaler()\nX= scaler.fit_transform(X)","metadata":{"execution":{"iopub.status.busy":"2022-09-01T11:23:41.299445Z","iopub.execute_input":"2022-09-01T11:23:41.304455Z","iopub.status.idle":"2022-09-01T11:23:41.388023Z","shell.execute_reply.started":"2022-09-01T11:23:41.304421Z","shell.execute_reply":"2022-09-01T11:23:41.386865Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## PCA ","metadata":{}},{"cell_type":"code","source":"from sklearn import decomposition \npca = decomposition.PCA(0.90)\nprint(\"Shape of X before PCA -\", X.shape)\nX = pca.fit_transform(X)\nprint(\"Shape of X after PCA -\", X.shape)","metadata":{"execution":{"iopub.status.busy":"2022-09-01T11:23:41.390911Z","iopub.execute_input":"2022-09-01T11:23:41.391835Z","iopub.status.idle":"2022-09-01T11:23:41.820011Z","shell.execute_reply.started":"2022-09-01T11:23:41.391791Z","shell.execute_reply":"2022-09-01T11:23:41.818829Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"As we can see only 33 features have explained the 90% of the variance in our data which is exactly what we wanted.","metadata":{}},{"cell_type":"code","source":"fig = px.scatter(\n                   x=X[:,0],\n                   y=X[:,1],\n                   template='simple_white',\n                   color_discrete_sequence=px.colors.qualitative.G10,\n                   title='PCA Scatterplot'\n                 )\n\nfig.update_layout(xaxis_title=\"PC1 \",yaxis_title=\"PC2\")\n\nfig.show()","metadata":{"execution":{"iopub.status.busy":"2022-09-01T11:23:41.821922Z","iopub.execute_input":"2022-09-01T11:23:41.822625Z","iopub.status.idle":"2022-09-01T11:23:41.979295Z","shell.execute_reply.started":"2022-09-01T11:23:41.822588Z","shell.execute_reply":"2022-09-01T11:23:41.972834Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Oversampling","metadata":{}},{"cell_type":"code","source":"oversample = RandomOverSampler(sampling_strategy='minority')\nX, y = oversample.fit_resample(X, y)","metadata":{"execution":{"iopub.status.busy":"2022-09-01T11:23:41.980804Z","iopub.execute_input":"2022-09-01T11:23:41.981401Z","iopub.status.idle":"2022-09-01T11:23:42.047511Z","shell.execute_reply.started":"2022-09-01T11:23:41.981365Z","shell.execute_reply":"2022-09-01T11:23:42.046507Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print('X Shape -', X.shape)\nprint('y Shape -',y.shape)","metadata":{"execution":{"iopub.status.busy":"2022-09-01T11:23:42.049049Z","iopub.execute_input":"2022-09-01T11:23:42.049656Z","iopub.status.idle":"2022-09-01T11:23:42.055445Z","shell.execute_reply.started":"2022-09-01T11:23:42.049621Z","shell.execute_reply":"2022-09-01T11:23:42.054101Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Training Model","metadata":{}},{"cell_type":"code","source":"X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.33, random_state=42)","metadata":{"execution":{"iopub.status.busy":"2022-09-01T11:23:42.057345Z","iopub.execute_input":"2022-09-01T11:23:42.058580Z","iopub.status.idle":"2022-09-01T11:23:42.098083Z","shell.execute_reply.started":"2022-09-01T11:23:42.058535Z","shell.execute_reply":"2022-09-01T11:23:42.096940Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print(X_train.shape[0], 'rows for training.')\nprint(X_test.shape[0], 'rows for validation.')","metadata":{"execution":{"iopub.status.busy":"2022-09-01T11:23:42.099540Z","iopub.execute_input":"2022-09-01T11:23:42.100563Z","iopub.status.idle":"2022-09-01T11:23:42.107643Z","shell.execute_reply.started":"2022-09-01T11:23:42.100521Z","shell.execute_reply":"2022-09-01T11:23:42.106482Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"names = ['Logistic Regression', \"KNN\", \"Linear_SVM\", \"XGBClassifier\", \"Random Forest\"]\nclassifiers = [\n    LogisticRegression(solver='liblinear', penalty='l2', n_jobs=-1, random_state=0),\n    KNeighborsClassifier(n_neighbors=3),\n    SVC(kernel=\"linear\", C=0.025),\n    XGBClassifier(objective='binary:hinge'),\n    RandomForestClassifier(max_depth=5, n_estimators=100)]\n\nmodelscores = []\nfor name, clf in zip(names, classifiers):\n    clf.fit(X_train, y_train)\n    score = clf.score(X_test, y_test)\n    modelscores.append(score)\n    \nscores_df = pd.DataFrame()\nscores_df['name'] = names\nscores_df['score'] = modelscores\nscores_df.sort_values('score', ascending= False)","metadata":{"execution":{"iopub.status.busy":"2022-09-01T11:23:42.109596Z","iopub.execute_input":"2022-09-01T11:23:42.110309Z","iopub.status.idle":"2022-09-01T11:28:27.692059Z","shell.execute_reply.started":"2022-09-01T11:23:42.110274Z","shell.execute_reply":"2022-09-01T11:28:27.690720Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"for name, clf in zip(names, classifiers):\n    clf.fit(X_train, y_train)\n    y_pred = clf.predict(X_test)\n    print(name, \"Precision - \",precision_score(y_test, y_pred, average='weighted'))\n    print(name, \"Recall - \",recall_score(y_test,y_pred, average='weighted'))\n    print(name, \"F1 score - \",f1_score(y_test,y_pred, average='weighted'))","metadata":{"execution":{"iopub.status.busy":"2022-09-01T11:38:01.751457Z","iopub.execute_input":"2022-09-01T11:38:01.751871Z","iopub.status.idle":"2022-09-01T11:42:47.670952Z","shell.execute_reply.started":"2022-09-01T11:38:01.751837Z","shell.execute_reply":"2022-09-01T11:42:47.669815Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Deep Learning Framework\n\nWe are using a DL framework to predict if we can tell given the features, if there is going to be an injury or not. If the business requirement is more explainability then one can always move back to Machine Learning Classifiers where there are more ways to explain the model and the importance of a certain features.","metadata":{}},{"cell_type":"code","source":"model = Sequential()\nmodel.add(Dense(33,activation='relu',input_dim=33)) #Input layer\nmodel.add(Dropout(0.5))\nmodel.add(Dense(15,activation='relu'))\nmodel.add(Dropout(0.5))\nmodel.add(Dense(1,activation='sigmoid'))\nmodel.summary()","metadata":{"execution":{"iopub.status.busy":"2022-09-01T11:42:47.672728Z","iopub.execute_input":"2022-09-01T11:42:47.673037Z","iopub.status.idle":"2022-09-01T11:42:47.711807Z","shell.execute_reply.started":"2022-09-01T11:42:47.673011Z","shell.execute_reply":"2022-09-01T11:42:47.710729Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"optimizer = keras.optimizers.Adam(learning_rate=0.0001)\ncallback = EarlyStopping(\n    monitor=\"val_loss\",\n    min_delta=0.0001,\n    patience=5,\n    verbose=1,\n    mode=\"auto\",\n    baseline=None,\n    restore_best_weights=False\n)\nmodel.compile(loss='binary_crossentropy',optimizer=optimizer,metrics=['accuracy'])","metadata":{"execution":{"iopub.status.busy":"2022-09-01T11:42:47.713246Z","iopub.execute_input":"2022-09-01T11:42:47.713589Z","iopub.status.idle":"2022-09-01T11:42:47.724834Z","shell.execute_reply.started":"2022-09-01T11:42:47.713554Z","shell.execute_reply":"2022-09-01T11:42:47.723855Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"history = model.fit(X_train,y_train,batch_size=200, epochs=100,\n                    validation_data = (X_test,y_test), verbose=1, callbacks=callback)","metadata":{"execution":{"iopub.status.busy":"2022-09-01T11:42:47.727803Z","iopub.execute_input":"2022-09-01T11:42:47.728245Z","iopub.status.idle":"2022-09-01T11:45:09.612083Z","shell.execute_reply.started":"2022-09-01T11:42:47.728204Z","shell.execute_reply":"2022-09-01T11:45:09.611116Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"fig = px.line(x = range(len(history.history['loss'])),\n              y=history.history['loss'], template='simple_white',title = 'Training Loss Plot', markers=True)\n\nfig.update_layout(xaxis_title=\"Epochs\",yaxis_title=\" Training Loss\")\n\nfig1 = px.line(x = range(len(history.history['val_loss'])),\n               y=history.history['val_loss'], template='simple_white',\n               color_discrete_sequence=[\"red\"],\n               title = 'Validation Loss Plot', markers=True)\n\nfig1.update_layout(xaxis_title=\"Epochs\",yaxis_title=\" Validation Loss\")\n\nfig.show()\nfig1.show()","metadata":{"execution":{"iopub.status.busy":"2022-09-01T11:45:09.613506Z","iopub.execute_input":"2022-09-01T11:45:09.613882Z","iopub.status.idle":"2022-09-01T11:45:09.710213Z","shell.execute_reply.started":"2022-09-01T11:45:09.613845Z","shell.execute_reply":"2022-09-01T11:45:09.709143Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"fig = px.line(x = range(len(history.history['accuracy'])),\n              y=history.history['accuracy'], \n              template='simple_white',\n              title = 'Training Accuracy Plot', \n              #text=[ '%.3f' % elem for elem in history.history['accuracy'] ],\n              markers=True)\n\nfig.update_layout(xaxis_title=\"Epochs\",yaxis_title=\" Accuracy\")\n\nfig.update_traces(textposition=\"top center\")\n\n#2nd Plot\nfig1 = px.line(x= range(len(history.history['val_accuracy'])), \n    y= history.history['val_accuracy'], \n               template='simple_white',\n               title = 'Validation Accuracy Plot', \n               color_discrete_sequence=[\"indianred\"],\n               markers=True)\n\nfig1.update_layout(xaxis_title=\"Epochs\",yaxis_title=\"Validation Accuracy\")\nfig1.update_traces(textposition=\"top center\")\n\nfig.show()\nfig1.show()","metadata":{"execution":{"iopub.status.busy":"2022-09-01T11:45:09.711865Z","iopub.execute_input":"2022-09-01T11:45:09.712298Z","iopub.status.idle":"2022-09-01T11:45:09.805033Z","shell.execute_reply.started":"2022-09-01T11:45:09.712262Z","shell.execute_reply":"2022-09-01T11:45:09.803975Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"y_prob = model.predict(X_test)\npreds = y_prob.argmax(axis=1)\nprint(classification_report(y_test, preds))","metadata":{"execution":{"iopub.status.busy":"2022-09-01T11:45:09.806617Z","iopub.execute_input":"2022-09-01T11:45:09.806974Z","iopub.status.idle":"2022-09-01T11:45:11.195955Z","shell.execute_reply.started":"2022-09-01T11:45:09.806937Z","shell.execute_reply":"2022-09-01T11:45:11.194992Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"confusionmatrix = np.around(confusion_matrix(y_test, preds, normalize='true'),2)\n\nfig = px.imshow(confusionmatrix, \n                text_auto=True, \n                color_continuous_scale=px.colors.sequential.Viridis, \n                aspect=\"auto\", \n                labels=dict(x=\"Predicted Label\", y=\"Correct Label\"))\n\nfig.update_xaxes(side=\"top\")\n\nfig.show()","metadata":{"execution":{"iopub.status.busy":"2022-09-01T11:45:11.197706Z","iopub.execute_input":"2022-09-01T11:45:11.198102Z","iopub.status.idle":"2022-09-01T11:45:11.248566Z","shell.execute_reply.started":"2022-09-01T11:45:11.198063Z","shell.execute_reply":"2022-09-01T11:45:11.247075Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"We can clearly see that the Neural Network is performing terribly on the data while the algorithms like KNN and XGBoost is performing well.","metadata":{}}]}