{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# Derby 2022\nAnalyze horse racing data to improve the health of the horse and strategy of competition","metadata":{}},{"cell_type":"markdown","source":"# Importing Libraries","metadata":{}},{"cell_type":"code","source":"import numpy as np\nimport pandas as pd\nimport seaborn as sns\n\nimport matplotlib.pyplot as plt\n%matplotlib inline\n\nimport warnings \nwarnings.filterwarnings('ignore')\n","metadata":{"execution":{"iopub.status.busy":"2022-08-22T17:05:28.793510Z","iopub.execute_input":"2022-08-22T17:05:28.794126Z","iopub.status.idle":"2022-08-22T17:05:29.466613Z","shell.execute_reply.started":"2022-08-22T17:05:28.794065Z","shell.execute_reply":"2022-08-22T17:05:29.465116Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Loading Data","metadata":{}},{"cell_type":"markdown","source":"### File descriptions\n* nyra_start_table.csv - horse/jockey race data\n* nyra_race_table.csv - racetrack race data\n* nyra_tracking_table.csv - tracking data\n* nyra_2019_complete.csv - combined table of three above files","metadata":{}},{"cell_type":"code","source":"# raceData = pd.read_csv('./input/nyra_start_table.csv', parse_dates=['race_date'])\n# racetrackData = pd.read_csv('./input/nyra_race_table.csv', parse_dates=['race_date'])\n# trackingData = pd.read_csv('./input/nyra_tracking_table.csv', parse_dates=['race_date'])\n","metadata":{"execution":{"iopub.status.busy":"2022-08-22T17:05:29.469997Z","iopub.execute_input":"2022-08-22T17:05:29.471177Z","iopub.status.idle":"2022-08-22T17:05:29.476777Z","shell.execute_reply.started":"2022-08-22T17:05:29.471124Z","shell.execute_reply":"2022-08-22T17:05:29.475301Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"completeData = pd.read_csv('/kaggle/input/big-data-derby-2022/nyra_2019_complete.csv', parse_dates=['race_date'], dtype={ 'post_time': str})\ncompleteData.head()","metadata":{"execution":{"iopub.status.busy":"2022-08-22T17:06:35.281670Z","iopub.execute_input":"2022-08-22T17:06:35.282079Z","iopub.status.idle":"2022-08-22T17:06:51.626976Z","shell.execute_reply.started":"2022-08-22T17:06:35.282046Z","shell.execute_reply":"2022-08-22T17:06:51.625702Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"nyra_2019_complete.csv - This file is the combined 3 files into one table. The keys to join them trakus with race - track_id, race_date, race_number. To join trakus with start - track_id, race_date, race_number, program_number.\n\n* track_id - 3 character id for the track the race took place at. \n
\n \n
\n* race_date - date the race took place (YYYY-MM-DD)\n* race_number - Number of the race. Passed as 3 characters but can be cast or converted to int for this data set.\n* program_number - Program number of the horse in the race passed as 3 characters. Should remain 3 characters as it isn't limited to just numbers. Is essentially the unique identifier of the horse in the race.\n* weight_carried - An integer of the weight carried by the horse in the race.\n* jockey - Name of the jockey on the horse in the race. 50 character max.\n* odds - Odds to win the race passed as an integer. Divide by 100 to derive the odds to 1. Example - 1280 would be 12.8-1.\n* distance_id - Distance of the race in furlongs passed as an integer. Example - 600 would be 6 furlongs.\n* course_type - The course the race was run over passed as one character.\n
\n \n
\n* track_condition - The condition of the course the race was run on passed as three characters. \n
\n \n
\n* run_up_distance - Distance in feet of the gate to the start of the race passed as an integer.\n* race_type - The classification of the race passed as as five characters. \n
\n \n
\n* purse - Purse in US dollars of the race passed as an money with two decimal places.\n* post_time - Time of day the race began passed as 5 character. Example - 01220 would be 12:20.\n* trakus_index - The common collection of point of the lat / long of the horse in the race passed as an integer. From what we can tell, it's collected every 0.25 seconds.\n* latitude - The latitude of the horse in the race passed as a float.\n* longitude - The longitude of the horse in the race passed as a float.\n","metadata":{}},{"cell_type":"code","source":"completeData.info()","metadata":{"execution":{"iopub.status.busy":"2022-08-22T17:06:51.654924Z","iopub.execute_input":"2022-08-22T17:06:51.655853Z","iopub.status.idle":"2022-08-22T17:06:51.672643Z","shell.execute_reply.started":"2022-08-22T17:06:51.655802Z","shell.execute_reply":"2022-08-22T17:06:51.671412Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"completeData.describe()","metadata":{"execution":{"iopub.status.busy":"2022-08-22T17:06:52.929603Z","iopub.execute_input":"2022-08-22T17:06:52.930062Z","iopub.status.idle":"2022-08-22T17:06:54.182784Z","shell.execute_reply.started":"2022-08-22T17:06:52.930025Z","shell.execute_reply":"2022-08-22T17:06:54.181538Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Data Preprocessing","metadata":{}},{"cell_type":"code","source":"completeData.isna().sum()","metadata":{"execution":{"iopub.status.busy":"2022-08-22T17:07:02.163183Z","iopub.execute_input":"2022-08-22T17:07:02.164776Z","iopub.status.idle":"2022-08-22T17:07:04.891225Z","shell.execute_reply.started":"2022-08-22T17:07:02.164721Z","shell.execute_reply":"2022-08-22T17:07:04.889823Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"completeData['odds'] = completeData['odds']/100\ncompleteData['post_time'] = completeData['post_time'].apply(lambda x:''.join([x[:3].lstrip(\"0\"), ':', x[3:]]))\n","metadata":{"execution":{"iopub.status.busy":"2022-08-22T17:07:04.892999Z","iopub.execute_input":"2022-08-22T17:07:04.893336Z","iopub.status.idle":"2022-08-22T17:07:09.081538Z","shell.execute_reply.started":"2022-08-22T17:07:04.893306Z","shell.execute_reply":"2022-08-22T17:07:09.080232Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"completeData.head()","metadata":{"execution":{"iopub.status.busy":"2022-08-22T17:07:09.083571Z","iopub.execute_input":"2022-08-22T17:07:09.083949Z","iopub.status.idle":"2022-08-22T17:07:09.107256Z","shell.execute_reply.started":"2022-08-22T17:07:09.083906Z","shell.execute_reply":"2022-08-22T17:07:09.105809Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Categorical Data","metadata":{}},{"cell_type":"code","source":"completeData['track_id'] = completeData['track_id'].astype('category')\ncompleteData['course_type'] = completeData['course_type'].astype('category')\ncompleteData['track_condition'] = completeData['track_condition'].astype('category')\ncompleteData['race_type'] = completeData['race_type'].astype('category')","metadata":{"execution":{"iopub.status.busy":"2022-08-22T17:07:12.638245Z","iopub.execute_input":"2022-08-22T17:07:12.638660Z","iopub.status.idle":"2022-08-22T17:07:15.473266Z","shell.execute_reply.started":"2022-08-22T17:07:12.638623Z","shell.execute_reply":"2022-08-22T17:07:15.472204Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Data Exploration and Analysis","metadata":{}},{"cell_type":"code","source":"sns.stripplot(x = 'track_id', y = 'race_date', data = completeData )\nplt.grid(True)","metadata":{"execution":{"iopub.status.busy":"2022-08-22T17:07:20.368886Z","iopub.execute_input":"2022-08-22T17:07:20.369280Z","iopub.status.idle":"2022-08-22T17:07:47.445484Z","shell.execute_reply.started":"2022-08-22T17:07:20.369249Z","shell.execute_reply":"2022-08-22T17:07:47.444555Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"columnsName = ['track_id', 'race_number', 'program_number',\n 'course_type', 'track_condition',\n 'race_type' ]\nplt.figure(figsize=(20, 17))\nfor i, c in enumerate(columnsName):\n plt.subplot(3, 2, i+1)\n plt.title(columnsName[i])\n sns.countplot(completeData[c])","metadata":{"execution":{"iopub.status.busy":"2022-08-22T17:07:52.629116Z","iopub.execute_input":"2022-08-22T17:07:52.629486Z","iopub.status.idle":"2022-08-22T17:07:58.000206Z","shell.execute_reply.started":"2022-08-22T17:07:52.629452Z","shell.execute_reply":"2022-08-22T17:07:57.998926Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"columnsName = ['distance_id', 'run_up_distance', 'purse', 'weight_carried', 'odds'] \nplt.figure(figsize=(20, 10))\nfor i, c in enumerate(columnsName):\n plt.subplot(2, 3, i+1)\n plt.title(columnsName[i])\n sns.distplot(completeData[c])","metadata":{"execution":{"iopub.status.busy":"2022-08-22T17:11:23.626603Z","iopub.execute_input":"2022-08-22T17:11:23.627243Z","iopub.status.idle":"2022-08-22T17:13:06.589737Z","shell.execute_reply.started":"2022-08-22T17:11:23.627206Z","shell.execute_reply":"2022-08-22T17:13:06.588367Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plt.figure(figsize=(20, 10))\nfor i, c in enumerate(columnsName):\n plt.subplot(2, 3, i+1)\n plt.title(columnsName[i])\n sns.boxplot(completeData[c], palette='PRGn')","metadata":{"execution":{"iopub.status.busy":"2022-08-22T17:13:15.250898Z","iopub.execute_input":"2022-08-22T17:13:15.251601Z","iopub.status.idle":"2022-08-22T17:13:19.823609Z","shell.execute_reply.started":"2022-08-22T17:13:15.251545Z","shell.execute_reply":"2022-08-22T17:13:19.822379Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Track visualization","metadata":{}},{"cell_type":"code","source":"plt.figure(figsize = (15, 25))\nfor i, trackID in enumerate(completeData['track_id'].unique()):\n plt.subplot(3, 1, i+1)\n sns.scatterplot(data = completeData[completeData['track_id'] == trackID], x = \"longitude\", y = \"latitude\", hue = \"race_number\", palette = \"Paired\")\n plt.title(f\"{trackID} track visualization by race\")\n plt.legend()","metadata":{"execution":{"iopub.status.busy":"2022-08-22T17:15:52.473135Z","iopub.execute_input":"2022-08-22T17:15:52.473513Z","iopub.status.idle":"2022-08-22T17:18:26.299084Z","shell.execute_reply.started":"2022-08-22T17:15:52.473478Z","shell.execute_reply":"2022-08-22T17:18:26.297923Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Correlation Heatmap","metadata":{}},{"cell_type":"code","source":"plt.figure(figsize=(15,10))\nheatmap = sns.heatmap(completeData.corr(), cmap = \"Blues\", annot=True, linewidth=3)\nheatmap.set_title('Correlation Heatmap', fontdict={'fontsize':12}, pad=12)","metadata":{"execution":{"iopub.status.busy":"2022-08-22T17:18:35.107715Z","iopub.execute_input":"2022-08-22T17:18:35.108420Z","iopub.status.idle":"2022-08-22T17:18:37.234150Z","shell.execute_reply.started":"2022-08-22T17:18:35.108380Z","shell.execute_reply":"2022-08-22T17:18:37.232884Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Modeling","metadata":{}},{"cell_type":"markdown","source":"### Train Test Split","metadata":{}},{"cell_type":"code","source":"from sklearn.model_selection import train_test_split\n\n\nfeatures = ['track_id', 'race_number', 'race_date', 'program_number', 'trakus_index',\n 'course_type', 'track_condition', 'race_type', 'latitude', 'longitude',\n 'distance_id', 'run_up_distance', 'purse', 'weight_carried']\n\nX = completeData[features].copy()\nX['track_id'] = X['track_id'].cat.codes.astype(int)\nX['course_type'] = X['course_type'].cat.codes.astype(int)\nX['track_condition'] = X['track_condition'].cat.codes.astype(int)\nX['race_type'] = X['race_type'].cat.codes.astype(int)\nX['program_number'] = X['program_number'].astype('category').cat.codes.astype(int)\nX['race_date'] = X['race_date'].astype('int64')\ny = completeData['odds']\nX_train, X_test, y_train, y_test=train_test_split(X, y, random_state=50)\n\n","metadata":{"execution":{"iopub.status.busy":"2022-08-22T17:19:42.535051Z","iopub.execute_input":"2022-08-22T17:19:42.535823Z","iopub.status.idle":"2022-08-22T17:19:45.687305Z","shell.execute_reply.started":"2022-08-22T17:19:42.535767Z","shell.execute_reply":"2022-08-22T17:19:45.686127Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### DecisionTreeRegressor","metadata":{}},{"cell_type":"code","source":"from sklearn.tree import DecisionTreeRegressor\n\ndtree = DecisionTreeRegressor()\ndtree = dtree.fit(X_train, y_train)","metadata":{"execution":{"iopub.status.busy":"2022-08-22T17:19:45.712819Z","iopub.execute_input":"2022-08-22T17:19:45.713455Z","iopub.status.idle":"2022-08-22T17:20:37.471188Z","shell.execute_reply.started":"2022-08-22T17:19:45.713419Z","shell.execute_reply":"2022-08-22T17:20:37.469923Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"score = dtree.score(X_train, y_train)\nprint(\"R-squared:\", score) ","metadata":{"execution":{"iopub.status.busy":"2022-08-22T17:20:41.293084Z","iopub.execute_input":"2022-08-22T17:20:41.295190Z","iopub.status.idle":"2022-08-22T17:20:43.350824Z","shell.execute_reply.started":"2022-08-22T17:20:41.295140Z","shell.execute_reply":"2022-08-22T17:20:43.349314Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"from sklearn.metrics import mean_squared_error\n\nypred = dtree.predict(X_test)\n\nmse = mean_squared_error(y_test, ypred)\nprint(\"MSE: \", mse)\nprint(\"RMSE: \", mse**(1/2.0)) ","metadata":{"execution":{"iopub.status.busy":"2022-08-22T17:20:45.287620Z","iopub.execute_input":"2022-08-22T17:20:45.288969Z","iopub.status.idle":"2022-08-22T17:20:45.891138Z","shell.execute_reply.started":"2022-08-22T17:20:45.288928Z","shell.execute_reply":"2022-08-22T17:20:45.889931Z"},"trusted":true},"execution_count":null,"outputs":[]}]}