{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"code","source":"# This Python 3 environment comes with many helpful analytics libraries installed\n# It is defined by the kaggle/python Docker image: https://github.com/kaggle/docker-python\n# For example, here's several helpful packages to load\n\nimport numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\n\n# Input data files are available in the read-only \"../input/\" directory\n# For example, running this (by clicking run or pressing Shift+Enter) will list all files under the input directory\n\nimport os\nfor dirname, _, filenames in os.walk('/kaggle/input'):\n    for filename in filenames:\n        print(os.path.join(dirname, filename))\n\n# You can write up to 20GB to the current directory (/kaggle/working/) that gets preserved as output when you create a version using \"Save & Run All\" \n# You can also write temporary files to /kaggle/temp/, but they won't be saved outside of the current session","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","execution":{"iopub.status.busy":"2022-12-08T14:31:03.860471Z","iopub.execute_input":"2022-12-08T14:31:03.860989Z","iopub.status.idle":"2022-12-08T14:31:03.896549Z","shell.execute_reply.started":"2022-12-08T14:31:03.860881Z","shell.execute_reply":"2022-12-08T14:31:03.895359Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**Columns\nnyra_start_table.csv**\n* track_id - 3 character id for the track the race took place at. AQU -Aqueduct, BEL - Belmont, SAR - Saratoga.\n* race_date - date the race took place. YYYY-MM-DD.\n* race_number - Number of the race. Passed as 3 characters but can be cast or converted to int for this data set.\n* program_number - Program number of the horse in the race passed as 3 characters. Should remain 3 characters as it isn't limited to just numbers. Is essentially the unique identifier of the horse in the race.\n* weight_carried - An integer of the weight carried by the horse in the race.\n* jockey - Name of the jockey on the horse in the race. 50 character max.\n* odds - Odds to win the race passed as an integer. Divide by 100 to derive the odds to 1. Example - 1280 would be 12.8-1.\n* position_at_finish - An integer of the horse's finishing position. (added to the dataset 9/8/22)\n\n**nyra_race_table.csv**\n\n* track_id - 3 character id for the track the race took place at. AQU -Aqueduct, BEL - Belmont, SAR - Saratoga.\n* race_date - date the race took place. YYYY-MM-DD.\n* race_number - Number of the race. Passed as 3 characters but can be cast or converted to int for this data set.\n* distance_id - Distance of the race in furlongs passed as an integer. Example - 600 would be 6 furlongs.\n* course_type - The course the race was run over passed as one character. M - Hurdle, D - Dirt, O - Outer turf, I - Inner turf, T - turf.\n* track_condition - The condition of the course the race was run on passed as three characters. YL - Yielding, FM - Firm, SY - Sloppy, GD - Good, FT - Fast, MY - Muddy, SF - Soft.\n* run_up_distance - Distance in feet of the gate to the start of the race passed as an integer.\n* race_type - The classification of the race passed as as five characters. STK - Stakes, WCL - Waiver Claiming, WMC - Waiver Maiden Claiming, SST - Starter Stakes, SHP - Starter Handicap, CLM - Claiming, STR - Starter Allowance, AOC - Allowance Optionl Claimer, SOC - Starter Optional Claimer, MCL - Maiden Claiming, ALW - Allowance, MSW - Maiden Special Weight.\n* purse - Purse in US dollars of the race passed as an money with two decimal places.\n* post_time - Time of day the race began passed as 5 character. Example - 01220 would be 12:20.\n\n\n**nyra_tracking_table.csv**\n\n\n* track_id - 3 character id for the track the race took place at. AQU -Aqueduct, BEL - Belmont, SAR - Saratoga.\n* race_date - date the race took place. YYYY-MM-DD.\n* race_number - Number of the race. Passed as 3 characters but can be cast or converted to int for this data set.\n* program_number - Program number of the horse in the race passed as 3 characters. Should remain 3 characters as it isn't limited to just numbers. Is essentially the unique identifier of the horse in the race.\n* trakus_index - The common collection of point of the lat / long of the horse in the race passed as an integer. From what we can tell, it's collected every 0.25 seconds.\n* latitude - The latitude of the horse in the race passed as a float.\n* longitude - The longitude of the horse in the race passed as a float.\n\n**nyra_2019_complete.csv** - This file is the combined 3 files into one table. The keys to join them trakus with race - track_id, race_date, race_number. To join trakus with start - track_id, race_date, race_number, program_number.\n\n\n\n* track_id - char(3)\n* race_date - date\n* race_number - char(3)\n* program_number - char(3)\n* trakus_index - int\n* latitude - float\n* longitude - float\n* distance_id - int\n* course_type - char(1)\n* track_condition - char(3)\n* run_up_distance - int\n* race_type - char(5)\n* post_time - char(5)\n* weight_carried - int\n* jockey - char(50)\n* odds - int\n* position_at_finish - An integer of the horse's finishing position. (added to the dataset 9/8/22)","metadata":{}},{"cell_type":"markdown","source":"**Note;This humble project focuses on the odds given to race participants who went on to win the race. In other words, this notebook focuses on determining how accurate the given odds are when compared to race results in terms of winners.It is also worth noting that this notebook does not advocate or advise anyone to engage in serious gambling. If it were possible to predict winners, the world would be filled with racetrack millionaires and the bookmakers would all be bankrupt.**","metadata":{}},{"cell_type":"code","source":"import numpy as np \nimport matplotlib.pyplot as plt \nimport plotly as py\nimport plotly.graph_objs as go\nimport pandas as pd\nimport matplotlib.pyplot as plt\nimport seaborn as sns\nfrom sklearn.ensemble import RandomForestRegressor, GradientBoostingRegressor\nfrom sklearn.model_selection import train_test_split\nfrom sklearn.metrics import r2_score, mean_squared_error\nfrom sklearn.model_selection import GridSearchCV\n","metadata":{"execution":{"iopub.status.busy":"2022-12-08T14:31:03.918712Z","iopub.execute_input":"2022-12-08T14:31:03.919166Z","iopub.status.idle":"2022-12-08T14:31:05.978178Z","shell.execute_reply.started":"2022-12-08T14:31:03.919128Z","shell.execute_reply":"2022-12-08T14:31:05.976872Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"cdf=pd.read_csv('../input/big-data-derby-2022/nyra_2019_complete.csv')\nrdf=pd.read_csv('../input/big-data-derby-2022/nyra_race_table.csv')\nsdf=pd.read_csv('../input/big-data-derby-2022/nyra_start_table.csv')\ntdf=pd.read_csv('../input/big-data-derby-2022/nyra_tracking_table.csv')","metadata":{"execution":{"iopub.status.busy":"2022-12-08T14:31:05.981691Z","iopub.execute_input":"2022-12-08T14:31:05.982830Z","iopub.status.idle":"2022-12-08T14:31:32.842542Z","shell.execute_reply.started":"2022-12-08T14:31:05.982776Z","shell.execute_reply":"2022-12-08T14:31:32.841273Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**Naming start table columns and Complete table**","metadata":{}},{"cell_type":"code","source":"cdf.info()","metadata":{"execution":{"iopub.status.busy":"2022-12-08T15:10:56.006292Z","iopub.execute_input":"2022-12-08T15:10:56.006764Z","iopub.status.idle":"2022-12-08T15:10:56.040774Z","shell.execute_reply.started":"2022-12-08T15:10:56.006716Z","shell.execute_reply":"2022-12-08T15:10:56.039380Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"rdf.info()","metadata":{"execution":{"iopub.status.busy":"2022-12-08T15:12:16.015104Z","iopub.execute_input":"2022-12-08T15:12:16.015520Z","iopub.status.idle":"2022-12-08T15:12:16.031436Z","shell.execute_reply.started":"2022-12-08T15:12:16.015486Z","shell.execute_reply":"2022-12-08T15:12:16.030521Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"sdf.info()","metadata":{"execution":{"iopub.status.busy":"2022-12-08T15:13:07.378222Z","iopub.execute_input":"2022-12-08T15:13:07.378650Z","iopub.status.idle":"2022-12-08T15:13:07.397434Z","shell.execute_reply.started":"2022-12-08T15:13:07.378613Z","shell.execute_reply":"2022-12-08T15:13:07.396363Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"cdf.info()","metadata":{"execution":{"iopub.status.busy":"2022-12-08T15:15:22.672336Z","iopub.execute_input":"2022-12-08T15:15:22.673077Z","iopub.status.idle":"2022-12-08T15:15:22.693292Z","shell.execute_reply.started":"2022-12-08T15:15:22.673038Z","shell.execute_reply":"2022-12-08T15:15:22.692087Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"rdf.info()","metadata":{"execution":{"iopub.status.busy":"2022-12-08T15:16:30.783950Z","iopub.execute_input":"2022-12-08T15:16:30.784389Z","iopub.status.idle":"2022-12-08T15:16:30.800362Z","shell.execute_reply.started":"2022-12-08T15:16:30.784345Z","shell.execute_reply":"2022-12-08T15:16:30.798954Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"tdf.info()","metadata":{"execution":{"iopub.status.busy":"2022-12-08T15:33:32.145922Z","iopub.execute_input":"2022-12-08T15:33:32.146452Z","iopub.status.idle":"2022-12-08T15:33:32.161774Z","shell.execute_reply.started":"2022-12-08T15:33:32.146402Z","shell.execute_reply":"2022-12-08T15:33:32.160450Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"sdf.columns = ['track_id', 'race_date', 'race_number', 'program_number', 'weight_carried', 'jockey', 'odds', 'position_at_finish']\nsdf['odds'] =sdf['odds']/100\n#sdf.head(15)\n#sdf.info()","metadata":{"execution":{"iopub.status.busy":"2022-12-08T14:31:32.843805Z","iopub.execute_input":"2022-12-08T14:31:32.844172Z","iopub.status.idle":"2022-12-08T14:31:32.857220Z","shell.execute_reply.started":"2022-12-08T14:31:32.844137Z","shell.execute_reply":"2022-12-08T14:31:32.855977Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"cdf.drop(['8'] , axis =1 ,inplace = True)\ncdf.columns = ['track_id', 'race_date', 'race_number', 'program_number','trakus_index','latitude','longitude','distance_id','course_type','track_condition','run_up_distance','race_type','purse','post_time','weight_carried','jockey','odds']","metadata":{"execution":{"iopub.status.busy":"2022-12-08T14:31:32.861447Z","iopub.execute_input":"2022-12-08T14:31:32.862009Z","iopub.status.idle":"2022-12-08T14:31:33.582584Z","shell.execute_reply.started":"2022-12-08T14:31:32.861957Z","shell.execute_reply":"2022-12-08T14:31:33.580809Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**Counting total number of races**","metadata":{}},{"cell_type":"code","source":"\nrdf.groupby(['race_date'])['race_number'].count().sum()","metadata":{"execution":{"iopub.status.busy":"2022-12-08T14:31:33.584131Z","iopub.execute_input":"2022-12-08T14:31:33.584489Z","iopub.status.idle":"2022-12-08T14:31:33.608682Z","shell.execute_reply.started":"2022-12-08T14:31:33.584455Z","shell.execute_reply":"2022-12-08T14:31:33.607324Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**Counting total number of races each day**","metadata":{}},{"cell_type":"code","source":"rdf.groupby(['race_date'])['race_number'].count()","metadata":{"execution":{"iopub.status.busy":"2022-12-08T14:31:33.611041Z","iopub.execute_input":"2022-12-08T14:31:33.611939Z","iopub.status.idle":"2022-12-08T14:31:33.628017Z","shell.execute_reply.started":"2022-12-08T14:31:33.611887Z","shell.execute_reply":"2022-12-08T14:31:33.626438Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**Total number of races in each track**","metadata":{}},{"cell_type":"code","source":"plt.bar(x = rdf['track_id'].value_counts().index, height = rdf['track_id'].value_counts())","metadata":{"execution":{"iopub.status.busy":"2022-12-08T14:31:33.629451Z","iopub.execute_input":"2022-12-08T14:31:33.629828Z","iopub.status.idle":"2022-12-08T14:31:33.886758Z","shell.execute_reply.started":"2022-12-08T14:31:33.629797Z","shell.execute_reply":"2022-12-08T14:31:33.885293Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**Winners of the each race **","metadata":{}},{"cell_type":"code","source":"sdf[sdf['position_at_finish'] == 1]","metadata":{"execution":{"iopub.status.busy":"2022-12-08T14:31:33.888649Z","iopub.execute_input":"2022-12-08T14:31:33.889177Z","iopub.status.idle":"2022-12-08T14:31:33.921304Z","shell.execute_reply.started":"2022-12-08T14:31:33.889126Z","shell.execute_reply":"2022-12-08T14:31:33.919767Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"\n> **Average performance of each horse (178 horses)\n**","metadata":{}},{"cell_type":"code","source":"sdf.loc[sdf.jockey == 'Cory Orm']","metadata":{"execution":{"iopub.status.busy":"2022-12-08T14:31:33.922801Z","iopub.execute_input":"2022-12-08T14:31:33.923176Z","iopub.status.idle":"2022-12-08T14:31:34.163367Z","shell.execute_reply.started":"2022-12-08T14:31:33.923135Z","shell.execute_reply":"2022-12-08T14:31:34.162120Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"> **This horse only competed in one race! We need a better way to rank horses based on their performance. The best ranking should have a low position in as many races as possible. Because this is difficult to model, my plan is to encode the position so that a higher number indicates a better position. After that, we can do position * number of races.**","metadata":{}},{"cell_type":"code","source":"best_jockey =sdf.copy()\nencoding = {'position_at_finish':  {1: 14, 2: 13, 3: 12, 4: 11, 5:10, 6: 9, 7:8, 8:7, 9:6, 10:5, 11:4, 12:3, 13:2, 14:1}}\nbest_jockey= best_jockey.replace(encoding)\nbest_jockey.head()","metadata":{"execution":{"iopub.status.busy":"2022-11-28T15:26:35.144425Z","iopub.execute_input":"2022-11-28T15:26:35.145234Z","iopub.status.idle":"2022-11-28T15:26:35.167019Z","shell.execute_reply.started":"2022-11-28T15:26:35.145198Z","shell.execute_reply":"2022-11-28T15:26:35.165814Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def performance(grp):\n    return grp.position_at_finish.mode() * grp.jockey.count()\n    \n    \n\nbest_horse = best_jockey.groupby('jockey').apply( lambda grp: grp['position_at_finish'].median()*grp['jockey'].count()).sort_values(ascending = False)\nbest_horse","metadata":{"execution":{"iopub.status.busy":"2022-11-28T15:26:35.168685Z","iopub.execute_input":"2022-11-28T15:26:35.169130Z","iopub.status.idle":"2022-11-28T15:26:35.249049Z","shell.execute_reply.started":"2022-11-28T15:26:35.169094Z","shell.execute_reply":"2022-11-28T15:26:35.247917Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"> **On average, the best jockeys appeared to be Manuel Franco, Jose Lezcano, Dylan Davis, Irad Ortiz Jr., and Junior Alvarado, with Suguru Hamanaka ranking last.**","metadata":{}},{"cell_type":"code","source":"fig, ax = plt.subplots()\nsns.countplot(data = sdf, ax=ax, x='track_id').set(title = 'Tracks Used')","metadata":{"execution":{"iopub.status.busy":"2022-11-28T15:33:27.432428Z","iopub.execute_input":"2022-11-28T15:33:27.432838Z","iopub.status.idle":"2022-11-28T15:33:27.659124Z","shell.execute_reply.started":"2022-11-28T15:33:27.432807Z","shell.execute_reply":"2022-11-28T15:33:27.657645Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"> **it looks like AQU is most used tracks among thers**","metadata":{}},{"cell_type":"code","source":"sdf['race_date'] = pd.to_datetime(sdf['race_date'])\nsdf.head()","metadata":{"execution":{"iopub.status.busy":"2022-11-28T15:26:35.466522Z","iopub.execute_input":"2022-11-28T15:26:35.466908Z","iopub.status.idle":"2022-11-28T15:26:35.490891Z","shell.execute_reply.started":"2022-11-28T15:26:35.466873Z","shell.execute_reply":"2022-11-28T15:26:35.489702Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"fig, ax = plt.subplots(figsize=(25,10))\n\nsns.histplot(data=sdf,hue=\"track_id\", ax=ax,x=\"race_date\")\nax.set_title(\"Most used tracks throughout year\", fontsize=25)","metadata":{"execution":{"iopub.status.busy":"2022-11-28T15:26:35.492277Z","iopub.execute_input":"2022-11-28T15:26:35.495059Z","iopub.status.idle":"2022-11-28T15:26:36.087134Z","shell.execute_reply.started":"2022-11-28T15:26:35.495014Z","shell.execute_reply":"2022-11-28T15:26:36.086209Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"> **From November to April (Winter), AQU was used, BEL from May to July and September to November (Summer/Autumn), and SAR from July to September (Summer).**","metadata":{}},{"cell_type":"code","source":"fig, ax = plt.subplots(figsize=(25,10))\nsns.countplot(data = sdf, ax=ax, x = 'race_number', hue = 'track_id')","metadata":{"execution":{"iopub.status.busy":"2022-11-28T15:26:36.089041Z","iopub.execute_input":"2022-11-28T15:26:36.089880Z","iopub.status.idle":"2022-11-28T15:26:36.483292Z","shell.execute_reply.started":"2022-11-28T15:26:36.089836Z","shell.execute_reply":"2022-11-28T15:26:36.481958Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"> **According to the records, they had up to 13 races per day, but it was usually 1-9.**","metadata":{}},{"cell_type":"code","source":"sdf['race_number'] = sdf['race_number'].astype('int8')","metadata":{"execution":{"iopub.status.busy":"2022-11-28T15:26:36.485041Z","iopub.execute_input":"2022-11-28T15:26:36.485745Z","iopub.status.idle":"2022-11-28T15:26:36.492792Z","shell.execute_reply.started":"2022-11-28T15:26:36.485676Z","shell.execute_reply":"2022-11-28T15:26:36.491444Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"num_race = sdf.groupby(['race_date'])['race_number'].max().to_frame(name = 'number of races').reset_index()\nnum_race","metadata":{"execution":{"iopub.status.busy":"2022-11-28T15:26:36.494375Z","iopub.execute_input":"2022-11-28T15:26:36.494854Z","iopub.status.idle":"2022-11-28T15:26:36.521506Z","shell.execute_reply.started":"2022-11-28T15:26:36.494797Z","shell.execute_reply":"2022-11-28T15:26:36.520444Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"fig, ax = plt.subplots(figsize=(20,10))\nsns.histplot(data =num_race, x = 'number of races')","metadata":{"execution":{"iopub.status.busy":"2022-11-28T15:38:43.912906Z","iopub.execute_input":"2022-11-28T15:38:43.914120Z","iopub.status.idle":"2022-11-28T15:38:44.257985Z","shell.execute_reply.started":"2022-11-28T15:38:43.914066Z","shell.execute_reply":"2022-11-28T15:38:44.256741Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"> **It is clear that around 7 to 12 races each day **","metadata":{}},{"cell_type":"code","source":"fig, ax = plt.subplots(figsize=(20,10))\nsns.kdeplot(data = sdf, x = 'weight_carried')","metadata":{"execution":{"iopub.status.busy":"2022-11-28T16:44:45.692690Z","iopub.execute_input":"2022-11-28T16:44:45.693119Z","iopub.status.idle":"2022-11-28T16:44:46.123020Z","shell.execute_reply.started":"2022-11-28T16:44:45.693085Z","shell.execute_reply":"2022-11-28T16:44:46.121643Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"> **The average weight carried by a horse is around 120 pounds.**","metadata":{}},{"cell_type":"code","source":"fig, ax = plt.subplots(figsize=(20,10))\nsns.kdeplot(data = sdf, x = 'odds')","metadata":{"execution":{"iopub.status.busy":"2022-11-28T15:26:37.234591Z","iopub.execute_input":"2022-11-28T15:26:37.234923Z","iopub.status.idle":"2022-11-28T15:26:37.644484Z","shell.execute_reply.started":"2022-11-28T15:26:37.234895Z","shell.execute_reply":"2022-11-28T15:26:37.643594Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Track Data Analysis","metadata":{}},{"cell_type":"code","source":"rdf.head()\n","metadata":{"execution":{"iopub.status.busy":"2022-11-28T15:26:37.646086Z","iopub.execute_input":"2022-11-28T15:26:37.647687Z","iopub.status.idle":"2022-11-28T15:26:37.663865Z","shell.execute_reply.started":"2022-11-28T15:26:37.647634Z","shell.execute_reply":"2022-11-28T15:26:37.662763Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"rdf['race_date'] = pd.to_datetime(rdf['race_date'])\nrdf.info()","metadata":{"execution":{"iopub.status.busy":"2022-11-28T15:26:37.665394Z","iopub.execute_input":"2022-11-28T15:26:37.665846Z","iopub.status.idle":"2022-11-28T15:26:37.694254Z","shell.execute_reply.started":"2022-11-28T15:26:37.665806Z","shell.execute_reply":"2022-11-28T15:26:37.693027Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"sns.countplot(data = rdf, x = 'course_type')\n","metadata":{"execution":{"iopub.status.busy":"2022-11-28T15:26:37.695579Z","iopub.execute_input":"2022-11-28T15:26:37.695946Z","iopub.status.idle":"2022-11-28T15:26:37.918136Z","shell.execute_reply.started":"2022-11-28T15:26:37.695915Z","shell.execute_reply":"2022-11-28T15:26:37.916955Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"rdf.distance_id.value_counts()","metadata":{"execution":{"iopub.status.busy":"2022-11-28T15:26:37.919470Z","iopub.execute_input":"2022-11-28T15:26:37.919822Z","iopub.status.idle":"2022-11-28T15:26:37.929236Z","shell.execute_reply.started":"2022-11-28T15:26:37.919792Z","shell.execute_reply":"2022-11-28T15:26:37.927967Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"diag = rdf.groupby(['distance_id'])['distance_id'].count().to_frame(name = 'number of races').reset_index()\ndisplay\nfig, ax = plt.subplots(figsize=(8,5))\nax = sns.barplot(data = diag, x = 'distance_id', y = 'number of races')\nfor i in ax.containers:\n    ax.bar_label(i,)","metadata":{"execution":{"iopub.status.busy":"2022-11-28T15:45:55.956194Z","iopub.execute_input":"2022-11-28T15:45:55.956625Z","iopub.status.idle":"2022-11-28T15:45:56.411819Z","shell.execute_reply.started":"2022-11-28T15:45:55.956591Z","shell.execute_reply":"2022-11-28T15:45:56.410984Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"ax = sns.countplot(data = rdf, x = 'track_condition')\nfor i in ax.containers:\n    ax.bar_label(i,)","metadata":{"execution":{"iopub.status.busy":"2022-11-28T15:26:38.543311Z","iopub.execute_input":"2022-11-28T15:26:38.544158Z","iopub.status.idle":"2022-11-28T15:26:38.804268Z","shell.execute_reply.started":"2022-11-28T15:26:38.544110Z","shell.execute_reply":"2022-11-28T15:26:38.803149Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"> **From the above data we can say that all the tracks are fast**","metadata":{}},{"cell_type":"code","source":"ax = sns.countplot(data = rdf, x = 'race_type')\nfor i in ax.containers:\n    ax.bar_label(i,)","metadata":{"execution":{"iopub.status.busy":"2022-11-28T15:26:38.806899Z","iopub.execute_input":"2022-11-28T15:26:38.807358Z","iopub.status.idle":"2022-11-28T15:26:39.118733Z","shell.execute_reply.started":"2022-11-28T15:26:38.807305Z","shell.execute_reply":"2022-11-28T15:26:39.117649Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"> **The most popular types were Claiming, Maiden Special Weight, and Maiden Claiming.**","metadata":{}},{"cell_type":"code","source":"cdf.head()","metadata":{"execution":{"iopub.status.busy":"2022-11-28T15:26:39.124652Z","iopub.execute_input":"2022-11-28T15:26:39.125077Z","iopub.status.idle":"2022-11-28T15:26:39.147599Z","shell.execute_reply.started":"2022-11-28T15:26:39.125043Z","shell.execute_reply":"2022-11-28T15:26:39.145994Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"cdf['race_date']= pd.to_datetime(cdf['race_date'],format='%Y-%m-%d')\ncdf['month']= cdf['race_date'].dt.month\ncdf['day']= cdf['race_date'].dt.day","metadata":{"execution":{"iopub.status.busy":"2022-11-28T15:26:39.149686Z","iopub.execute_input":"2022-11-28T15:26:39.150800Z","iopub.status.idle":"2022-11-28T15:26:41.270158Z","shell.execute_reply.started":"2022-11-28T15:26:39.150708Z","shell.execute_reply":"2022-11-28T15:26:41.269029Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"cdf.head()","metadata":{"execution":{"iopub.status.busy":"2022-11-28T15:26:41.271909Z","iopub.execute_input":"2022-11-28T15:26:41.272511Z","iopub.status.idle":"2022-11-28T15:26:41.292042Z","shell.execute_reply.started":"2022-11-28T15:26:41.272477Z","shell.execute_reply":"2022-11-28T15:26:41.290924Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"cdf['race_date']= pd.to_datetime(cdf['race_date'],format='%Y-%m-%d')\ncdf['month']= cdf['race_date'].dt.month\ncdf['day']= cdf['race_date'].dt.day\n\ncdf.head()\n","metadata":{"execution":{"iopub.status.busy":"2022-11-28T15:26:41.293893Z","iopub.execute_input":"2022-11-28T15:26:41.294522Z","iopub.status.idle":"2022-11-28T15:26:42.435510Z","shell.execute_reply.started":"2022-11-28T15:26:41.294490Z","shell.execute_reply":"2022-11-28T15:26:42.434652Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"cdf.describe()","metadata":{"execution":{"iopub.status.busy":"2022-11-28T15:26:42.436928Z","iopub.execute_input":"2022-11-28T15:26:42.437537Z","iopub.status.idle":"2022-11-28T15:26:44.656247Z","shell.execute_reply.started":"2022-11-28T15:26:42.437501Z","shell.execute_reply":"2022-11-28T15:26:44.654800Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"cdf.isna().sum()","metadata":{"execution":{"iopub.status.busy":"2022-11-28T15:26:44.658105Z","iopub.execute_input":"2022-11-28T15:26:44.659162Z","iopub.status.idle":"2022-11-28T15:26:46.142815Z","shell.execute_reply.started":"2022-11-28T15:26:44.659110Z","shell.execute_reply":"2022-11-28T15:26:46.142013Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"fig = plt.figure(figsize = (30,20))\nax1 = fig.add_subplot(3,2,1)\nax2 = fig.add_subplot(3,2,2)\nax3 = fig.add_subplot(3,2,3)\nax4 = fig.add_subplot(3,2,4)\nax5 = fig.add_subplot(3,2,5)\nax6 = fig.add_subplot(3,2,6)\n\nsns.countplot(x=cdf[\"track_id\"],ax=ax1)\nax1.set_title(\"Track Usage\")\n\nsns.countplot(y=cdf[\"course_type\"],ax=ax2)\nax2.set_title(\"Type of Race Course\")\n\nmonth_cdf = cdf.groupby(\"month\")\nmonthly_race_number=month_cdf[\"race_number\"].count()\ntotal_race_number=cdf[\"race_number\"].count()\npercentage_race_number=monthly_race_number/total_race_number\nlabels = list(month_cdf.groups)\n# months = [\"Jan\",\"Feb\",\"Mar\",\"Apr\",\"May\",\"Jun\",\"Jul\",\"Aug\",\"Sep\",\"Oct\",\"Nov\",\"Dec\"]\nax3.pie(percentage_race_number,labels=labels, autopct='%.2f',\n        explode=[0.1,0.1,0.1,0.1,0.1,0.1,0.1,0.1,0.1,0.1,0.1,0.1])\nax3.set_title(\"Percentage of Races Conducted for a  Month\")\n\nsns.countplot(x=cdf[\"track_condition\"],ax=ax4)\nax4.set_title(\"Track Condition\")\n\nsns.countplot(y=cdf[\"race_type\"],ax=ax5)\nax5.set_title(\"Type of Race\")\n\nsns.distplot(cdf[\"weight_carried\"],ax=ax6)\nax6.set_title(\"Distirbution of Weight Carried\")\n\nfig.suptitle(\"Basic Exploration\")\nplt.tight_layout\nplt.show()\n","metadata":{"execution":{"iopub.status.busy":"2022-11-28T15:26:46.144140Z","iopub.execute_input":"2022-11-28T15:26:46.144951Z","iopub.status.idle":"2022-11-28T15:27:21.361235Z","shell.execute_reply.started":"2022-11-28T15:26:46.144917Z","shell.execute_reply":"2022-11-28T15:27:21.360089Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#  1.AQU Track Analysis","metadata":{}},{"cell_type":"code","source":"cdf1 = cdf.copy()","metadata":{"execution":{"iopub.status.busy":"2022-11-28T15:27:21.363249Z","iopub.execute_input":"2022-11-28T15:27:21.364021Z","iopub.status.idle":"2022-11-28T15:27:22.067292Z","shell.execute_reply.started":"2022-11-28T15:27:21.363978Z","shell.execute_reply":"2022-11-28T15:27:22.066142Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"cond1 = cdf1[\"track_id\"] == \"AQU\"\ncdf1_AQU = cdf1[cond1]\ncdf1_AQU.head()\ncdf1_AQU[\"track_id\"].unique()","metadata":{"execution":{"iopub.status.busy":"2022-11-28T15:27:22.068791Z","iopub.execute_input":"2022-11-28T15:27:22.069202Z","iopub.status.idle":"2022-11-28T15:27:22.966796Z","shell.execute_reply.started":"2022-11-28T15:27:22.069164Z","shell.execute_reply":"2022-11-28T15:27:22.965494Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"fig=plt.figure(figsize=(19,12))\nax1 = fig.add_subplot(111)\nsns.scatterplot(data=cdf1_AQU,x = \"longitude\", y = \"latitude\", hue = \"race_number\",label = \"race_number\", palette = \"Set2\")\nax1.set_title(\"Track: AQU\")\nplt.legend()","metadata":{"execution":{"iopub.status.busy":"2022-11-28T15:51:18.705882Z","iopub.execute_input":"2022-11-28T15:51:18.706253Z","iopub.status.idle":"2022-11-28T15:53:01.411013Z","shell.execute_reply.started":"2022-11-28T15:51:18.706222Z","shell.execute_reply":"2022-11-28T15:53:01.409626Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# 2.BLE Track Analysis","metadata":{}},{"cell_type":"code","source":"cond1 = cdf1[\"track_id\"] == \"BEL\"\ncdf1_BEL = cdf1[cond1]\ncdf1_BEL.head()\ncdf1_BEL[\"track_id\"].unique()","metadata":{"execution":{"iopub.status.busy":"2022-11-28T15:29:06.284851Z","iopub.execute_input":"2022-11-28T15:29:06.285878Z","iopub.status.idle":"2022-11-28T15:29:07.110942Z","shell.execute_reply.started":"2022-11-28T15:29:06.285834Z","shell.execute_reply":"2022-11-28T15:29:07.110144Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"fig=plt.figure(figsize=(30,18))\nax1 = fig.add_subplot(111)\nsns.scatterplot(data=cdf1_BEL,x = \"longitude\", y = \"latitude\", hue = \"race_number\",label = \"race_number\", palette = \"Set2\")\nax1.set_title(\"Track: BEL\")\nplt.legend()","metadata":{"execution":{"iopub.status.busy":"2022-11-28T15:29:07.112152Z","iopub.execute_input":"2022-11-28T15:29:07.112643Z","iopub.status.idle":"2022-11-28T15:30:35.109402Z","shell.execute_reply.started":"2022-11-28T15:29:07.112613Z","shell.execute_reply":"2022-11-28T15:30:35.108160Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# 3.SAR Track Analysis","metadata":{}},{"cell_type":"code","source":"cond1 = cdf1[\"track_id\"] == \"SAR\"\ncdf1_SAR = cdf1[cond1]\ncdf1_SAR.head()\ncdf1_SAR[\"track_id\"].unique()","metadata":{"execution":{"iopub.status.busy":"2022-11-28T15:30:35.110690Z","iopub.execute_input":"2022-11-28T15:30:35.111046Z","iopub.status.idle":"2022-11-28T15:30:35.720089Z","shell.execute_reply.started":"2022-11-28T15:30:35.111016Z","shell.execute_reply":"2022-11-28T15:30:35.718952Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"fig=plt.figure(figsize=(30,18))\nax1 = fig.add_subplot(111)\nsns.scatterplot(data=cdf1_SAR,x = \"longitude\", y = \"latitude\", hue = \"race_number\",label = \"race_number\", palette = \"Set2\")\nax1.set_title(\"Track: SAR\")\nplt.legend()","metadata":{"execution":{"iopub.status.busy":"2022-11-28T15:30:35.721236Z","iopub.execute_input":"2022-11-28T15:30:35.721559Z","iopub.status.idle":"2022-11-28T15:31:30.438237Z","shell.execute_reply.started":"2022-11-28T15:30:35.721531Z","shell.execute_reply":"2022-11-28T15:31:30.436912Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"fig = plt.figure(figsize = (18,20))\nax1 = fig.add_subplot(211)\n\nsns.heatmap(cdf1.corr(),cmap = \"coolwarm\",vmin = -1, vmax = 1,annot=True)\nax1.set_title(\"Correlation of features to winnning odds\")\n\nplt.tight_layout\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-11-28T15:31:30.440679Z","iopub.execute_input":"2022-11-28T15:31:30.441166Z","iopub.status.idle":"2022-11-28T15:31:34.130647Z","shell.execute_reply.started":"2022-11-28T15:31:30.441125Z","shell.execute_reply":"2022-11-28T15:31:34.129457Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def _2lines(data, figsize=(15, 8), caption='', labels=[]):\n    #['teal', 'lightseagreen', 'powderblue', 'lightcyan',  'azure','darkslategray']\n    fig, ax = plt.subplots(figsize = figsize)\n    ax2 = ax.twinx()\n    ax.set_title(caption, fontsize=15, loc='left')\n    #ax.set_xlabel('time')\n\n    ax.plot(data.index, data.iloc[:, 0], marker='', markerfacecolor='teal', markersize=2, color='lightseagreen', linewidth=2)\n    ax2.plot(data.index, data.iloc[:, 1], marker='', markerfacecolor='tomato', color='salmon', linewidth=2, linestyle='--')\n\n    ax.set_ylabel(labels[0])\n    ax2.set_ylabel(labels[1])\n    ax.legend([labels[0]], loc='upper left')\n    ax2.legend([labels[1]], loc='upper center')\n    ax.yaxis.grid(color='lightgray', linestyle='dashed')\n\n    plt.tight_layout()\n\n    plt.show()\n    \n    \ndef lines(data, figsize=(15, 8), caption='', labels=[]):\n    plt.figure(figsize=figsize)#dpi=100\n    colors = ['lightseagreen', 'salmon', 'lightskyblue', 'mediumseagreen', 'sandybrown', 'lightslategray', 'wheat', 'crimson']\n    mcolors = ['teal', 'tomato', 'skyblue', 'mediumseagreen', 'peru', 'slategray', 'tan',  'firebrick']\n    \n    for i in range(data.shape[1]):\n        plt.plot(data.index, data.iloc[:, i], marker='', markerfacecolor=mcolors[i], markersize=2, color=colors[i], linewidth=2, label=labels[i])\n        plt.text(data.shape[0] + 1, data.values[-1, i ], labels[i], fontsize=11, color=mcolors[i])\n\n    #plt.legend()\n    plt.title(caption,  fontsize=14, loc='left')\n\n    plt.grid(visible=True, which='major', axis='both', color='lightgray', linestyle='-', linewidth=0.2)\n    plt.gca().spines[['top', 'right']].set_visible(False)\n    plt.gca().spines[['bottom', 'left']].set_edgecolor('whitesmoke')\n    \n    plt.yticks(fontsize=9, )#rotation=90\n    plt.xticks(fontsize=9, )#rotation=90\n    plt.tight_layout()\n    plt.show()","metadata":{"execution":{"iopub.status.busy":"2022-11-28T15:31:34.132318Z","iopub.execute_input":"2022-11-28T15:31:34.132782Z","iopub.status.idle":"2022-11-28T15:31:34.148402Z","shell.execute_reply.started":"2022-11-28T15:31:34.132743Z","shell.execute_reply":"2022-11-28T15:31:34.147477Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"jockeys = ['Manuel Franco', 'Jose Lezcano', 'Javier Castellano', 'Irad Ortiz Jr.']\n\ncs_ = sdf[['race_date', 'race_number']].drop_duplicates()\n\n\nfor jk in jockeys:\n    cs = sdf[sdf['jockey']==jk].sort_values(by=['race_date', 'race_number'])\n    cs.reset_index(drop=True, inplace=True)\n    cs['win'] = (cs['position_at_finish']<2).astype(int)\n    cs['place'] = (cs['position_at_finish']<4).astype(int)\n    cs['win_cumsum'] = cs['win'].cumsum()\n    cs['place_cumsum'] = cs['place'].cumsum()\n    \n    cs_ = cs_.merge(cs[['race_date', 'race_number', 'win']], on=['race_date', 'race_number'], how='left')\n    cs_.sort_values(by=['race_date', 'race_number'], inplace=True)\n    cs_.rename(columns={'win':jk}, inplace=True)\n    cs_[jk].fillna(0, inplace=True)\n    cs_[jk] = cs_[jk].cumsum()\n    \n    \n    _2lines(cs[['win_cumsum', 'place_cumsum']], figsize=(10, 5),  \n            caption=f\"{jk} Number of wins and places\", \n            labels = ['wins compared ', ' places compaerd '])\n    \ncs_.sort_values(by=['race_date', 'race_number'], inplace=True)","metadata":{"execution":{"iopub.status.busy":"2022-11-28T16:19:55.859928Z","iopub.execute_input":"2022-11-28T16:19:55.860361Z","iopub.status.idle":"2022-11-28T16:19:57.504785Z","shell.execute_reply.started":"2022-11-28T16:19:55.860327Z","shell.execute_reply":"2022-11-28T16:19:57.503578Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"lines(cs_.iloc[:, 2:], figsize=(13, 6), caption='Cumulative num. of wins over time for each jockey',  labels = cs_.columns[2:])","metadata":{"execution":{"iopub.status.busy":"2022-11-28T15:31:35.807017Z","iopub.execute_input":"2022-11-28T15:31:35.807420Z","iopub.status.idle":"2022-11-28T15:31:36.161979Z","shell.execute_reply.started":"2022-11-28T15:31:35.807386Z","shell.execute_reply":"2022-11-28T15:31:36.160888Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"jockeys = ['Manuel Franco', 'Jose Lezcano', 'Javier Castellano', 'Irad Ortiz Jr.']\n\ncs_ = sdf[['race_date', 'race_number']].drop_duplicates()\n\n\nfor jk in jockeys:\n    cs = sdf[sdf['jockey']==jk].sort_values(by=['race_date', 'race_number'])\n    cs.reset_index(drop=True, inplace=True)\n    cs['win'] = (cs['position_at_finish']<2).astype(int)\n    cs['place'] = (cs['position_at_finish']<4).astype(int)\n    cs['run']=1\n    cs['win_cumsum'] = (cs['win'].cumsum())/(cs['run'].cumsum())\n    cs['place_cumsum'] = (cs['place'].cumsum())/(cs['run'].cumsum())\n    \n    \n    cs_ = cs_.merge(cs[['race_date', 'race_number', 'win', 'run']], on=['race_date', 'race_number'], how='left')\n    cs_.sort_values(by=['race_date', 'race_number'], inplace=True)\n    cs_.rename(columns={'win':jk, 'run':f'{jk}run'}, inplace=True)\n    cs_.fillna(0, inplace=True)\n    cs_[jk] = (cs_[jk].cumsum())/(cs_[f'{jk}run'].cumsum())\n    cs_.drop(columns=[f'{jk}run'], inplace=True)\n    \n    \n    lines(cs[['win_cumsum', 'place_cumsum']], figsize=(12, 6),  \n            caption=f\"{jk}'s journey: cumulative win rate and place rate over time\", \n            labels = ['cumulative win rate', 'cumulative place rate'])\n    \ncs_.sort_values(by=['race_date', 'race_number'], inplace=True)\ncs_ = cs_.replace([-np.inf, np.inf], np.nan).fillna(0)","metadata":{"execution":{"iopub.status.busy":"2022-11-28T15:31:36.163632Z","iopub.execute_input":"2022-11-28T15:31:36.164371Z","iopub.status.idle":"2022-11-28T15:31:37.559834Z","shell.execute_reply.started":"2022-11-28T15:31:36.164337Z","shell.execute_reply":"2022-11-28T15:31:37.558994Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"lines(cs_.iloc[:, 2:], figsize=(15, 8), caption='Cumulative win rate over time for each jockey',  labels = cs_.columns[2:])","metadata":{"execution":{"iopub.status.busy":"2022-11-28T15:31:37.561203Z","iopub.execute_input":"2022-11-28T15:31:37.562095Z","iopub.status.idle":"2022-11-28T15:31:37.919878Z","shell.execute_reply.started":"2022-11-28T15:31:37.562060Z","shell.execute_reply":"2022-11-28T15:31:37.918693Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"","metadata":{}},{"cell_type":"code","source":"tdf.to_csv('submission.csv', index=False)\nprint(\"Your submission was successfully saved!\")","metadata":{"execution":{"iopub.status.busy":"2022-11-28T15:31:37.921513Z","iopub.execute_input":"2022-11-28T15:31:37.922659Z","iopub.status.idle":"2022-11-28T15:31:59.444655Z","shell.execute_reply.started":"2022-11-28T15:31:37.922613Z","shell.execute_reply":"2022-11-28T15:31:59.443657Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"","metadata":{}}]}