{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"code","source":"# This Python 3 environment comes with many helpful analytics libraries installed\n# It is defined by the kaggle/python Docker image: https://github.com/kaggle/docker-python\n# For example, here's several helpful packages to load\n\nimport numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\nimport seaborn as sns\nfrom matplotlib import pyplot as plt\n\n# Input data files are available in the read-only \"../input/\" directory\n# For example, running this (by clicking run or pressing Shift+Enter) will list all files under the input directory\n\nimport os\nfor dirname, _, filenames in os.walk('/kaggle/input'):\n    for filename in filenames:\n        print(os.path.join(dirname, filename))\n\n# You can write up to 20GB to the current directory (/kaggle/working/) that gets preserved as output when you create a version using \"Save & Run All\" \n# You can also write temporary files to /kaggle/temp/, but they won't be saved outside of the current session","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","execution":{"iopub.status.busy":"2022-11-11T00:59:23.826866Z","iopub.execute_input":"2022-11-11T00:59:23.827237Z","iopub.status.idle":"2022-11-11T00:59:23.836009Z","shell.execute_reply.started":"2022-11-11T00:59:23.827201Z","shell.execute_reply":"2022-11-11T00:59:23.835076Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Import data - change to split data tables for other analyses?\ncol_names_all = [\"track_id\", \"race_date\", \"race_number\", \"program_number\", \"trakus_index\", \"latitude\", \"longitude\", \"distance_id\", \"course_type\", \"track_condition\", \"run_up_distance\", \"race_type\", \"purse\", \"post_time\", \"weight_carried\", \"jockey\", \"odds\", \"position_at_finish\"]\ncomplete_data = pd.read_csv(\"/kaggle/input/big-data-derby-2022/nyra_2019_complete.csv\", names = col_names_all)\n# Assign unique IDs for races\ncomplete_data[\"race_id\"] = complete_data[\"track_id\"] + complete_data[\"race_date\"] + complete_data[\"race_number\"].astype(str)\ncomplete_data[\"race_id\"] = complete_data[\"race_id\"].str.replace(\" \", \"\")\n# Assign unique IDs for horses in races\ncomplete_data[\"horse_id\"] = complete_data[\"track_id\"] + complete_data[\"race_date\"] + complete_data[\"race_number\"].astype(str) + complete_data[\"program_number\"].astype(str)\ncomplete_data[\"horse_id\"] = complete_data[\"horse_id\"].str.replace(\" \", \"\")","metadata":{"execution":{"iopub.status.busy":"2022-11-11T00:59:23.837798Z","iopub.execute_input":"2022-11-11T00:59:23.838992Z","iopub.status.idle":"2022-11-11T00:59:41.097205Z","shell.execute_reply.started":"2022-11-11T00:59:23.838951Z","shell.execute_reply":"2022-11-11T00:59:41.095670Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Maiden Special Weight races are races for horses that have not yet won, but have shown pre-race promise. These are ideal races for a horse's first win, and graduating from (winning) these races will allow a horse to progress to more prestigious races with larger purses. Trainers will endeavour to set horses up with race conditions that give a horse the best chance of winning.\n\nUnlike the majority of other race types listed here, weights carried by horses competing in Maiden Special Weight races are not adjusted by past horse performance. The goal of these adjusted carrying weights is to even the field, and to make the races more competitive. Some race types listed here will have horses weighted to even the field as much as possible (e.g. handicap races), while some race types will have variable methods for weighting horses (e.g. weight carried in stakes races can be adjusted by past performance, weight-for-age, etc.). Without all the relevant data for methods by which weight is assigned, including horse age and gender, predictions of horse performance in these cases are likely to be highly confounded.\n\nWeights carried in Maiden Special Weight races are adjusted based on basic attributes of the competing horse (e.g. age, gender). Without the past performance adjustment, a better racing method should be a clearer predictor of race result. The analyses performed here may be expanded to other race types in future work, but these will require more data on the horses racing to be optimal.","metadata":{}},{"cell_type":"code","source":"msw = complete_data[complete_data[\"race_type\"] == \"MSW\"]\n# Make list of unique horses and races in MSW dataframe\nhorse_id_list = list(set(msw[\"horse_id\"]))\nrace_id_list = list(set(msw[\"race_id\"]))","metadata":{"execution":{"iopub.status.busy":"2022-11-11T00:59:41.098624Z","iopub.execute_input":"2022-11-11T00:59:41.098929Z","iopub.status.idle":"2022-11-11T00:59:42.608207Z","shell.execute_reply.started":"2022-11-11T00:59:41.098905Z","shell.execute_reply":"2022-11-11T00:59:42.607029Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"col_names_start = [\"track_id\", \"race_date\", \"race_number\", \"program_number\", \"weight_carried\", \"jockey\", \"odds\", \"position_at_finish\"]\nstart = pd.read_csv(\"/kaggle/input/big-data-derby-2022/nyra_start_table.csv\", names = col_names_start)\nstart[\"horse_id\"] = start[\"track_id\"] + start[\"race_date\"] + start[\"race_number\"].astype(str) + start[\"program_number\"].astype(str)\nstart[\"horse_id\"] = start[\"horse_id\"].str.replace(\" \", \"\")\n# Subset start data frame to only include horses competing in MSW races\nmsw_start = start[start[\"horse_id\"].isin(horse_id_list)]","metadata":{"execution":{"iopub.status.busy":"2022-11-11T00:59:42.613163Z","iopub.execute_input":"2022-11-11T00:59:42.613546Z","iopub.status.idle":"2022-11-11T00:59:42.655510Z","shell.execute_reply.started":"2022-11-11T00:59:42.613504Z","shell.execute_reply":"2022-11-11T00:59:42.654089Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Show distribution of carried weight and predicted odds to final finishing position\nsns.boxplot(data=msw_start, y=\"weight_carried\", x=\"position_at_finish\"); plt.xlabel(\"Position at finish\"); plt.ylabel(\"Weight carried\"); plt.title(\"Weight carried\"); plt.show()\nsns.boxplot(data=msw_start, y=\"odds\", x=\"position_at_finish\"); plt.xlabel(\"Position at finish\"); plt.ylabel(\"Odds\"); plt.title(\"Odds\"); plt.show()","metadata":{"execution":{"iopub.status.busy":"2022-11-11T00:59:42.656829Z","iopub.execute_input":"2022-11-11T00:59:42.657138Z","iopub.status.idle":"2022-11-11T00:59:43.225030Z","shell.execute_reply.started":"2022-11-11T00:59:42.657109Z","shell.execute_reply":"2022-11-11T00:59:43.224091Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"The weight carried by the horses does not have a clear correlation with finishing position, indicating that the weights assigned in the MSW races are appropriate for the age and gender of the horse, and do not introduce undue bias.\n\nThere is a clear correlation between good pre-race odds and a good finishing position, indicating that, even if these horses have not yet won a race, past performance is a good indicator of performance in the current race.","metadata":{}},{"cell_type":"code","source":"race = pd.read_csv(\"/kaggle/input/big-data-derby-2022/nyra_race_table.csv\")\nrace[\"race_id\"] = race[\"track_id\"] + race[\"race_date\"] + race[\"race_number\"].astype(str)\nrace[\"race_id\"] = race[\"race_id\"].str.replace(\" \", \"\")\nmsw_race = race[race[\"race_id\"].isin(race_id_list)] # Subset race data frame to only include MSW races\n# Group races by conditions to find comparable races\nmsw_race[\"race_conditions_id\"] = msw_race[\"track_id\"] + msw_race[\"distance_id\"].astype(str) + msw_race[\"course_type\"] + msw_race[\"track_condition\"] + msw_race[\"run_up_distance\"].astype(str)\nmsw_race[\"race_conditions_id\"] = msw_race[\"race_conditions_id\"].str.replace(\" \", \"\")","metadata":{"execution":{"iopub.status.busy":"2022-11-11T00:59:43.226072Z","iopub.execute_input":"2022-11-11T00:59:43.226309Z","iopub.status.idle":"2022-11-11T00:59:43.252862Z","shell.execute_reply.started":"2022-11-11T00:59:43.226286Z","shell.execute_reply":"2022-11-11T00:59:43.251457Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Visualise frequency of different race conditions\nsns.histplot(data = msw_race, x = \"track_id\"); plt.xlabel('Track'); plt.title('Track frequency, MSW races'); plt.show()\nsns.histplot(data = msw_race, x = \"distance_id\"); plt.xlabel('Race distance ID (furlongs x 100)'); plt.title('Race distance frequency, MSW races'); plt.show()\nsns.histplot(data = msw_race, x = \"course_type\"); plt.xlabel('Course type'); plt.title('Course type frequency, MSW races'); plt.show()\nsns.histplot(data = msw_race, x = \"track_condition\"); plt.xlabel('Track condition'); plt.title('Track condition frequency, MSW races'); plt.show()\nsns.histplot(data = msw_race, x = \"run_up_distance\"); plt.xlabel('Run up distance (feet)'); plt.title('Run up distance frequency, MSW races'); plt.show()","metadata":{"execution":{"iopub.status.busy":"2022-11-11T00:59:43.254264Z","iopub.execute_input":"2022-11-11T00:59:43.254691Z","iopub.status.idle":"2022-11-11T00:59:43.995575Z","shell.execute_reply.started":"2022-11-11T00:59:43.254656Z","shell.execute_reply":"2022-11-11T00:59:43.994265Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Find most common combination of race conditions\nprint(msw_race[\"race_conditions_id\"].mode())","metadata":{"execution":{"iopub.status.busy":"2022-11-11T00:59:43.997005Z","iopub.execute_input":"2022-11-11T00:59:43.997333Z","iopub.status.idle":"2022-11-11T00:59:44.003919Z","shell.execute_reply.started":"2022-11-11T00:59:43.997305Z","shell.execute_reply":"2022-11-11T00:59:44.002888Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"The most common race conditions (26 races) for MSW races is AQU800DFT54:\n* Race is at Aqueduct race track.\n* Race is 8 furlongs\n* Course is dirt.\n* Track condition is fast.\n* Run up distance is 54 feet.\n\nThis is an average length race with common conditions. Further analysis will be performed on these races as they can be easily compared, and have a large available dataset.","metadata":{}},{"cell_type":"code","source":"# Subset complete database to only include AQU800DFT54 races\nAQU800DFT54 = msw_race[msw_race[\"race_conditions_id\"] == \"AQU800DFT54\"]\nAQU800DFT54_race_list = list(set(AQU800DFT54[\"race_id\"]))\nAQU800DFT54_tracking = complete_data[complete_data[\"race_id\"].isin(AQU800DFT54_race_list)]\nAQU800DFT54_tracking = AQU800DFT54_tracking.sort_values([\"horse_id\", \"trakus_index\"], ascending = [True, True])\nAQU800DFT54_tracking[\"latitude_diff\"] = AQU800DFT54_tracking[\"latitude\"].diff()\nAQU800DFT54_tracking.loc[AQU800DFT54_tracking[\"trakus_index\"] == 1, \"latitude_diff\"] = 0 \nAQU800DFT54_tracking[\"longitude_diff\"] = AQU800DFT54_tracking[\"longitude\"].diff()\nAQU800DFT54_tracking.loc[AQU800DFT54_tracking[\"trakus_index\"] == 1, \"longitude_diff\"] = 0\nAQU800DFT54_tracking[\"speed\"] = ((AQU800DFT54_tracking[\"latitude_diff\"] ** 2) + (AQU800DFT54_tracking[\"longitude_diff\"] ** 2)) ** 0.5","metadata":{"execution":{"iopub.status.busy":"2022-11-11T00:59:44.005198Z","iopub.execute_input":"2022-11-11T00:59:44.005558Z","iopub.status.idle":"2022-11-11T00:59:44.292330Z","shell.execute_reply.started":"2022-11-11T00:59:44.005490Z","shell.execute_reply":"2022-11-11T00:59:44.291700Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Visualise track shape by plotting longitude and latitude against trakus index\nsns.scatterplot(data = AQU800DFT54_tracking, x = \"longitude\", y = \"latitude\", hue = \"trakus_index\"); plt.title('Track shape'); plt.show()","metadata":{"execution":{"iopub.status.busy":"2022-11-11T00:59:44.294362Z","iopub.execute_input":"2022-11-11T00:59:44.294643Z","iopub.status.idle":"2022-11-11T00:59:46.143636Z","shell.execute_reply.started":"2022-11-11T00:59:44.294611Z","shell.execute_reply":"2022-11-11T00:59:46.142974Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Visualise speed over race by final position\nfig, ax = plt.subplots(figsize=(18, 12))\nlineplot = sns.lineplot(data = AQU800DFT54_tracking, x = \"trakus_index\", y = \"speed\", hue = \"position_at_finish\", ax = ax); plt.xlabel('Trakus index'); plt.title('Speed over race'); plt.show()","metadata":{"execution":{"iopub.status.busy":"2022-11-11T00:59:46.144577Z","iopub.execute_input":"2022-11-11T00:59:46.145015Z","iopub.status.idle":"2022-11-11T01:00:45.890394Z","shell.execute_reply.started":"2022-11-11T00:59:46.144990Z","shell.execute_reply":"2022-11-11T01:00:45.888972Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"This data indicates that in the early stage of the race, there is no substantial difference in speed between horses. The race winners here start to pull ahead from trakus index 150-200, and extend their lead from here.","metadata":{}},{"cell_type":"code","source":"# Race stages are designated as Early, Transitional (where the eventual winners start to pull ahead), and Late (where the eventual winners extend their lead)\nAQU800DFT54_tracking[\"race_stage\"] = \"Early\"\nAQU800DFT54_tracking.loc[(AQU800DFT54_tracking[\"trakus_index\"] > 150 ) & (AQU800DFT54_tracking[\"trakus_index\"] < 200 ), \"race_stage\"] = \"Transitional\"\nAQU800DFT54_tracking.loc[AQU800DFT54_tracking[\"trakus_index\"] > 200 , \"race_stage\"] = \"Late\"\n# Visualise race stages on track\nsns.scatterplot(data = AQU800DFT54_tracking, x = \"longitude\", y = \"latitude\", hue = \"race_stage\"); plt.title('Track shape'); plt.show()","metadata":{"execution":{"iopub.status.busy":"2022-11-11T01:00:45.891672Z","iopub.execute_input":"2022-11-11T01:00:45.891994Z","iopub.status.idle":"2022-11-11T01:00:47.654513Z","shell.execute_reply.started":"2022-11-11T01:00:45.891963Z","shell.execute_reply":"2022-11-11T01:00:47.653019Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"In the studied races, positioning as the horses head into the first turn appears to be very important for final race placing. This may allow early horses to cut closer to the barrier, and to expand their lead. Future analyses regarding conditions required to obtain an advantage at this point may give an advantage across a range of race types.","metadata":{}},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]}]}