{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"","metadata":{"pycharm":{"name":"#%% md\n"}}},{"cell_type":"markdown","source":"## Goal of the Competition 👑\nThe goal of this competition is to **analyze horse racing tactics, drafting strategies, and path efficiency.** You will develop a model using never-before-released coordinate data along with basic race information.\n\nYour work will help racing horse owners, trainers, and veterinarians better understand how equine performance and welfare fit together. With better data analysis, equine welfare could significantly improve.","metadata":{"pycharm":{"name":"#%% md\n"}}},{"cell_type":"markdown","source":"## Infomation of Dataset 📄\n- nyra_start_table.csv: horse/jockey race data\n- nyra_race_table.csv: racetrack race data\n- nyra_tracking_table.csv: tracking data\n- nyra_2019_complete.csv: combined table of three above files\n
\n\n### nyra_start_table.csv: horse/jockey race data\n- track_id: 3 character id for the track the race took place at\n- race_date: Date the race took place\n- race_number: Number of race\n- program_number: Program number of the horse in the race passed as 3 characters\n- weight_carried: An integer of the weight carried by the horse in the race\n- jockey: Name of the jokey on the horse in the race\n- odds: odds to win the race passed as an integer\n\n### nyra_race_table.csv: racetrack race data\n- track_id: 3 character id for the track the race took place at\n- race_date: Date the race took place\n- race_number: Number of race\n- distance_id: Distance of the race in furlongs passed as an integer\n- course_type: The course the race was run over passed as one character\n- trak_condition: The condition of the course the race was run on passed as three characters\n- run_up_distance: Distance in feet of the gate to the start of the race passed as an integer\n- race type: The classification of the race passed as as five characters\n- purse: Purse in US dollars of the race passed as an money with two decimal places\n- post_time: Time of day the race began passed as 5 character\n\n### nyra_tracking_table.csv: tracking fata\n- track_id: 3 character id for the track the race took place at\n- race_date: Date the race took place\n- race_number: Number of race\n- program_number: Program number of the horse in the race passed as 3 characters\n- trakus_index: The common collection of point if the lat/long of the horse in race passed as an integer\n- latitude: The latitude of the horse in the race passed as a float\n- longitude: The longitude of the horse in the race passed as a float\n\n### nyra_2019_complete.csv: combined table of three above files","metadata":{"pycharm":{"name":"#%% md\n"}}},{"cell_type":"code","source":"import numpy as np\nimport pandas as pd\nimport plotly.express as px #graphing\nimport plotly.graph_objects as go #graphing\nfrom plotly.subplots import make_subplots #graphing\nimport plotly.figure_factory as ff #graphing\nimport matplotlib.pyplot as plt #graphing\nimport seaborn as sns #graphing\nimport missingno as msno #describe data\nimport os\ncolors = [\"#FFFFFF\", \"#6CD4FF\", \"#F7DF00\", \"#E60000\"]","metadata":{"pycharm":{"name":"#%%\n"},"execution":{"iopub.status.busy":"2022-08-29T01:53:35.918888Z","iopub.execute_input":"2022-08-29T01:53:35.919275Z","iopub.status.idle":"2022-08-29T01:53:39.429250Z","shell.execute_reply.started":"2022-08-29T01:53:35.919191Z","shell.execute_reply":"2022-08-29T01:53:39.428153Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"nyra_tracking = pd.read_csv('../input/big-data-derby-2022/nyra_tracking_table.csv')\nnyra_start = pd.read_csv('../input/big-data-derby-2022/nyra_start_table.csv')\nnyra_race = pd.read_csv('../input/big-data-derby-2022/nyra_race_table.csv')\nnyra_2019 = pd.read_csv('../input/big-data-derby-2022/nyra_2019_complete.csv')","metadata":{"pycharm":{"name":"#%%\n"},"execution":{"iopub.status.busy":"2022-08-29T01:53:39.431209Z","iopub.execute_input":"2022-08-29T01:53:39.431598Z","iopub.status.idle":"2022-08-29T01:53:39.624489Z","shell.execute_reply.started":"2022-08-29T01:53:39.431565Z","shell.execute_reply":"2022-08-29T01:53:39.620426Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## EDA 🎼","metadata":{}},{"cell_type":"code","source":"nyra_2019.head()","metadata":{"execution":{"iopub.status.busy":"2022-08-29T01:53:39.626018Z","iopub.status.idle":"2022-08-29T01:53:39.626697Z","shell.execute_reply.started":"2022-08-29T01:53:39.626347Z","shell.execute_reply":"2022-08-29T01:53:39.626378Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"nyra_2019.describe().style.background_gradient(cmap = 'Purples')","metadata":{"execution":{"iopub.status.busy":"2022-08-29T01:53:39.629352Z","iopub.status.idle":"2022-08-29T01:53:39.629996Z","shell.execute_reply.started":"2022-08-29T01:53:39.629694Z","shell.execute_reply":"2022-08-29T01:53:39.629724Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"nyra_tracking['track_id'].value_counts()","metadata":{"execution":{"iopub.status.busy":"2022-08-29T01:53:39.631477Z","iopub.status.idle":"2022-08-29T01:53:39.632075Z","shell.execute_reply.started":"2022-08-29T01:53:39.631781Z","shell.execute_reply":"2022-08-29T01:53:39.631809Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"nyra_tracking['race_number'].value_counts()","metadata":{"execution":{"iopub.status.busy":"2022-08-29T01:53:39.633735Z","iopub.status.idle":"2022-08-29T01:53:39.634314Z","shell.execute_reply.started":"2022-08-29T01:53:39.634024Z","shell.execute_reply":"2022-08-29T01:53:39.634053Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"nyra_tracking['program_number'].value_counts()","metadata":{"execution":{"iopub.status.busy":"2022-08-29T01:53:39.636003Z","iopub.status.idle":"2022-08-29T01:53:39.636565Z","shell.execute_reply.started":"2022-08-29T01:53:39.636296Z","shell.execute_reply":"2022-08-29T01:53:39.636323Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Get date detail\nnyra_tracking['race_date'] = pd.to_datetime(nyra_tracking['race_date'])\nnyra_tracking['day'], nyra_tracking['month'] = nyra_tracking['race_date'].dt.day, nyra_tracking['race_date'].dt.month","metadata":{"execution":{"iopub.status.busy":"2022-08-29T01:53:39.637862Z","iopub.status.idle":"2022-08-29T01:53:39.638422Z","shell.execute_reply.started":"2022-08-29T01:53:39.638145Z","shell.execute_reply":"2022-08-29T01:53:39.638174Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Track Race Plot 🛣","metadata":{}},{"cell_type":"code","source":"nyra_tracking0 = nyra_tracking[nyra_tracking['track_id'] == \"AQU\"]\nnyra_tracking1 = nyra_tracking[nyra_tracking['month'] == 11]\n\nplt.style.use(\"dark_background\")\nplt.figure(figsize=(16, 8))\nsns.scatterplot(data=nyra_tracking1, x = 'longitude', y = \"latitude\", hue = \"race_number\", palette = \"Paired\")\nplt.title(\"track_id = AQU, Longitude and Latitude by Race Number\")\nplt.legend()","metadata":{"execution":{"iopub.status.busy":"2022-08-29T01:53:39.640644Z","iopub.status.idle":"2022-08-29T01:53:39.641215Z","shell.execute_reply.started":"2022-08-29T01:53:39.640937Z","shell.execute_reply":"2022-08-29T01:53:39.640966Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"nyra_tracking0 = nyra_tracking[nyra_tracking['track_id'] == \"BEL\"]\nnyra_tracking1 = nyra_tracking[nyra_tracking['month'] == 5]\n\nplt.style.use(\"dark_background\")\nplt.figure(figsize=(16, 8))\nsns.scatterplot(data=nyra_tracking1, x = 'longitude', y = \"latitude\", hue = \"race_number\", palette = \"Paired\")\nplt.title(\"track_id = BEL, Longitude and Latitude by Race Number\")\nplt.legend()","metadata":{"execution":{"iopub.status.busy":"2022-08-29T01:53:39.642886Z","iopub.status.idle":"2022-08-29T01:53:39.643441Z","shell.execute_reply.started":"2022-08-29T01:53:39.643151Z","shell.execute_reply":"2022-08-29T01:53:39.643177Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"nyra_tracking0 = nyra_tracking[nyra_tracking[\"track_id\"] == \"SAR\"]\nnyra_tracking1 = nyra_tracking0[nyra_tracking0[\"month\"] == 9]\n\nplt.style.use(\"dark_background\")\nplt.figure(figsize = (16, 8))\nsns.scatterplot(data = nyra_tracking1, x = \"longitude\", y = \"latitude\", hue = \"race_number\", palette = \"Paired\")\n\nplt.title(\"track_id = SAR, Longitude and Latitude by Race Number\")\nplt.legend()","metadata":{"execution":{"iopub.status.busy":"2022-08-29T01:53:39.644671Z","iopub.status.idle":"2022-08-29T01:53:39.645233Z","shell.execute_reply.started":"2022-08-29T01:53:39.644954Z","shell.execute_reply":"2022-08-29T01:53:39.644980Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"nyra_2019.race_number.hist(bins=12)","metadata":{"execution":{"iopub.status.busy":"2022-08-29T01:53:39.646969Z","iopub.status.idle":"2022-08-29T01:53:39.647537Z","shell.execute_reply.started":"2022-08-29T01:53:39.647238Z","shell.execute_reply":"2022-08-29T01:53:39.647265Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Trakus Index Race Animation 🏎","metadata":{}},{"cell_type":"code","source":"nyra_2019[\"jockey\"]","metadata":{"execution":{"iopub.status.busy":"2022-08-29T01:53:39.648930Z","iopub.status.idle":"2022-08-29T01:53:39.649476Z","shell.execute_reply.started":"2022-08-29T01:53:39.649190Z","shell.execute_reply":"2022-08-29T01:53:39.649215Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"jockey = nyra_2019[nyra_2019[\"jockey\"] == \"Samuel Camacho Jr.\"]\njockey6 = jockey[jockey[\"race_number\"] == 6]\njockeyBEL = jockey6[jockey6[\"track_id\"] == \"BEL\"]\njockeyALW = jockeyBEL[jockeyBEL[\"race_type\"] == \"ALW\"]\njockeyCLM = jockeyBEL[jockeyBEL[\"race_type\"] == \"CLM\"]\njockeyALW = jockeyALW.sort_values(by = \"trakus_index\", ascending = True)\n\nfig = px.scatter(jockeyALW, x = \"longitude\", y = \"latitude\", hover_data = [\"race_type\", \"track_id\"],\n animation_frame = \"longitude\",\n range_y = (min(jockeyALW['latitude']), max(jockeyALW['latitude'])),\n range_x = (max(jockeyALW['longitude']), min(jockeyALW['longitude'])),\n #size = \"trakus_index\",\n color = \"trakus_index\", color_continuous_scale = \"Turbo\", range_color = (0, 299),\n title = \"Samuel Camacho Jr. - Trakus Index, Race Number = 6, track_id = BEL\")\n\nfig.layout.updatemenus[0].buttons[0].args[1]['frame']['duration'] = 30\nfig.layout.updatemenus[0].buttons[0].args[1]['transition']['duration'] = 5\nfig.update_traces(marker = dict(symbol = \"circle-open\", size = 20)) # scaling the markers\nfig.update_layout(template = \"plotly_dark\", font = dict(family = \"PT Sans\", size = 12))\nfig.show()","metadata":{"execution":{"iopub.status.busy":"2022-08-29T01:53:39.651298Z","iopub.status.idle":"2022-08-29T01:53:39.651867Z","shell.execute_reply.started":"2022-08-29T01:53:39.651588Z","shell.execute_reply":"2022-08-29T01:53:39.651615Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"nyra_2019['jockey'].value_counts('Samuel Camacho Jr.')","metadata":{"execution":{"iopub.status.busy":"2022-08-29T01:53:39.653357Z","iopub.status.idle":"2022-08-29T01:53:39.653961Z","shell.execute_reply.started":"2022-08-29T01:53:39.653669Z","shell.execute_reply":"2022-08-29T01:53:39.653698Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"jockey = nyra_2019[nyra_2019[\"jockey\"] == \"Flavien Prat\"]\njockey6 = jockey[jockey[\"race_number\"] == 6]\njockeyBEL = jockey6[jockey6[\"track_id\"] == \"BEL\"]\njockeyALW = jockeyBEL[jockeyBEL[\"race_type\"] == \"ALW\"]\njockeyCLM = jockeyBEL[jockeyBEL[\"race_type\"] == \"CLM\"]\njockeyALW = jockeyALW.sort_values(by = \"trakus_index\", ascending = True)\n\nfig = px.scatter(jockeyALW, x = \"longitude\", y = \"latitude\", hover_data = [\"race_type\", \"track_id\"],\n animation_frame = \"longitude\",\n range_y = (min(jockeyALW['latitude']), max(jockeyALW['latitude'])),\n range_x = (max(jockeyALW['longitude']), min(jockeyALW['longitude'])),\n # size = \"trakus_index\",\n color = \"trakus_index\", color_continuous_scale = \"Turbo\", range_color = (0, 299),\n title = \"Flavien Prat - Trakus Index, Race Number = 6, track_id = BEL\")\n\nfig.layout.updatemenus[0].buttons[0].args[1]['frame']['duration'] = 30\nfig.layout.updatemenus[0].buttons[0].args[1]['transition']['duration'] = 5\nfig.update_traces(marker = dict(symbol = \"circle-open\", size = 20)) # scaling the markers\nfig.update_layout(template = \"plotly_dark\", font = dict(family = \"PT Sans\", size = 12))\nfig.show()","metadata":{"execution":{"iopub.status.busy":"2022-08-29T01:53:39.655487Z","iopub.status.idle":"2022-08-29T01:53:39.656100Z","shell.execute_reply.started":"2022-08-29T01:53:39.655809Z","shell.execute_reply":"2022-08-29T01:53:39.655838Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Jockey Flavien Prat - Trakus Index Race Pathing by Odds of Winning 📈","metadata":{}},{"cell_type":"code","source":"jockey = nyra_2019[nyra_2019[\"jockey\"] == \"Flavien Prat\"]\nraces_bel = jockey[jockey[\"track_id\"] == 'BEL']\n\nfor odds in sorted([690, 650, 1840, 70, 520, 2140, 400, 2775, 700, 160, 1910, 1030, 660]):\n \n df0 = races_bel[races_bel[\"odds\"] == odds]\n \n fig = px.scatter(df0, x = \"longitude\", y = \"latitude\", opacity = 0.8,\n hover_data = [\"race_date\", \"distance_id\", \"purse\"],\n color = \"trakus_index\", color_continuous_scale = \"Turbo\", #range_color = (0, 399),\n title = f\"Flavien Prat Trakus Index Race Pathing, Odds: {odds} track_id = BEL\")\n\n fig.update_traces(marker = dict(symbol = \"square-open\", size = 20)) # scaling the markers\n fig.update_layout(template = \"plotly_dark\", font = dict(family = \"PT Sans\", size = 12))\n fig.show()","metadata":{"execution":{"iopub.status.busy":"2022-08-29T01:53:39.657905Z","iopub.status.idle":"2022-08-29T01:53:39.658493Z","shell.execute_reply.started":"2022-08-29T01:53:39.658185Z","shell.execute_reply":"2022-08-29T01:53:39.658212Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"nyra_2019.head()","metadata":{"execution":{"iopub.status.busy":"2022-08-29T01:53:39.660233Z","iopub.status.idle":"2022-08-29T01:53:39.660823Z","shell.execute_reply.started":"2022-08-29T01:53:39.660526Z","shell.execute_reply":"2022-08-29T01:53:39.660552Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"nyra_2019.head()","metadata":{"execution":{"iopub.status.busy":"2022-08-29T01:53:39.662480Z","iopub.status.idle":"2022-08-29T01:53:39.663056Z","shell.execute_reply.started":"2022-08-29T01:53:39.662764Z","shell.execute_reply":"2022-08-29T01:53:39.662792Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"nyra_start['race_date'] = pd.to_datetime(nyra_start['race_date'])\nnyra_start['day'], nyra_start['month'] = nyra_start['race_date'].dt.day, nyra_start['race_date'].dt.month","metadata":{"execution":{"iopub.status.busy":"2022-08-29T01:53:39.664683Z","iopub.status.idle":"2022-08-29T01:53:39.665232Z","shell.execute_reply.started":"2022-08-29T01:53:39.664967Z","shell.execute_reply":"2022-08-29T01:53:39.664992Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"fig = px.scatter(nyra_start, x = \"weight_carried\", y = \"odds\", hover_data = [\"race_number\", \"jockey\"],\n color = \"odds\", color_continuous_scale = colors, range_color = (0, 9999),\n title = \"Odds by Weight Carried\")\n\nfig.update_traces(marker = dict(size = 8, symbol = \"circle\")) # scaling the markers\nfig.update_layout(template = \"plotly_dark\", font = dict(family = \"PT Sans\", size = 20))\nfig.show()","metadata":{"execution":{"iopub.status.busy":"2022-08-29T01:53:39.666884Z","iopub.status.idle":"2022-08-29T01:53:39.667538Z","shell.execute_reply.started":"2022-08-29T01:53:39.667152Z","shell.execute_reply":"2022-08-29T01:53:39.667179Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"\nfig = px.scatter(nyra_start, x = \"weight_carried\", y = \"day\", hover_data = [\"race_number\", \"jockey\"],\n color = \"odds\", color_continuous_scale = colors, range_color = (0, 9999),\n title = \"Odds by Day of Event and Weight Carried\")\n\nfig.update_traces(marker = dict(size = 9, symbol = \"square\")) # scaling the markers\nfig.update_layout(template = \"plotly_dark\", font = dict(family = \"PT Sans\", size = 20))\nfig.show()","metadata":{"execution":{"iopub.status.busy":"2022-08-29T01:53:39.670561Z","iopub.status.idle":"2022-08-29T01:53:39.671168Z","shell.execute_reply.started":"2022-08-29T01:53:39.670862Z","shell.execute_reply":"2022-08-29T01:53:39.670891Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"\nfig = px.scatter(nyra_start, x = \"weight_carried\", y = \"month\", hover_data = [\"race_number\", \"jockey\"],\n color = \"odds\", color_continuous_scale = colors, range_color = (0, 9999),\n title = \"Odds by Month of Event and Weight Carried\")\n\nfig.update_traces(marker = dict(size = 11, symbol = \"circle\")) # scaling the markers\nfig.update_layout(template = \"plotly_dark\", font = dict(family = \"PT Sans\", size = 20))\nfig.show()","metadata":{"execution":{"iopub.status.busy":"2022-08-29T01:53:39.681297Z","iopub.status.idle":"2022-08-29T01:53:39.682738Z","shell.execute_reply.started":"2022-08-29T01:53:39.682324Z","shell.execute_reply":"2022-08-29T01:53:39.682351Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"nyra_race['track_condition'].value_counts()","metadata":{"execution":{"iopub.status.busy":"2022-08-29T01:53:39.683741Z","iopub.status.idle":"2022-08-29T01:53:39.684137Z","shell.execute_reply.started":"2022-08-29T01:53:39.683950Z","shell.execute_reply":"2022-08-29T01:53:39.683968Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"fig = px.scatter(nyra_race, x = \"post_time\", y = \"run_up_distance\", hover_data = [\"race_number\", \"course_type\"],\n color = \"race_number\", color_discrete_sequence = colors, #range_color = (0, 9999),\n title = \"Post Time by Run Up Distance\")\n\nfig.update_traces(marker = dict(size = 4, symbol = \"circle\")) # scaling the markers\nfig.update_layout(template = \"plotly_dark\", font = dict(family = \"PT Sans\", size = 20))\nfig.show()","metadata":{"execution":{"iopub.status.busy":"2022-08-29T01:53:39.686104Z","iopub.status.idle":"2022-08-29T01:53:39.686545Z","shell.execute_reply.started":"2022-08-29T01:53:39.686321Z","shell.execute_reply":"2022-08-29T01:53:39.686339Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]}]}