{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"code","source":"# Import desired packages\nimport numpy as np \nimport pandas as pd\nimport geopy.distance\nimport plotly.express as px\n\nfrom IPython.display import Image","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-09-14T07:00:10.224188Z","iopub.execute_input":"2022-09-14T07:00:10.224635Z","iopub.status.idle":"2022-09-14T07:00:11.949314Z","shell.execute_reply.started":"2022-09-14T07:00:10.224546Z","shell.execute_reply":"2022-09-14T07:00:11.948084Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Aims\n\nThe key aim is to transform the GPS data for a race into an x,y based visualisation upon which other metrics such as speed, distance and energy could be calculated. It may not be the best method of transforming the data but it appears to do a pretty good job.","metadata":{}},{"cell_type":"code","source":"Image(filename=\"../input/belmontgoogleearth/Belmont Park Earth.png\")","metadata":{"execution":{"iopub.status.busy":"2022-09-13T11:42:33.576253Z","iopub.execute_input":"2022-09-13T11:42:33.576645Z","iopub.status.idle":"2022-09-13T11:42:33.643930Z","shell.execute_reply.started":"2022-09-13T11:42:33.576610Z","shell.execute_reply":"2022-09-13T11:42:33.642653Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Extra data needed\n\nTo make the plots, some additional data is required, but not a lot. The Google Earth image above shows all the additional inputs. I drew a line level with the home straight at Belmont Park and identified the approximate location of the start of the home straight and the finish line based on the big red roset thing.\n\nThe latitude/longitude co-ordinates for these elements are:  \nHome Line 1:   40.71475945171949, -73.7291971926194  \nHome Line 2:   40.71140369439631, -73.71632313180034  \nHome Straight: 40.71405567333758, -73.72649990114551  \nFinish Line:   40.71330475501588, -73.72260194071144 \n\nI've added these to belmont_earthlatlong.csv, but they appear to show one less decimal point for some of the locations so it's worth checking them and how much of a difference this may make. I will type them in raw format for my peace of mind.","metadata":{}},{"cell_type":"markdown","source":"# Load in GPS Data\n\nTo keep computation times low and everything simple, I'm going to load the dataset, then immediately reduce it down to the only race I'm interested in. In this case the Woody Stephens stakes - a 7f race on Belmont Stakes Day.\n","metadata":{}},{"cell_type":"code","source":"# Load data and rename columns.\n# Thank you to Dmitry Uarov on the forum for this little snippet of code.\ndata = pd.read_csv('../input/big-data-derby-2022/nyra_2019_complete.csv', header=None)\n\ncols = ['track_id', 'race_date', 'race_number', 'program_number', 'trakus_index', 'latitude', 'longitude', 'distance_id', 'course_type', 'track_condition', 'run_up_distance', 'race_type', 'purse', 'post_time', 'weight_carried', 'jockey', 'odds', 'final_place']\n\ndata.rename({i: col for i, col in enumerate(cols)}, axis=1, inplace=True)\n\n# Take only the race data. We are interested in track_id 'BEL', race_date '2019-06-08', race_number 8.\nrace_data = data.loc[(data['track_id'] == 'BEL') & (data['race_date'] == '2019-06-08') & (data['race_number'] == 8)].copy()\nrace_data.head()","metadata":{"execution":{"iopub.status.busy":"2022-09-13T11:01:28.751349Z","iopub.execute_input":"2022-09-13T11:01:28.751936Z","iopub.status.idle":"2022-09-13T11:01:45.562522Z","shell.execute_reply.started":"2022-09-13T11:01:28.751886Z","shell.execute_reply":"2022-09-13T11:01:45.561252Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Organising the data and removing any potential typos\n\nThe data is in a random order so let's sort it by program_number and trakus_index so that each horse has all of it's data grouped together in order. Before we do that, we will give all the program_numbers new uniform values so that there can't be any labelling issues for any of the horses. For some reason, the labels seem to switch between a 3-character string (e.g. '1  ') and the integer equivalent.","metadata":{}},{"cell_type":"code","source":"# Create dictionary of string numbers\nnew_prog_numbers = {'1  ': '001', 1: '001', \n                    '2  ': '002', 2: '002', \n                    '3  ': '003', 3: '003', \n                    '4  ': '004', 4: '004', \n                    '5  ': '005', 5: '005', \n                    '6  ': '006', 6: '006', \n                    '7  ': '007', 7: '007', \n                    '8  ': '008', 8: '008',\n                    '9  ': '009', 9: '009', \n                    '10 ': '010', 10: '010', \n                    '11 ': '011', 11: '011'}\nrace_data['program_number'] = race_data['program_number'].map(new_prog_numbers).astype('str')\nrace_data = race_data.sort_values(['program_number', 'trakus_index'])\nrace_data.head()","metadata":{"execution":{"iopub.status.busy":"2022-09-13T11:01:50.446500Z","iopub.execute_input":"2022-09-13T11:01:50.446876Z","iopub.status.idle":"2022-09-13T11:01:50.483050Z","shell.execute_reply.started":"2022-09-13T11:01:50.446845Z","shell.execute_reply":"2022-09-13T11:01:50.481565Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Calculating x,y co-ordinates\n\nCalculating the xy co-ordinates is done by reverse-engineering some trigonometry equations. All co-ordinates are created relative to the home straight line we drew in Google Earth. We calculate the length of the home straight line, the distance from point 1 to the GPS co-ordinate and the distance from point 2 to the GPS co-ordinate. Using these three distances, we can create a triangle, which allows us to triangulate the position of the horse within our co-ordinate system. All distances are calculated in metres using the vincenty distance, which accounts for the curvature of the earth and provides 'flat' distances. \n\nUnfortunately, I can't find the link to the question I used to get here. I googled something along the lines of \"convert latitude longitude to xy\" if you want to try and find it.","metadata":{}},{"cell_type":"code","source":"def xy_coordinates(xy, latitude, longitude):\n    \n    # Home straight latitude, longitude tuples\n    HS1 = (40.71475945171949, -73.7291971926194)\n    HS2 = (40.71140369439631, -73.71632313180034)\n    \n    # Horse GPS location tuple\n    location = (float(latitude), float(longitude))\n    \n    # Calculate distances between the three points of the triangle\n    HS1_HS2 = geopy.distance.geodesic(HS1, HS2).m\n    HS1_loc = geopy.distance.geodesic(HS1, location).m\n    HS2_loc = geopy.distance.geodesic(HS2, location).m\n    \n    # Triangulate the x and y co-ordinates of the horse\n    loc_x = ((HS1_HS2**2) + (HS1_loc**2) - (HS2_loc**2)) / (2*HS1_HS2)\n    loc_y = abs((HS1_loc**2) - (loc_x**2)) ** 0.5\n    \n    # Return x or y co-ordinate\n    if xy == 'x':\n        return loc_x\n    elif xy == 'y':\n        return loc_y\n    else:\n        raise ValueError(\"xy should be 'x' for x co-ordinate or 'y' for y co-ordinate\")\n\nrace_data['x'] = race_data.apply(lambda row: xy_coordinates('x', row['latitude'], row['longitude']), axis=1)\nrace_data['y'] = race_data.apply(lambda row: xy_coordinates('y', row['latitude'], row['longitude']), axis=1)","metadata":{"execution":{"iopub.status.busy":"2022-09-13T11:01:59.373950Z","iopub.execute_input":"2022-09-13T11:01:59.375898Z","iopub.status.idle":"2022-09-13T11:02:04.772265Z","shell.execute_reply.started":"2022-09-13T11:01:59.375836Z","shell.execute_reply":"2022-09-13T11:02:04.771108Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Creating movement vectors\n\nTo help us to identify the rail location, we are now going to calculate the movement vectors of the horses. We do this by subtracting the previous location from the current location to give us a 0.25s movement vector.","metadata":{}},{"cell_type":"code","source":"# First shift up the previous timepoint so the calculations are easier. Make sure this is grouped by program_number so\n# only the horse's specific locations are considered.\nrace_data['x1'] = race_data.groupby(['program_number']).x.shift(1)\nrace_data['y1'] = race_data.groupby(['program_number']).y.shift(1)\n\n# Calculate movement vector in x and y directions\nrace_data['x_x1'] = race_data['x'] - race_data['x1']\nrace_data['y_y1'] = race_data['y'] - race_data['y1']","metadata":{"execution":{"iopub.status.busy":"2022-09-13T11:02:30.062877Z","iopub.execute_input":"2022-09-13T11:02:30.063270Z","iopub.status.idle":"2022-09-13T11:02:30.076941Z","shell.execute_reply.started":"2022-09-13T11:02:30.063239Z","shell.execute_reply":"2022-09-13T11:02:30.074628Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Calculating the rail location\n\nThis part is a bit more fiddly. I've only tested it on this 7f race with one curve, but that's all I'll need for my submission so you might find you have to make changes for any races with more than one curve.  \n\nWe take the perpendicular vector from each horse to find a possible rail location, then take the rail location that is closest to a point in the centre of the course. This central point is either (mean x-coordinate of all horses, middle of track) or (home straight entrance x, middle of track) dependent on whether the horses are on the curve or straight.\n\nCalculate this for all trakus ids and add the coordinates to a list so we can plot them.","metadata":{}},{"cell_type":"code","source":"# First get home straight x and y co-ordinates\nhome_x = xy_coordinates('x', 40.71405567333758, -73.7264990114551)\nhome_y = xy_coordinates('y', 40.71405567333758, -73.7264990114551)\n\n# Get the finishing line y-coordinate so the average can be taken between this and an approximate upper rail location\n# for the central point\nfinish_x = xy_coordinates('x', 40.71330475501588, -73.72260194071144)\nfinish_y = xy_coordinates('y', 40.71330475501588, -73.72260194071144)\n\n# Set up loop\nx_coords = []\ny_coords = []\nindexes = sorted(race_data.trakus_index.unique())\n\n# Start from trakus index 2 because we have no movement vector for trakus index 1\nfor idx in indexes[1:]:\n    \n    # Get data for trakus index only\n    line_data = race_data.loc[race_data['trakus_index'] == idx].copy()\n    \n    # Set values for the comparison point\n    # If any of the horses are going around the bend, use the home straight x co-ordinate\n    if line_data['x'].min() < home_x:\n        x_mean = home_x\n    # Otherwise, take the mean of all horses in the race and use that as the comparison point\n    else:\n        x_mean = race_data['x'].mean()\n        \n    # The y coordinate is the same throughout. I eyeballed the 420 as an approximate rail position - you could use\n    # Google Earth to plot the rail as well. Finish_y is the inside rail on the finishing straight because that's\n    # where the winning post is.\n    y_mean = (420-finish_y)/2\n    \n    # Now we find the perpendicular vector. I provide the equation to get the unit vector. My vector has a magnitude of\n    # 10 (vec_norm is divided by 10) for plotting purposes (i.e. so we can see the rail). This is not the actual position\n    # of the rail and will be changed for my full analysis.\n    line_data['vec_norm'] = np.linalg.norm(line_data[['x_x1', 'y_y1']].values) / 10\n    line_data['perp_x'] = line_data['y_y1']/line_data['vec_norm']\n    line_data['perp_y'] = -line_data['x_x1']/line_data['vec_norm']\n    \n    # Subtracting the perpendicular vector from the horses position gives the potential rail coordinates\n    line_data['rail_x'] = line_data['x'] - line_data['perp_x']\n    line_data['rail_y'] = line_data['y'] - line_data['perp_y']\n    \n    # Now we calculate the distances to the centre point\n    line_data['centre_dist'] = np.linalg.norm(line_data[['rail_x','rail_y']].values - np.array([x_mean, y_mean]))\n    # Then take the coordinate with the minimum distance from the central point\n    x_coords.append(line_data.loc[line_data['centre_dist'].idxmin()]['rail_x'])\n    y_coords.append(line_data.loc[line_data['centre_dist'].idxmin()]['rail_y'])","metadata":{"execution":{"iopub.status.busy":"2022-09-13T11:20:07.061792Z","iopub.execute_input":"2022-09-13T11:20:07.062231Z","iopub.status.idle":"2022-09-13T11:20:08.996339Z","shell.execute_reply.started":"2022-09-13T11:20:07.062192Z","shell.execute_reply":"2022-09-13T11:20:08.995210Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Plotting the race\n\nNow we have everything we need so we can plot the race. I've added a small mapping function to provide the approximate\ncolours of each horse on the plot as well. I use plotly express here as I've picked it up from other examples. In truth I expect it would be more valuable to create an animation which follows the horses around in close up - I just don't know how to do that (yet).","metadata":{}},{"cell_type":"code","source":"# Jockey colours\nrace_colour_map = {\"001\": \"pink\", \"002\": \"navy\", \"003\": \"red\", \"004\": \"white\", \"005\": \"teal\", \"006\": \"maroon\",\n                 \"007\": \"yellow\", \"008\": \"lime\", \"009\": \"cyan\", \"010\": \"gold\", \"011\": \"black\"}\n\n# Create figure\nfig = px.scatter(race_data.sort_values(by='trakus_index'),\n                x='x', y='y', animation_frame='trakus_index',\n                range_x=[30,1050], range_y=[0,450],\n                color='program_number', color_discrete_map=race_colour_map)\n\n# Add the rail\nfig.add_scatter(x=x_coords, y=y_coords)\n\n# Finally, add the finish line as we have the coordinates\nfig.add_shape(type='line', x0=finish_x, y0=0, x1=finish_x, y1=finish_y,\n             line=dict(color='black',), xref='x', yref='y')\n\nfig.show()","metadata":{"execution":{"iopub.status.busy":"2022-09-13T11:34:26.472965Z","iopub.execute_input":"2022-09-13T11:34:26.473408Z","iopub.status.idle":"2022-09-13T11:34:39.066610Z","shell.execute_reply.started":"2022-09-13T11:34:26.473373Z","shell.execute_reply":"2022-09-13T11:34:39.065743Z"},"trusted":true},"execution_count":null,"outputs":[]}]}