{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# Introduction\n\nRearing horses to become championship calibre is not a cheap and easy task. While breed and jockey skill are important, a horse must be appropriately trained and certain progress benchmarks must be met. Once a horse is racing, they have the ability to make up for the costs of their training through winnings. The race and strategies are complex and dynamic. A jockey needs to control the race strategy - for example, by conserving their horse's energy via drafting while avoiding their horse getting boxed in and losing position. Given the costs and complexity of horse racing, statistical models capable of better understanding and valuing horses, jockeys, and strategies can greatly benefit owners and team members by providing insights into their horses. \n\nIn this project, we develop a workhorse simulation model which allows us to create accurate and realistic simulations of horse races and to answer new questions not previously possible with quantitative methods. The underlying philosophy of our workhorse model is derived from visualizing horse movement at the frame-by-frame level as the composition of two types of movement: forward and side-to-side (lateral). Together, the forward and lateral movements tell us exactly where a horse is on the track at every moment in time. By analyzing at the frame level, we are able to model and better understand the complicated nature of how a horse's movement depends on the movement and positioning of all other horses on the track. Getting to the inside of the track requires an open gap, while drafting requires a horse to be in front of you. The benefits of drafting increase as the position of the lead horse becomes more optimal for the trailing horses. Since the exact positioning of each horse depends on the speed and manouevering in all previous frames, we are able to link the impact of movements at this granular level to the possible progressions of the race from that point onward.\n\nAt a high-level, our model can describe how a horse's forward and lateral movements depend on various factors, such as the jockey, the track (e.g. turf vs dirt; muddy, sloppy, good, fast, etc.), the current portion of the track (e.g. chute, turn, stretch), and the relative dynamic positioning of the horse with respect to all other horses racing, in addition to other factors. In particular, forward and lateral movement will depend on the horse's profile (i.e. racing characteristics at various race stages relative to the average horse). This is described in more detail later on. The modelling framework and methodology we develop for our analysis is versatile such that it can be applied to most race situations. It takes the form of a dynamic model which can simulate thousands of replications of any race, for any horses, with any set of jockeys, from any point in the race, and on any combination of track and track condition. By engineering a drafting model for horses and using statistical techniques, we can provide a continuous time estimate of a horse's in-race finish placement probability. This is done by executing a counterfactual strategy by computing in-race simulations of placement scenarios using two fitted models: one for a horse's forward speed and one for its lateral speed. All performance analysis that we investigate, including attributes related to the horse, jockey, and strategy, allows us to accurately produce this placement probability. The adaptability of the novel modelling methodology we propose is abundant. Our methods can be adapted to answer horse welfare questions, compute and analyze race collision probabilities, and analyze horse and jockey profiles and strategies, given appropriate supplemental data.","metadata":{}},{"cell_type":"markdown","source":"# Data Preparation\n\nWe perform multiple operations in order to transform the data to suit our needs. Firstly, track outline and finish line data is manually gathered using Google Earth. The on-track position of each horse is determined by converting the provided longitude and latitude coordinates to metres via the Haversine formula. With the Haversine formula and manually gathered track data, we can determine each horses' location with respect to the track. We create a framed data set to determine each horse's location on the track with respect to (i) the distance they have travelled forward along the track, (ii) their lateral positioning from the inside of the track, and (iii) the total raw (Euclidean) distance they have travelled at each frame (every 0.25 seconds). We use these measures of location (forward, lateral and Euclidean) to create a suite of metrics that quantify a horse's positioning relative to the competition. Figure 1 illustrates the distance to the nearest horse for each of these three measurements. The forward distance is calculated parallel to the race direction, Euclidean distance is calculated as the total distance travelled, and the lateral distance is the distance travelled perpendicular to the inner boundary of the track. Using these distances travelled between frames, we are able to determine horse positions during the race. Further, we can determine the forward, lateral, and Euclidean distance between any two horses; from this we can determine if a horse is in a draft position as well as its future possible trajectories. When necessary, we smooth the trajectory of a horse using an imputation based on their opponents acceleration patterns. Going forward, we use the term frame to refer to the race at a specific time.\n\n
\"centered
\nFigure 1: An animation of Race 3 at Belmont Park on 2019-05-16. Circles represent the forward, Euclidean, and lateral distances to the nearest horse, respectively. (Please be patient as the gif may take a moment to load properly)","metadata":{}},{"cell_type":"markdown","source":"Figure 2 provides a full overview of our data preparation process for this project. A large portion of our work involved extracting and matching track outlines to the tracking data and smoothing out inconsistencies to help improve model accuracy and performance.\n\n
\"centered
\nFigure 2: An overview of our data preparation process.","metadata":{}},{"cell_type":"markdown","source":"# Feature Engineering\n\nWe construct multiple features used for the novel methodology. By computing the forward, Euclidean, and lateral distance (and position) of each horse frame-by-frame, we are able to determine for each horse the number of opponents a horse is surrounded by on either side and in front during a race. With this, we can construct a set of dynamic features for each horse that describes its relative position and movable space in its immediate area. This further allows us to engineer a drafting model, which we describe in the next subsection. We further adapt our methodology on track characteristics, adjust for the effect of course type and track condition. We also generate horse and jockey effects features, which is discussed in the next section. The figure below provides a summary of features, in addition to which predictors will be used in our model going forward. \n\n
\"centered
\nFigure 3: A list of covariates and random effects used in our models for forward and lateral movement. Note that $*$ items are omitted from the forward movement model and † items are omitted from the lateral movement model.","metadata":{}},{"cell_type":"markdown","source":"## Drafting\n\nThe primary feature we engineer is a horse drafting model which has been shown to be an important factor in horse racings ([Spence et al](https://royalsocietypublishing.org/doi/10.1098/rsbl.2011.1120)). In order to draft, a horse is required to remain behind another in order to draft at all, potentially sacrificing position and/or speed in that moment. The benefit comes in the form of saved energy and potentially increased speed in the later stages of the race. What matters when deciding to draft is whether the set of race paths from that moment forward are improved or not. One also has to be careful to separate horses and jockeys particularly adapted to certain strategies from the strategies themselves. \n\nTo create our drafting feature, we created a three-dimensional computer aided design (CAD) of a horse and jockey. With this design, we used the open source software OpenFOAM to analyze the computational fluid dynamics of the model and construct simulations. We obtain multiple drag coefficients for various race scenarios, including when a horse is in clear air and when it is drafting. See Figure 4 for visualizations of the fluid dynamics. Using the found drag coefficients, we are able to compute the estimated energy expenditure of a horse over time in clear air and in a drafting position. We further calculate the drag force that is exerted on a horse at various speeds using these coefficients.\n\n
\"centered
\nFigure 5: An illustration of our drafting model and simulations.","metadata":{}},{"cell_type":"markdown","source":"# Model and Simulation\n\nTo build our model, we use races that are one-mile long. Our methodology can easily be extended to races of differing lengths in a hierarchical scheme, and we make this choice for both demonstrative purposes and computational efficiency. We develop two models for estimating a horse's speed in different spatial dimensions, which when combined allows us to generate race simulations. This can be seen as a multivariate extension to work modelling non-rank outcomes in multi-competitor games, in particular [Glickman and Che (2022)](https://arxiv.org/abs/2205.10746).\n\nThe first is a forward-movement (FM) model. This models the forward speed of a horse as a function of its cumulative distance on the track, both of which use the forward distance of a horse which we compute in the data preparation. For each horse, we perform a technique called b-splines regression, which allows us to account for individual horse strengths and strategies at different stages along the track. This spline technique encodes the knowledge that a horse's average speed at any point in a race is likely to smooth without assuming too much about what that function looks like exactly and using the data to best decide. Specifically, spline regression is a type of non-linear regression technique that incorporates both linear and polynomial functions to fit to the data. Spline regression can incorporate breakpoints (knots) at certain data values where the fitted model fundamentally changes, which can be learned by the model. This technique generalizes the standard linear regression framework which allows for the flexibility and versatility necessary for modelling horse speeds across an entire race. The splines are fitted using all tracks, and the knot placement for the splines was decided both using a leave-one-out cross-validation technique as well as a visual assessment. The knot placement corresponds roughly to strategy transitions, for instance the end of the initial acceleration at the start of the race as well as the final quarter-mile. Using the b-splines generated with the [splines2 package](https://cran.r-project.org/web/packages/splines2/index.html), we fit an approximate Bayesian model (MAP estimates) with the optim functions in the [rstan package](https://mc-stan.org/users/interfaces/rstan) to estimate the horses' speed. Specifically, we model\n\n$$ \\text{Speed}_F \\sim N(\\beta_{FM} H_{spline} + \\gamma_{FM} X_{FM},\\sigma_{FM}),$$\n\nwhere $\\text{Speed}_F$ represents the estimated forward speed, $H_{spline}$ contains the horses' fitted b-spline, $\\beta_{FM}$ the corresponding coefficients, $X_{FM}$ contains select predictors from the original data and those we compute, and $\\gamma_{FM}$ the corresponding predictor coefficients. For $\\beta_{FM}$, $\\gamma_{FM}$, and $\\sigma_{FM}$ we choose weakly informative priors that allow us to influence the model fitting for speed as little as possible. The predictors we include in the FM model are all those listed in Figure 3 omitting $*$ items.\n\nThe second model we construct is a side movement (SM) model. This models the lateral speed of each horse. Throughout this work we use the terms side movement and lateral movement interchangeably. For this model, we again use a Bayesian framework of the form\n\n$$\\text{Speed}_l \\sim N(\\beta_{SM} PLS + \\gamma_{SM} X_{SM} + \\sigma_{SM} ), $$\n\nwhere $\\text{Speed}_l$ represents the estimated lateral speed of the horse, $PLS$ is the horses' lateral speed from the previous frame, $\\beta_{SM}$ it's corresponding fitted coefficient, $X_{SM}$ the selected predictors for the SM model, and $\\gamma_{SM}$ the corresponding predictor coefficients. We use semi-informative priors for $\\beta_{SM}$, $\\gamma_{SM}$, and $\\sigma_{SM}$ that are scaled proportionally to the movement speeds that are possibly for a horse, however we maintain that these are chosen in such a way that avoids heavily influencing the model outcome. The predictors we use for the SM model all those listed in Figure 3 omitting $\\dagger$ items. By modelling a horse's lateral speed, and determining its lateral movement and whether or not it is drafting, we are able to analyze a jockey's strategy. Further, by analyzing the horse's position with respect to its opponents, we can determine whether it is currently boxed in (i.e. immediately surrounded by opponents on either side and in front), and hence unable to break away from the pack at a faster pace than their opponents.\n\nBy using both the FM model and SM model, we can simulate races using a horses' current position. We produce these simulations frame-by-frame, and simulate the forward and lateral speeds and compute the corresponding movements of each horse in a race to compute the dynamic win probability for each horse. See Figure 5 for a summary of the simulation algorithm. The simulated lateral and forward speeds for each horse, in addition to the horse's modelled spline, produces a horse profile, which can be compared across different horses to analyze strategy and race performance on a more granular level. The horse profile can also be used to compare novice horses to seasoned race horses to determine if they are ready to compete in high stakes races. We summarize our end-to-end modelling procedure including model applications in Figure 6.\n\n
\"centered
\nFigure 5: An illustration of our algorithm to run race simulations. Note that side movement is exaggerated in the middle panel for illustrative purposes.","metadata":{}},{"cell_type":"markdown","source":"
\"centered
\nFigure 6: An illustration of our end-to-end model workflow.","metadata":{}},{"cell_type":"markdown","source":"# Results","metadata":{}},{"cell_type":"markdown","source":"## Race Simulation and Dynamic Win Probability\n\nAt each race frame, our model performs multiple simulations that predict the remainder of the race. Using the extracted finishing position of horses in these simulations, we are able to construct a dynamic race win probability for each horse. As more information is provided with each frame that passes, these predictions become more accurate and converge to the true race result. We present how our dynamic win probabilities behave using race three from the Belmont which occurred May 16, 2019.\n\n![](https://github.com/brenkumi/canadian-pharoah/blob/main/Figures/BEL_20190516_3_dynamic_win_prob.gif?raw=true)\nFigure 7: An animation of the dynamic win probability estimates from our model simulations for Race 3 at Belmont Park on 2019-05-16. (Please be patient as the gif may take a moment to load properly)\n\nOn the left table of the visualization, the left-most column corresponds to the probability a horse finishes in first, and the right-most column corresponds to a horse finishing last. On the right is how the actual race progressed. As the true race progresses, the win probabilities are being updated given the new information of the race characteristics until the race finishes and the true finish order is determined. Analyzing this visualization we see that Curlin's New Moon is sitting back in the first half and in a draft position. Despite sitting around fifth and seventh place, our model predicts that this horse will have a top finish placement during the early stages of the race. Curlin's New Moon ultimately finished second this race, demonstrating our model's capability to capture effects that would otherwise be lost to the human eye. This not only demonstrates the effectiveness of our dynamic win probability model, but also provides validity to our choice to incorporate a drafting feature. We can construct the above visualization for any mile-long race in the originally provided data set.\n\nThis is just one demonstration of the power of our dynamic simulation model capable of forward simulating from any point and position of any race. Working together with horse racing experts one could use this same simulator to better understant to value of different positions at different points in the race. A simple such application would be testing the value of different starting lanes for different horses or horse-types.","metadata":{}},{"cell_type":"markdown","source":"## Jockey Ratings\n\nFrom the FM model, we can compute the posterior mean of the jockey's random effect to produce a jockey rankings measure. The ability to compute this demonstrates the benefit of our choice to model in a Bayesian framework as well as our methodology's flexibility. This measure quantitatively describes the positive impact that a jockey has on their horse's estimated final position (i.e. the higher the rating, the greater the positive impact on race result). See Figure 8 for the top ten jockey ratings produced by the modelling procedure. We compare our rating to three true track ranks for our model's top ten jockeys. We select Saratoga, Belmont, and Aqueduct for the track. We find that Irad Ortiz Jr., the top rated jockey from our model, also ranks first at Saratoga and Belmont, and fourth at Aqueduct. Our model's top jockeys have performed admirably at at least one of the selected tracks, omitting Joe Bravo who has still achieved 54 first place finishes in 2022 by the date of this report. Our model is able to accurately determine who the best jockeys are and their impact on race results. Further research can investigate these top jockeys and both quantitatively and qualitatively analyze strategy decisions made by the jockeys in a race.\n\n
\"centered
\nFigure 8: A table with the top ten rated jockeys from our model with our jockey rating along with each jockey's rank provided by Saratoga, Belmont Park and Aqueduct, respectively.","metadata":{}},{"cell_type":"markdown","source":"## Horse Profiles\n\nUsing the B-splines from our FM model that quantify the horse’s effect on winning a race, we can analyze horse tendencies and strengths with respect to the distance travelled along the track. To make these splines more digestible we perform hierarchical clustering with Ward’s linkage on all horses that competed in at least 5 races in our data. As a result, we obtain 3 clusters which we label “Strong Build, Slow Finish” (blue), “Medium Build, Medium Finish” (red), and “Slow Build, Strong Finish” (green) as shown in Figure 9. The Strong Build, Slow Finish group has exceptional acceleration over the first 100m but slowly trails off throughout the race. The Medium Build, Medium Finish group takes a bit longer to reach its top speed but maintains it well throughout the race. The Slow Start, Strong Finish group takes longer to reach the top speed but holds a higher impact on speed throughout most of the race and has an additional burst of energy at the end of the race.\n\n
\"centered
\nFigure 9: Splines for each of the 3 cluster centres.","metadata":{}},{"cell_type":"markdown","source":"Additionally, we provide the resulting dendrogram from our hierarchical clustering results in Figure 10. This gives us a sense of the relationship between horses with respect to their speed patterns.\n\n
\"centered
\nFigure 10: A hierarchical clustering dendrogram based on all horses with 5 or more 1 mile races. Red borders are used to divide clusters.","metadata":{}},{"cell_type":"markdown","source":"## Shiny App\n\nTaken together, our forward and lateral movement models and simulation functions allow us to develop a race simulator. In Figure 11, we demonstrate an app we created using RShiny which allows the user to specify race characteristics and conditions, and choose horses, jockeys, and their starting positions at the gate. A user-specified number of simulations with this race setup will be run, and using the results of all simulations, final finishing position probabilities will be obtained for each horse/jockey pair. Although more work would be needed to fully develop the app, our model framework provides plenty of opportunity to expand on this. In particular, we could obtain more detailed simulation summaries, such as finishing time probability distributions for each horse. We could also modify the simulator to start the simulations from any point along the track, with any chosen positioning of the horses relative to each other to determine how these positions impact the race. In consultation with horse racing experts, this would allow us to answer questions regarding optimal strategy.\n\n
\"centered
\nFigure 11: A video of our proposed Shiny app for simulating race outcome and obtaining win probability.","metadata":{}},{"cell_type":"markdown","source":"# Conclusion and Future Work\n\nOur multi-step modelling approach for producing dynamic win probabilities for each horse is a novel methodology for horse racing analysis. The spline regression, FM, and SM models allow us to effectively simulate possible race outcomes, in addition to generating specific horse profiles which can be used to analyze race strategy and horse characteristics. The work performed is of importance to the sport for a variety of reasons. Due to the expensive process of rearing a horse to become race calibre, owners need to recuperate the costs through race winnings. With our model, an owner can compare their horse in different scenarios and estimate their horse's race outcome prior to the race beginning, and dynamically update its predicted placement during the race. An owner can additionally pair their horse with a jockey who's strategy decisions optimally fit with the horse's profile. While our model is only generated using mile-long races, it is easily extendable to different track lengths.\n\nDue to the versatility of the model in addition to the data preparation performed, the future work with the framework we have produced is abundant assuming appropriate predictors. Since we computed the number of opponents beside and in front of a horse, as well as their distances, we are able to detect how boxed in a horse is. If given historical data of racing accidents, we can extend our current model to compute a dynamic collision probability of a horse in addition to its win probability. In addition, if we are provided with in race heart rate and injury data for a horse, we can adapt our model to analyze a horse's welfare. This can be done be producing an injury probability of a horse or estimating the physical stress a horse will face in a future race. We can further investigate racing strategies at a more granular level. The proposed modelling framework can be adapted to investigate the advantages and disadvantages of a horse's starting position, and it's impact on win probability. Using the computed drafting feature, one can also construct the win probability as a function of a drafting strategy, as well as how deviations from the drafting will impact win probability.","metadata":{}}]}