{"metadata":{"kernelspec":{"name":"ir","display_name":"R","language":"R"},"language_info":{"name":"R","codemirror_mode":"r","pygments_lexer":"r","mimetype":"text/x-r-source","file_extension":".r","version":"4.0.5"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"code","source":"suppressMessages({\n    library(dplyr)\n    library(ggplot2)\n    library(sf)\n    library(units)    \n})","metadata":{"_uuid":"051d70d956493feee0c6d64651c6a088724dca2a","_execution_state":"idle","execution":{"iopub.status.busy":"2022-09-30T17:39:43.830476Z","iopub.execute_input":"2022-09-30T17:39:43.832565Z","iopub.status.idle":"2022-09-30T17:39:45.133778Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Introduction","metadata":{}},{"cell_type":"markdown","source":"The intent of this investigation is to find races that ended up with a horse getting injured or getting in some kind of distress. The simplest way I can think of to detect the situation like that is by finding cases when a horse did not finish the race. Possible ways to detect that:\n- find races where the horse speed suddenly drops down to zero some time after having run for a while and never recovers\n- on the last tracking snapshot find horses that are significantly behind other horses in the race\n- find distance traveled by the horse during the race and find out if that distance is shorter than the set race length\n\nI am trying to use the last approach at first as it seems to be the simplest. The other two approaches are TBD.\nDistance traveled by horse during the race can be calculated with the `sf` package `st_length` function.","metadata":{}},{"cell_type":"markdown","source":"## Find the distances traveled during the race","metadata":{}},{"cell_type":"markdown","source":"Read the race tracking data.","metadata":{}},{"cell_type":"code","source":"tt <- read.csv(\"../input/big-data-derby-2022/nyra_tracking_table.csv\")\ntt$track_id <- factor(tt$track_id)\ntt$race_date <- as.Date(tt$race_date)\ntt$program_number <- factor(trimws(tt$program_number))","metadata":{"execution":{"iopub.status.busy":"2022-09-30T17:39:59.389293Z","iopub.execute_input":"2022-09-30T17:39:59.425143Z","iopub.status.idle":"2022-09-30T17:41:03.419615Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Transform loaded lat/long coordinates into SF simple features object","metadata":{}},{"cell_type":"code","source":"tt_sf <- st_as_sf(x = tt, coords = c('longitude', 'latitude')) %>%  st_set_crs(\"EPSG:4326\")","metadata":{"execution":{"iopub.status.busy":"2022-09-30T17:41:03.422187Z","iopub.execute_input":"2022-09-30T17:41:03.423511Z","iopub.status.idle":"2022-09-30T17:41:07.200211Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Using SF distance function find the path that each horse followed and the length. Transform individual simple feature points into multipoint paths for each race and each horse and apply `st_lenght` to the path the horse followed. This operation takes a bit of time.","metadata":{}},{"cell_type":"code","source":"tt_sf_dist <- tt_sf %>% group_by(track_id, race_date, race_number, program_number) %>%  \n  arrange(trakus_index) %>% summarise(do_union = F, .groups=\"drop\") %>%\n  st_cast(to='LINESTRING') %>% mutate(distance=st_length(geometry))","metadata":{"execution":{"iopub.status.busy":"2022-09-30T17:41:24.278825Z","iopub.execute_input":"2022-09-30T17:41:24.280444Z","iopub.status.idle":"2022-09-30T17:48:21.587249Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Compare distance traveled with the race length","metadata":{}},{"cell_type":"markdown","source":"Load track information. We care about actual expected race length and the track kind for future evaluation. Track data tells us the expected race length track condition and the race type.","metadata":{}},{"cell_type":"code","source":"rt <- read.csv(\"../input/big-data-derby-2022/nyra_race_table.csv\")\nrt$race_date <- as.Date(rt$race_date)\nrt_cats <- c(\"track_id\", \"course_type\", \"track_condition\", \"race_type\")\nrt[rt_cats] <- lapply(rt[rt_cats], factor)\nrt$track_distance_m <- rt$distance_id/100*201.168\nrt$run_up_distance_m <- rt$run_up_distance*0.3048","metadata":{"execution":{"iopub.status.busy":"2022-09-30T17:48:21.590954Z","iopub.execute_input":"2022-09-30T17:48:21.592541Z","iopub.status.idle":"2022-09-30T17:48:21.639403Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Merge horse trajectory and distance travelled and the track information.\nNow we know the distance each horse travelled and the race length. We can find if there are cases when the horse never travelled the whole length of the race. Marked these cases as failed.\n\nDistance ID is converted into categorical variable for better plotting that follows.","metadata":{}},{"cell_type":"code","source":"ttrt_sf_dist <- merge(tt_sf_dist, rt) %>% \n  mutate(dist_diff=as.numeric(distance) - track_distance_m, \n         failed=dist_diff<0, \n         distance_id=factor(distance_id))","metadata":{"execution":{"iopub.status.busy":"2022-09-30T17:48:21.643140Z","iopub.execute_input":"2022-09-30T17:48:21.644797Z","iopub.status.idle":"2022-09-30T17:48:21.870985Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Now I want to took at the suspected failed cases trajectories. Does it actully look like unfinished race? It seems like it does.","metadata":{}},{"cell_type":"code","source":"options(repr.plot.width = 15, repr.plot.height = 10)\nttrt_sf_dist %>% filter(failed, track_id==\"SAR\") %>% \n  ggplot() + geom_sf(aes(col=factor(race_number))) + facet_wrap(~race_date)","metadata":{"execution":{"iopub.status.busy":"2022-09-30T17:48:21.875039Z","iopub.execute_input":"2022-09-30T17:48:21.876797Z","iopub.status.idle":"2022-09-30T17:48:24.362632Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Find factors that may affect the chances of finishing the race","metadata":{}},{"cell_type":"markdown","source":"For each categorical variable that is describing the track and the race type find out if any of the factor levels are significant. \nUsing Fisher exact test as it is simple and is recommended for the case of 2x2 contingency tables. (There are way too many ways to evaluate \"significance\", I just want to use something simple and known that works for a start).","metadata":{}},{"cell_type":"code","source":"chi_sq_by_one <- function(cat) {\n  sapply(levels(cat), function(l) {\n    res = fisher.test(cat==l, ttrt_sf_dist$failed)\n    c(res$statistic, p_value=res$p.value)\n  })\n}\n\nlapply(ttrt_sf_dist[c(\"track_id\", \"race_type\",\"course_type\",\"track_condition\",\"distance_id\")],chi_sq_by_one)","metadata":{"execution":{"iopub.status.busy":"2022-09-30T17:50:22.000965Z","iopub.execute_input":"2022-09-30T17:50:22.002685Z","iopub.status.idle":"2022-09-30T17:50:22.449907Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Given significance test resukts let's look at the specific cases the test finds with low p-value. In which direction the balance is leaning?\n\nIt seems that all 3 tracks have about the same ratio of successful to failed races. We can ignore this factor.\n\nThere are too many track distance IDs with p-value that varies substantially even across close distances. Ignore that for now. It is possible that the track distances are specific for some race types and by itself are not significant. But that requires more detailed analysis.\n\nLook at what remains. Which features have effect on failure rate? Picking levels that give me the lowest p value in each track category. \n\n- `race_type`","metadata":{}},{"cell_type":"code","source":"table(SOC_race=ttrt_sf_dist$race_type==\"SOC\",failed=ttrt_sf_dist$failed)","metadata":{"execution":{"iopub.status.busy":"2022-09-30T17:55:45.836557Z","iopub.execute_input":"2022-09-30T17:55:45.838544Z","iopub.status.idle":"2022-09-30T17:55:45.862351Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"prop.table(table(SOC_race=ttrt_sf_dist$race_type==\"SOC\",failed=ttrt_sf_dist$failed),1)","metadata":{"execution":{"iopub.status.busy":"2022-09-30T17:55:49.131635Z","iopub.execute_input":"2022-09-30T17:55:49.133445Z","iopub.status.idle":"2022-09-30T17:55:49.160365Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"- `course_type`","metadata":{}},{"cell_type":"code","source":"table(M_course=ttrt_sf_dist$course_type==\"M\",failed=ttrt_sf_dist$failed)","metadata":{"execution":{"iopub.status.busy":"2022-09-30T17:56:06.861204Z","iopub.execute_input":"2022-09-30T17:56:06.862987Z","iopub.status.idle":"2022-09-30T17:56:06.888874Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"prop.table(table(M_course=ttrt_sf_dist$course_type==\"M\",failed=ttrt_sf_dist$failed),1)","metadata":{"execution":{"iopub.status.busy":"2022-09-30T17:56:07.401747Z","iopub.execute_input":"2022-09-30T17:56:07.403574Z","iopub.status.idle":"2022-09-30T17:56:07.429627Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"- `track_condition`","metadata":{}},{"cell_type":"code","source":"table(GD_track=ttrt_sf_dist$track_condition==\"GD \",failed=ttrt_sf_dist$failed)","metadata":{"execution":{"iopub.status.busy":"2022-09-30T17:56:40.950390Z","iopub.execute_input":"2022-09-30T17:56:40.952314Z","iopub.status.idle":"2022-09-30T17:56:40.975847Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"prop.table(table(GD_track=ttrt_sf_dist$track_condition==\"GD \",failed=ttrt_sf_dist$failed),1)","metadata":{"execution":{"iopub.status.busy":"2022-09-30T17:56:41.487884Z","iopub.execute_input":"2022-09-30T17:56:41.489822Z","iopub.status.idle":"2022-09-30T17:56:41.515350Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Some conclusions\n\n- SOC (Starter Optional Claimer) race type and M (Hurdle) course type seems to raise the risk of the horse faltering and not finishing the race. That seems strange that the type of external to the actual race condition actually affects the number of unfinished races. That needs further investigation. That effect may be spurious.\n- track type GD (Good) seems to be safer than others, with lower ratio of failure.\n\nOverall it seems that there is very little effect of the factors considered on the chances of the horse not finising the race ","metadata":{}},{"cell_type":"markdown","source":"## TODO\n- apply other technics to find faults, such as drop in speed, getting separated from the main group, possible others\n- do more carefull and less superficial statistical analysis of the failed cases\n- consider all other factors, such as the amount of betting, jockey weight, etc. If the Starter Optional Claimer race type has an effect on the outcome that may mean that psychological factors may be at play that affect the race results. Or maybe even cheating. ","metadata":{}},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]}]}