{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"code","source":"# This Python 3 environment comes with many helpful analytics libraries installed\n# It is defined by the kaggle/python Docker image: https://github.com/kaggle/docker-python\n# For example, here's several helpful packages to load\n\nimport numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\nimport seaborn as sns\nimport matplotlib.pyplot as plt\nimport plotly.express as px\nimport plotly.graph_objs as go\n\nimport plotly\nplotly.offline.init_notebook_mode(connected=True)\n\n#Ignore warnings\nimport warnings\nwarnings.filterwarnings('ignore')\n\n# Input data files are available in the read-only \"../input/\" directory\n# For example, running this (by clicking run or pressing Shift+Enter) will list all files under the input directory\n\nimport os\nfor dirname, _, filenames in os.walk('/kaggle/input'):\n    for filename in filenames:\n        print(os.path.join(dirname, filename))\n\n# You can write up to 20GB to the current directory (/kaggle/working/) that gets preserved as output when you create a version using \"Save & Run All\" \n# You can also write temporary files to /kaggle/temp/, but they won't be saved outside of the current session","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","_kg_hide-input":true,"_kg_hide-output":true,"execution":{"iopub.status.busy":"2022-08-25T02:42:49.606101Z","iopub.execute_input":"2022-08-25T02:42:49.606792Z","iopub.status.idle":"2022-08-25T02:42:52.204770Z","shell.execute_reply.started":"2022-08-25T02:42:49.606648Z","shell.execute_reply":"2022-08-25T02:42:52.203760Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#Big Data Derby 2022\n\n\"The goal of this competition is to analyze horse racing tactics, drafting strategies, and path efficiency. You will develop a model using never-before-released coordinate data along with basic race information.\"\n\n\"Your work will help racing horse owners, trainers, and veterinarians better understand how equine performance and welfare fit together. With better data analysis, equine welfare could significantly improve.\"\n\nhttps://www.kaggle.com/competitions/big-data-derby-2022/overview/description","metadata":{}},{"cell_type":"markdown","source":"<iframe width=\"936\" height=\"530\" src=\"https://www.youtube.com/embed/r86BJkLZp9w\" title=\"(My Fair Lady) Come on, Dover!  Move your bloomin' arse!\" frameborder=\"0\" allow=\"accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture\" allowfullscreen></iframe>","metadata":{}},{"cell_type":"markdown","source":"#(My Fair Lady) Come on, Dover! Move your bloomin' arse!\nhttps://www.youtube.com/watch?v=r86BJkLZp9w\n\n\"\"C'mon Dover! Move yer bloomin' arse!\"\n\n\"And with that, the exquisitely dressed Audrey Hepburn -- playing common Cockney girl Eliza Doolittle -- smashes her genteel demeanor to smithereens, horrifying the lords and ladies gathered at Britain's most prestigious race meeting.\nSo scandalous is Hepburn's legendary outburst in the Oscar-winning 1964 film, My Fair Lady, one nearby aristocrat actually faints in shock.\"\n\n\"It may seem like an overblown reaction -- by today's standards, Hepburn isn't even swearing -- but the famous scene at Royal Ascot, which kicked off on Tuesday, points to the strict protocol still permeating the historic 300-year-old horse race.\"\n\nhttps://edition.cnn.com/2013/06/20/sport/royal-ascot-enclosure-my-fair-lady/index.html","metadata":{}},{"cell_type":"code","source":"df = pd.read_csv(\"/kaggle/input/big-data-derby-2022/nyra_2019_complete.csv\", delimiter=',', encoding='utf8')\npd.set_option('display.max_columns', None)\ndf.head().style.set_properties(**{'background-color':'black',\n                                     'color': '#03e8fc'})","metadata":{"execution":{"iopub.status.busy":"2022-08-25T02:42:56.044841Z","iopub.execute_input":"2022-08-25T02:42:56.045262Z","iopub.status.idle":"2022-08-25T02:43:12.030336Z","shell.execute_reply.started":"2022-08-25T02:42:56.045227Z","shell.execute_reply":"2022-08-25T02:43:12.029151Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df1 = pd.read_csv(\"/kaggle/input/big-data-derby-2022/nyra_tracking_table.csv\", delimiter=',', encoding='utf8')\npd.set_option('display.max_columns', None)\ndf1.head()","metadata":{"execution":{"iopub.status.busy":"2022-08-25T02:40:07.191406Z","iopub.execute_input":"2022-08-25T02:40:07.192128Z","iopub.status.idle":"2022-08-25T02:40:13.301124Z","shell.execute_reply.started":"2022-08-25T02:40:07.192089Z","shell.execute_reply":"2022-08-25T02:40:13.300045Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df2 = pd.read_csv(\"/kaggle/input/big-data-derby-2022/nyra_start_table.csv\", delimiter=',', encoding='utf8')\npd.set_option('display.max_columns', None)\ndf2.head(3)","metadata":{"execution":{"iopub.status.busy":"2022-08-25T02:40:17.597448Z","iopub.execute_input":"2022-08-25T02:40:17.597919Z","iopub.status.idle":"2022-08-25T02:40:17.634442Z","shell.execute_reply.started":"2022-08-25T02:40:17.597867Z","shell.execute_reply":"2022-08-25T02:40:17.633134Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df3 = pd.read_csv(\"/kaggle/input/big-data-derby-2022/nyra_race_table.csv\", delimiter=',', encoding='utf8')\npd.set_option('display.max_columns', None)\ndf3.head()","metadata":{"execution":{"iopub.status.busy":"2022-08-25T02:40:22.770141Z","iopub.execute_input":"2022-08-25T02:40:22.771107Z","iopub.status.idle":"2022-08-25T02:40:22.798710Z","shell.execute_reply.started":"2022-08-25T02:40:22.771062Z","shell.execute_reply":"2022-08-25T02:40:22.797543Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df[\"track_id\"].value_counts()","metadata":{"_kg_hide-output":true,"execution":{"iopub.status.busy":"2022-08-25T02:40:27.756766Z","iopub.execute_input":"2022-08-25T02:40:27.757181Z","iopub.status.idle":"2022-08-25T02:40:28.087687Z","shell.execute_reply.started":"2022-08-25T02:40:27.757148Z","shell.execute_reply":"2022-08-25T02:40:28.086496Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df[\"jockey\"].value_counts()","metadata":{"execution":{"iopub.status.busy":"2022-08-25T02:40:32.447092Z","iopub.execute_input":"2022-08-25T02:40:32.447632Z","iopub.status.idle":"2022-08-25T02:40:32.780097Z","shell.execute_reply.started":"2022-08-25T02:40:32.447587Z","shell.execute_reply":"2022-08-25T02:40:32.779097Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#Code by Juan Cruz Martinez https://livecodestream.dev/post/how-to-plot-your-data-on-maps-using-python-and-folium/\n#When I wrote on the tooltip: Click Here For More Info, I could only read in the viewer but not in the edit\n\nimport folium\n\n#We're in Aqueduct, are we Samsung?\n\nm = folium.Map(location=[40.672902, -73.827607], zoom_start=15)\n\ntooltip = \"Move your bloomin Arse Dover\"\nfolium.TileLayer('stamentoner').add_to(m)#That changes the map to black/white\n\nmarker = folium.CircleMarker(\n    location=[40.672902,-73.827607 ],\n    radius=50,\n    popup=\"<stong>Aqueduct Racetrack</stong>\",\n    tooltip=tooltip)\nmarker.add_to(m)\n\nm","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-08-25T02:43:22.493729Z","iopub.execute_input":"2022-08-25T02:43:22.494206Z","iopub.status.idle":"2022-08-25T02:43:22.753958Z","shell.execute_reply.started":"2022-08-25T02:43:22.494164Z","shell.execute_reply":"2022-08-25T02:43:22.752803Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"sar = df[(df['track_id']=='SAR')].reset_index(drop=True)\nsar.head()","metadata":{"execution":{"iopub.status.busy":"2022-08-25T02:43:33.338212Z","iopub.execute_input":"2022-08-25T02:43:33.339060Z","iopub.status.idle":"2022-08-25T02:43:33.957603Z","shell.execute_reply.started":"2022-08-25T02:43:33.339005Z","shell.execute_reply":"2022-08-25T02:43:33.956450Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#Are we in Saratoga, are we Android/Samsung?","metadata":{}},{"cell_type":"code","source":"#Code by Juan Cruz Martinez https://livecodestream.dev/post/how-to-plot-your-data-on-maps-using-python-and-folium/\n#When I wrote on the tooltip: Click Here For More Info, I could only read in the viewer but not in the edit\n\nimport folium\n\n#We're in Saratoga, are we Samsung?\n\nm = folium.Map(location=[43.071967868268999, -73.770947920393297], zoom_start=15)\n\ntooltip = \"Move your bloomin Arse Dover\"\nfolium.TileLayer('stamentoner').add_to(m)#That changes the map to black/white\n\nmarker = folium.CircleMarker(\n    location=[43.069895,-73.769578 ],\n    radius=50,\n    popup=\"<stong>Saratoga Race Course</stong>\",\n    tooltip=tooltip)\nmarker.add_to(m)\n\nm","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-08-25T02:43:40.824609Z","iopub.execute_input":"2022-08-25T02:43:40.825074Z","iopub.status.idle":"2022-08-25T02:43:40.845073Z","shell.execute_reply.started":"2022-08-25T02:43:40.825005Z","shell.execute_reply":"2022-08-25T02:43:40.843921Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#I will try ANAI/ML\n\nScript by Arsh Anwar https://www.kaggle.com/code/d4rklucif3r/introducing-anai-stroke-prediction","metadata":{}},{"cell_type":"code","source":"%pip install anai-opensource==0.1.2-alpha-5","metadata":{"_kg_hide-output":true,"execution":{"iopub.status.busy":"2022-08-25T02:43:53.401070Z","iopub.execute_input":"2022-08-25T02:43:53.401505Z","iopub.status.idle":"2022-08-25T02:44:48.391401Z","shell.execute_reply.started":"2022-08-25T02:43:53.401467Z","shell.execute_reply":"2022-08-25T02:44:48.389767Z"},"collapsed":true,"jupyter":{"outputs_hidden":true},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#Importing Libraries","metadata":{}},{"cell_type":"code","source":"import anai\nfrom anai.preprocessing import Preprocessor\n\n\nimport plotly.express as px","metadata":{"execution":{"iopub.status.busy":"2022-08-25T02:44:51.793411Z","iopub.execute_input":"2022-08-25T02:44:51.793864Z","iopub.status.idle":"2022-08-25T02:44:57.504911Z","shell.execute_reply.started":"2022-08-25T02:44:51.793827Z","shell.execute_reply":"2022-08-25T02:44:57.503852Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df4 = anai.load(df_filepath = '../input/big-data-derby-2022/nyra_start_table.csv')","metadata":{"execution":{"iopub.status.busy":"2022-08-25T02:45:01.204029Z","iopub.execute_input":"2022-08-25T02:45:01.204895Z","iopub.status.idle":"2022-08-25T02:45:05.576955Z","shell.execute_reply.started":"2022-08-25T02:45:01.204847Z","shell.execute_reply":"2022-08-25T02:45:05.573830Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df4.head(2)","metadata":{"execution":{"iopub.status.busy":"2022-08-25T02:45:07.651157Z","iopub.execute_input":"2022-08-25T02:45:07.651797Z","iopub.status.idle":"2022-08-25T02:45:07.814880Z","shell.execute_reply.started":"2022-08-25T02:45:07.651742Z","shell.execute_reply":"2022-08-25T02:45:07.813661Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"prep = Preprocessor(dataset = df4, target = 'odds')","metadata":{"execution":{"iopub.status.busy":"2022-08-25T02:45:13.219166Z","iopub.execute_input":"2022-08-25T02:45:13.219944Z","iopub.status.idle":"2022-08-25T02:45:13.230517Z","shell.execute_reply.started":"2022-08-25T02:45:13.219897Z","shell.execute_reply":"2022-08-25T02:45:13.229019Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#That takes a lot of time with the complete table. Then I decided to try with nyra_start_table \n\n#Code by Arsh Anwar https://www.kaggle.com/code/d4rklucif3r/introducing-anai-stroke-prediction\n\nsummary = prep.summary()\nsummary.head(10)","metadata":{"execution":{"iopub.status.busy":"2022-08-25T02:45:18.592791Z","iopub.execute_input":"2022-08-25T02:45:18.594465Z","iopub.status.idle":"2022-08-25T02:45:25.997671Z","shell.execute_reply.started":"2022-08-25T02:45:18.594404Z","shell.execute_reply":"2022-08-25T02:45:25.995862Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#Code by Arsh Anwar https://www.kaggle.com/code/d4rklucif3r/introducing-anai-stroke-prediction\n\ncolumn_summary = prep.column_summary()\ncolumn_summary.head()","metadata":{"execution":{"iopub.status.busy":"2022-08-25T02:45:36.801267Z","iopub.execute_input":"2022-08-25T02:45:36.802662Z","iopub.status.idle":"2022-08-25T02:46:02.020217Z","shell.execute_reply.started":"2022-08-25T02:45:36.802557Z","shell.execute_reply":"2022-08-25T02:46:02.019287Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#ANAI Visulizations","metadata":{}},{"cell_type":"code","source":"#Code by Arsh Anwar https://www.kaggle.com/code/d4rklucif3r/introducing-anai-stroke-prediction\n\nfig = px.histogram(df4._to_pandas(), y=\"weight_carried\", x=\"race_number\", color=\"track_id\",\n                   hover_data=df4.columns, template = 'plotly_dark', title = 'Horse Races')\nfig.show()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-08-25T02:46:10.239029Z","iopub.execute_input":"2022-08-25T02:46:10.239888Z","iopub.status.idle":"2022-08-25T02:46:11.300690Z","shell.execute_reply.started":"2022-08-25T02:46:10.239844Z","shell.execute_reply":"2022-08-25T02:46:11.297160Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#Code by Arsh Anwar https://www.kaggle.com/code/d4rklucif3r/introducing-anai-stroke-prediction\n\nfig = px.scatter(df4._to_pandas(), y=\"weight_carried\", x=\"race_number\", color=\"track_id\", size = 'odds',\n                   hover_data=df4.columns, template = 'plotly_dark', title = 'Horse Races')\nfig.show()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-08-25T02:46:17.483601Z","iopub.execute_input":"2022-08-25T02:46:17.484095Z","iopub.status.idle":"2022-08-25T02:46:18.492545Z","shell.execute_reply.started":"2022-08-25T02:46:17.484034Z","shell.execute_reply":"2022-08-25T02:46:18.490979Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#Code by Arsh Anwar https://www.kaggle.com/code/d4rklucif3r/introducing-anai-stroke-prediction\n\nfig = px.scatter(df4._to_pandas() ,  x = 'race_number', y= 'odds' , template = 'plotly_dark', title = 'Horse Races')\nfig.show()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-08-25T02:46:32.733029Z","iopub.execute_input":"2022-08-25T02:46:32.734342Z","iopub.status.idle":"2022-08-25T02:46:32.869956Z","shell.execute_reply.started":"2022-08-25T02:46:32.734281Z","shell.execute_reply":"2022-08-25T02:46:32.868946Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#Parallel Plot","metadata":{}},{"cell_type":"code","source":"#Code by Arsh Anwar https://www.kaggle.com/code/d4rklucif3r/introducing-anai-stroke-prediction\n\nfig = px.parallel_coordinates(df4.drop(['weight_carried', 'track_id'], axis=1)._to_pandas(), color = 'race_number',\n                              template = 'plotly_dark')\nfig.show()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-08-25T02:46:39.679426Z","iopub.execute_input":"2022-08-25T02:46:39.679899Z","iopub.status.idle":"2022-08-25T02:46:39.953395Z","shell.execute_reply.started":"2022-08-25T02:46:39.679860Z","shell.execute_reply":"2022-08-25T02:46:39.951903Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#Normali-fy","metadata":{}},{"cell_type":"code","source":"#Code by Arsh Anwar https://www.kaggle.com/code/d4rklucif3r/introducing-anai-stroke-prediction\n\ndf4 = prep.skewcorrect()","metadata":{"execution":{"iopub.status.busy":"2022-08-25T02:46:48.028926Z","iopub.execute_input":"2022-08-25T02:46:48.029420Z","iopub.status.idle":"2022-08-25T02:47:01.261269Z","shell.execute_reply.started":"2022-08-25T02:46:48.029377Z","shell.execute_reply":"2022-08-25T02:47:01.260101Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#Save for next time with ANAI since we don't have missing values\n\n#df1 = prep.impute(method = 'mean')","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#df1.isna().sum()","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#Prepare Data","metadata":{}},{"cell_type":"code","source":"features, labels = prep.encode(split = True)","metadata":{"execution":{"iopub.status.busy":"2022-08-25T02:47:09.597582Z","iopub.execute_input":"2022-08-25T02:47:09.598090Z","iopub.status.idle":"2022-08-25T02:47:10.036157Z","shell.execute_reply.started":"2022-08-25T02:47:09.598053Z","shell.execute_reply":"2022-08-25T02:47:10.034788Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"features.head(4)","metadata":{"execution":{"iopub.status.busy":"2022-08-25T02:47:15.166336Z","iopub.execute_input":"2022-08-25T02:47:15.166883Z","iopub.status.idle":"2022-08-25T02:47:15.256538Z","shell.execute_reply.started":"2022-08-25T02:47:15.166825Z","shell.execute_reply":"2022-08-25T02:47:15.255324Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"X_train, X_val, y_train, y_val, scaler = prep.prepare(features, labels, test_size = 0.2, random_state = 42, smote = False, k_neighbors = 3)","metadata":{"execution":{"iopub.status.busy":"2022-08-25T02:47:21.393194Z","iopub.execute_input":"2022-08-25T02:47:21.393707Z","iopub.status.idle":"2022-08-25T02:47:23.082104Z","shell.execute_reply.started":"2022-08-25T02:47:21.393665Z","shell.execute_reply":"2022-08-25T02:47:23.078719Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"X_train.shape, X_val.shape, y_val.shape, y_train.shape","metadata":{"execution":{"iopub.status.busy":"2022-08-25T02:47:26.519114Z","iopub.execute_input":"2022-08-25T02:47:26.519956Z","iopub.status.idle":"2022-08-25T02:47:26.527854Z","shell.execute_reply.started":"2022-08-25T02:47:26.519908Z","shell.execute_reply":"2022-08-25T02:47:26.526395Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#ANAI's AutoML Pipeline","metadata":{}},{"cell_type":"code","source":"#Code by Arsh Anwar https://www.kaggle.com/code/d4rklucif3r/introducing-anai-stroke-prediction\n\nai = anai.run(filepath = '../input/big-data-derby-2022/nyra_start_table.csv', target = 'odds', predictor = ['ada', 'cat', 'xgb', 'knn', 'lgbm'])","metadata":{"execution":{"iopub.status.busy":"2022-08-25T02:47:44.395336Z","iopub.execute_input":"2022-08-25T02:47:44.395822Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#Go to Google Cloud or Dismiss","metadata":{}},{"cell_type":"markdown","source":"![](http://38.media.tumblr.com/ba964e0cce26bdff6cdda9c968e8d0c9/tumblr_inline_mz5tevI6NL1r4j8j1.gif)filmicdisposition.blogspot.com","metadata":{}},{"cell_type":"markdown","source":"#Acknowledgements: \n\nhttps://github.com/Revca-ANAI/ANAI\n\nhttps://anai.readthedocs.io/en/main/\n\nArsh Anwar https://www.kaggle.com/code/d4rklucif3r/introducing-anai-stroke-prediction","metadata":{}},{"cell_type":"markdown","source":"#Equine Injury Database\n\n\"Between 700 and 800 racehorses are injured and die every year, with a national average of about two breakdowns for every 1,000 starts.6 According to The Jockey Club’s Equine Injury Database, nearly 10 horses died every week at American racetracks in 2018.\"\n\nhttps://www.peta.org/","metadata":{}},{"cell_type":"markdown","source":"![](https://images.slideplayer.com/32/10023006/slides/slide_5.jpg)slideplayer.com","metadata":{}},{"cell_type":"markdown","source":"#Racing horses that overcame the odds to win the sport’s biggest race, the Kentucky Derby. \n\nDonerail (1913) Odds: 91-1\n\nRich Strike (2022) Odds: 80-1\n\nCountry House (2019) Odds: 65-1\n\nMine That Bird (2009) Odds: 50-1\n\nGiacomo (2005) Odds: 50-1\n\nCharismatic (1999) Odds: 31-1\n\nThunder Gulch (1995) Odds: 25-1\n\nAnimal Kingdom (2011) Odds: 21-1\n\nWar Emblem (2002) Odds: 21-1\n\nLil E. Tee (1992) Odds: 17-1\n\nhttps://www.nbcsports.com/boston/what-biggest-upset-kentucky-derby-history-heres-where-rich-strike-ranks\n","metadata":{}}]}