{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"* File descriptions\n * nyra_start_table.csv - horse/jockey race data\n * nyra_race_table.csv - racetrack race data\n * nyra_tracking_table.csv - tracking data\n * nyra_2019_complete.csv - combined table of three above files\n* Columns\n * nyra_start_table.csv\n    * track_id - 3 character id for the track the race took place at. AQU -Aqueduct, BEL - Belmont, SAR - Saratoga.\n    * race_date - date the race took place. YYYY-MM-DD.\n    * race_number - Number of the race. Passed as 3 characters but can be cast or converted to int for this data set.\n    * program_number - Program number of the horse in the race passed as 3 characters. Should remain 3 characters as it isn't limited to just numbers. Is essentially the unique identifier of the horse in the race.\n    * weight_carried - An integer of the weight carried by the horse in the race.\n    * jockey - Name of the jockey on the horse in the race. 50 character max.\n    * odds - Odds to win the race passed as an integer. Divide by 100 to derive the odds to 1. Example - 1280 would be 12.8-1.\n * nyra_race_table.csv\n    * track_id - 3 character id for the track the race took place at. AQU -Aqueduct, BEL - Belmont, SAR - Saratoga.\n    * race_date - date the race took place. YYYY-MM-DD.\n    * race_number - Number of the race. Passed as 3 characters but can be cast or converted to int for this data set.\n    * distance_id - Distance of the race in furlongs passed as an integer. Example - 600 would be 6 furlongs.\n    * course_type - The course the race was run over passed as one character. M - Hurdle, D - Dirt, O - Outer turf, I - Inner turf, T - turf.\n    * track_condition - The condition of the course the race was run on passed as three characters. YL - Yielding, FM - Firm, SY - Sloppy, GD - Good, FT - Fast, MY - Muddy, SF - Soft.\n    * run_up_distance - Distance in feet of the gate to the start of the race passed as an integer.\n    * race_type - The classification of the race passed as as five characters. STK - Stakes, WCL - Waiver Claiming, WMC - Waiver Maiden Claiming, SST - Starter Stakes, SHP - Starter Handicap, CLM - Claiming, STR - Starter Allowance, AOC - Allowance Optionl Claimer, SOC - Starter Optional Claimer, MCL - Maiden Claiming, ALW - Allowance, MSW - Maiden Special Weight.\n    * purse - Purse in US dollars of the race passed as an money with two decimal places.\n    * post_time - Time of day the race began passed as 5 character. Example - 01220 would be 12:20.\n\n* nyra_tracking_table.csv\n\n    * track_id - 3 character id for the track the race took place at. AQU -Aqueduct, BEL - Belmont, SAR - Saratoga.\n    * race_date - date the race took place. YYYY-MM-DD.\n    * race_number - Number of the race. Passed as 3 characters but can be cast or converted to int for this data set.\n    * program_number - Program number of the horse in the race passed as 3 characters. Should remain 3 characters as it isn't limited to just numbers. Is essentially the unique identifier of the horse in the race.\n    * trakus_index - The common collection of point of the lat / long of the horse in the race passed as an integer. From what we can tell, it's collected every 0.25 seconds.\n    * latitude - The latitude of the horse in the race passed as a float.\n    * longitude - The longitude of the horse in the race passed as a float.\n* nyra_2019_complete.csv - This file is the combined 3 files into one table. The keys to join them trakus with race - track_id, race_date, race_number. To join trakus with start - track_id, race_date, race_number, program_number.\n    * track_id - char(3)\n    * race_date - date\n    * race_number - char(3)\n    * program_number - char(3)\n    * trakus_index - int\n    * latitude - float\n    * longitude - float\n    * distance_id - int\n    * course_type - char(1)\n    * track_condition - char(3)\n    * run_up_distance - int\n    * race_type - char(5)\n    * post_time - char(5)\n    * weight_carried - int\n    * jockey - char(50)\n    * odds - int","metadata":{}},{"cell_type":"code","source":"import numpy as np\nimport pandas as pd\nimport matplotlib.pyplot as plt\nimport seaborn as sns\nfrom datetime import datetime as dt\nfrom scipy import stats\nfrom statsmodels.stats.multicomp import pairwise_tukeyhsd","metadata":{"execution":{"iopub.status.busy":"2022-08-30T01:53:07.296522Z","iopub.execute_input":"2022-08-30T01:53:07.296879Z","iopub.status.idle":"2022-08-30T01:53:08.239844Z","shell.execute_reply.started":"2022-08-30T01:53:07.296809Z","shell.execute_reply":"2022-08-30T01:53:08.238955Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"data_1 = pd.read_csv(\"../input/big-data-derby-2022/nyra_2019_complete.csv\", low_memory = False)\ndata_2 = pd.read_csv(\"../input/big-data-derby-2022/nyra_race_table.csv\", low_memory = False)\ndata_3 = pd.read_csv(\"../input/big-data-derby-2022/nyra_start_table.csv\", low_memory = False)\ndata_4 = pd.read_csv(\"../input/big-data-derby-2022/nyra_tracking_table.csv\", low_memory = False)","metadata":{"execution":{"iopub.status.busy":"2022-08-30T01:53:08.241816Z","iopub.execute_input":"2022-08-30T01:53:08.242152Z","iopub.status.idle":"2022-08-30T01:53:25.818720Z","shell.execute_reply.started":"2022-08-30T01:53:08.242122Z","shell.execute_reply":"2022-08-30T01:53:25.817918Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"pd.options.display.float_format = '{:.5f}'.format","metadata":{"execution":{"iopub.status.busy":"2022-08-30T01:53:25.819692Z","iopub.execute_input":"2022-08-30T01:53:25.819918Z","iopub.status.idle":"2022-08-30T01:53:25.824588Z","shell.execute_reply.started":"2022-08-30T01:53:25.819896Z","shell.execute_reply":"2022-08-30T01:53:25.823593Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"data_1.head()","metadata":{"execution":{"iopub.status.busy":"2022-08-30T01:53:25.826408Z","iopub.execute_input":"2022-08-30T01:53:25.826673Z","iopub.status.idle":"2022-08-30T01:53:25.857052Z","shell.execute_reply.started":"2022-08-30T01:53:25.826630Z","shell.execute_reply":"2022-08-30T01:53:25.856177Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print(data_1.info())","metadata":{"execution":{"iopub.status.busy":"2022-08-30T01:53:25.858150Z","iopub.execute_input":"2022-08-30T01:53:25.858394Z","iopub.status.idle":"2022-08-30T01:53:25.880437Z","shell.execute_reply.started":"2022-08-30T01:53:25.858372Z","shell.execute_reply":"2022-08-30T01:53:25.879287Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print(data_2.info())","metadata":{"execution":{"iopub.status.busy":"2022-08-30T01:53:25.881624Z","iopub.execute_input":"2022-08-30T01:53:25.881985Z","iopub.status.idle":"2022-08-30T01:53:25.897099Z","shell.execute_reply.started":"2022-08-30T01:53:25.881953Z","shell.execute_reply":"2022-08-30T01:53:25.896407Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print(data_3.info())","metadata":{"execution":{"iopub.status.busy":"2022-08-30T01:53:25.897915Z","iopub.execute_input":"2022-08-30T01:53:25.898173Z","iopub.status.idle":"2022-08-30T01:53:25.915537Z","shell.execute_reply.started":"2022-08-30T01:53:25.898152Z","shell.execute_reply":"2022-08-30T01:53:25.914910Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print(data_4.info())","metadata":{"execution":{"iopub.status.busy":"2022-08-30T01:53:25.916387Z","iopub.execute_input":"2022-08-30T01:53:25.916842Z","iopub.status.idle":"2022-08-30T01:53:25.938308Z","shell.execute_reply.started":"2022-08-30T01:53:25.916818Z","shell.execute_reply":"2022-08-30T01:53:25.937213Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"data_1.describe().style.background_gradient(cmap='Greens')","metadata":{"execution":{"iopub.status.busy":"2022-08-30T01:53:25.940183Z","iopub.execute_input":"2022-08-30T01:53:25.941538Z","iopub.status.idle":"2022-08-30T01:53:27.024820Z","shell.execute_reply.started":"2022-08-30T01:53:25.941500Z","shell.execute_reply":"2022-08-30T01:53:27.023623Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plt.figure(figsize=(12,8))\ncorr = data_1.corr()\nmask = np.zeros_like(corr)\nmask[np.triu_indices_from(mask)] = True\n\nsns.heatmap(data = corr, mask = mask, cmap = \"YlGnBu\", annot= True, square=True, vmax=.3)","metadata":{"execution":{"iopub.status.busy":"2022-08-30T01:53:27.028591Z","iopub.execute_input":"2022-08-30T01:53:27.029004Z","iopub.status.idle":"2022-08-30T01:53:28.689051Z","shell.execute_reply.started":"2022-08-30T01:53:27.028980Z","shell.execute_reply":"2022-08-30T01:53:28.688143Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"data_tracking = data_1.loc[data_1['track_id'] == 'AQU']\n\nplt.style.use(\"dark_background\")\nplt.figure(figsize = (16, 10))\nsns.scatterplot(data = data_tracking, x = 'longitude', y = 'latitude',\n                hue = 'race_number', palette= \"Paired\")\nplt.title(\"Longitude and Latitude by race_number of AQU\")","metadata":{"execution":{"iopub.status.busy":"2022-08-30T01:53:28.690267Z","iopub.execute_input":"2022-08-30T01:53:28.690854Z","iopub.status.idle":"2022-08-30T01:54:50.516864Z","shell.execute_reply.started":"2022-08-30T01:53:28.690829Z","shell.execute_reply":"2022-08-30T01:54:50.515622Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"data_tracking = data_1.loc[data_1['track_id'] == 'BEL']\n\nplt.style.use(\"dark_background\")\nplt.figure(figsize = (16, 10))\nsns.scatterplot(data = data_tracking, x = 'longitude', y = 'latitude',\n                hue = 'race_number', palette= \"Paired\")\nplt.title(\"Longitude and Latitude by race_number of BEL\")","metadata":{"execution":{"iopub.status.busy":"2022-08-30T01:54:50.517586Z","iopub.execute_input":"2022-08-30T01:54:50.517906Z","iopub.status.idle":"2022-08-30T01:55:58.951509Z","shell.execute_reply.started":"2022-08-30T01:54:50.517880Z","shell.execute_reply":"2022-08-30T01:55:58.950634Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"data_tracking = data_1.loc[data_1['track_id'] == 'SAR']\n\nplt.style.use(\"dark_background\")\nplt.figure(figsize = (16, 10))\nsns.scatterplot(data = data_tracking, x = 'longitude', y = 'latitude',\n                hue = 'race_number', palette= \"Paired\")\nplt.title(\"Longitude and Latitude by race_number of SAR\")","metadata":{"execution":{"iopub.status.busy":"2022-08-30T01:55:58.952553Z","iopub.execute_input":"2022-08-30T01:55:58.953047Z","iopub.status.idle":"2022-08-30T01:56:41.741126Z","shell.execute_reply.started":"2022-08-30T01:55:58.953023Z","shell.execute_reply":"2022-08-30T01:56:41.740127Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plt.figure(figsize =(8,6))\nsns.histplot(data = data_1.race_number, kde= True)","metadata":{"execution":{"iopub.status.busy":"2022-08-30T01:56:41.742512Z","iopub.execute_input":"2022-08-30T01:56:41.742757Z","iopub.status.idle":"2022-08-30T01:57:04.917822Z","shell.execute_reply.started":"2022-08-30T01:56:41.742735Z","shell.execute_reply":"2022-08-30T01:57:04.916446Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"* the race number from 1 to 9 occur more frequently","metadata":{}},{"cell_type":"code","source":"plt.figure(figsize =(8,6))\ndata_tracking = data_1.groupby(['race_type'])['race_type'].count().sort_values().plot(kind='bar',\n                                                                                      legend = True)","metadata":{"execution":{"iopub.status.busy":"2022-08-30T01:57:04.919507Z","iopub.execute_input":"2022-08-30T01:57:04.919797Z","iopub.status.idle":"2022-08-30T01:57:05.814346Z","shell.execute_reply.started":"2022-08-30T01:57:04.919773Z","shell.execute_reply":"2022-08-30T01:57:05.813685Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"* Most of race type is Claiming (CLM)","metadata":{}},{"cell_type":"code","source":"data_1.race_date = data_1.race_date.astype('datetime64[ns]')","metadata":{"execution":{"iopub.status.busy":"2022-08-30T01:57:05.815452Z","iopub.execute_input":"2022-08-30T01:57:05.816210Z","iopub.status.idle":"2022-08-30T01:57:06.808609Z","shell.execute_reply.started":"2022-08-30T01:57:05.816186Z","shell.execute_reply":"2022-08-30T01:57:06.807717Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"day_in_week = []\nfor i in data_1.race_date:\n    day_in_week.append(i.weekday())\ndata_1['day_in_week'] = day_in_week\ndata_1['day_in_week'] = data_1['day_in_week'].astype('string')\n\n\nplt.figure(figsize =(8,6))\nsns.countplot(x= data_1.day_in_week)\nplt.title(\"Number of race in each day in a week\")\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-08-30T01:57:06.809984Z","iopub.execute_input":"2022-08-30T01:57:06.810488Z","iopub.status.idle":"2022-08-30T01:57:25.683139Z","shell.execute_reply.started":"2022-08-30T01:57:06.810456Z","shell.execute_reply":"2022-08-30T01:57:25.682299Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"* Most of race occur in the weekend (monday is 0 and so on to sunday is 6)","metadata":{}},{"cell_type":"code","source":"month_in_year = []\nfor i in data_1.race_date:\n    month_in_year.append(i.month)\ndata_1['month_in_year'] = month_in_year\ndata_1['month_in_year'] = data_1['month_in_year'].astype('string')\n\n\nplt.figure(figsize =(8,6))\nsns.countplot(x= data_1.month_in_year)\nplt.title(\"Number of race in each month in year\")\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-08-30T01:57:25.684427Z","iopub.execute_input":"2022-08-30T01:57:25.684878Z","iopub.status.idle":"2022-08-30T01:57:45.633391Z","shell.execute_reply.started":"2022-08-30T01:57:25.684848Z","shell.execute_reply":"2022-08-30T01:57:45.632684Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"* the most race occur in august\n* It seems like the race is occur more frequently in summer (from may to august)","metadata":{}},{"cell_type":"code","source":"plt.figure(figsize =(8,6))\ndata_tracking = data_1.groupby(['track_condition'])['track_condition'].count().sort_values().plot(\n    kind = \"bar\")\nplt.title(\"The number of the condition of the course race\")\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-08-30T01:57:45.634570Z","iopub.execute_input":"2022-08-30T01:57:45.635022Z","iopub.status.idle":"2022-08-30T01:57:46.314152Z","shell.execute_reply.started":"2022-08-30T01:57:45.634994Z","shell.execute_reply":"2022-08-30T01:57:46.313436Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"* Most of track condition is Fast","metadata":{}},{"cell_type":"code","source":"plt.figure(figsize =(8,6))\ndata_tracking = data_1.groupby(['course_type'])['course_type'].count().sort_values().plot(\n    kind = \"bar\")\nplt.title(\"The number of the condition of the course type\")\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-08-30T01:57:46.315322Z","iopub.execute_input":"2022-08-30T01:57:46.315767Z","iopub.status.idle":"2022-08-30T01:57:46.957149Z","shell.execute_reply.started":"2022-08-30T01:57:46.315740Z","shell.execute_reply":"2022-08-30T01:57:46.956433Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"* Most of course type is Dirt","metadata":{}},{"cell_type":"markdown","source":"### Is there any differences between the amount of odds in each day of week ?","metadata":{}},{"cell_type":"code","source":"odd_week = pd.DataFrame(columns= ['monday', 'tuesday', 'wednesday', \n                               'thusday', 'friday', 'saturday', 'sunday'])\n\ncount = 0\nfor i in odd_week:\n    sample_ = data_1.odds.loc[data_1['day_in_week'] == str(count)]\n    sample_, lmbda = stats.yeojohnson(sample_)\n    sample_ = pd.Series(sample_)\n    sample_ = np.array([np.mean(sample_.sample(10).values) for j in range(500)])\n    odd_week[i] = sample_\n    count += 1\nodd_week","metadata":{"execution":{"iopub.status.busy":"2022-08-30T01:57:46.958289Z","iopub.execute_input":"2022-08-30T01:57:46.958716Z","iopub.status.idle":"2022-08-30T01:59:03.493013Z","shell.execute_reply.started":"2022-08-30T01:57:46.958687Z","shell.execute_reply":"2022-08-30T01:59:03.491276Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"alpha = .05\nfor i in odd_week:\n    shapiro, p = stats.shapiro(odd_week[i])\n    print(f'Critical value (Shapiro-based) ({i}) = {shapiro:4f}, p-value = {p:4f}')\n    if (p < alpha):\n        print('p-value < alpha reject H0 ==> Not Normal!!!')\n    else:\n        print('Normality Test passed!!!!')","metadata":{"execution":{"iopub.status.busy":"2022-08-30T01:59:03.494421Z","iopub.execute_input":"2022-08-30T01:59:03.494825Z","iopub.status.idle":"2022-08-30T01:59:03.502890Z","shell.execute_reply.started":"2022-08-30T01:59:03.494791Z","shell.execute_reply":"2022-08-30T01:59:03.501949Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"f, p = stats.f_oneway(odd_week.iloc[:,0], odd_week.iloc[:,1], odd_week.iloc[:,2], odd_week.iloc[:,3],\n                     odd_week.iloc[:,4], odd_week.iloc[:,5], odd_week.iloc[:,6])\n\n# conclusion base on p-value\nprint(f'p-value = {p:.2f}, alpha = {alpha:.2f}')\nif (p < alpha):\n    print('p-value < alpha reject H0')\n    print('There is significant difference between amount of odds of each day in week')\n    print('Turkey HSD is required')\nelse:\n    print('There is no difference between day')\n\n#--------------------------------------------------------------------------------------------------\n\nodd_week['index'] = odd_week.index\nodd_week_melt = pd.melt(odd_week.reset_index(), id_vars = ['index'], \n                  value_vars = ['monday', 'tuesday', 'wednesday', \n                               'thusday', 'friday', 'saturday', 'sunday'])\nm_comp = pairwise_tukeyhsd(endog = odd_week_melt['value'], groups = odd_week_melt['variable'], \n                                           alpha = 0.05)\nprint(m_comp)","metadata":{"execution":{"iopub.status.busy":"2022-08-30T01:59:03.504183Z","iopub.execute_input":"2022-08-30T01:59:03.504455Z","iopub.status.idle":"2022-08-30T01:59:04.067784Z","shell.execute_reply.started":"2022-08-30T01:59:03.504386Z","shell.execute_reply":"2022-08-30T01:59:04.066858Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plt.figure(figsize=(10,8))\nsns.barplot(data = odd_week_melt, x = 'variable', y= 'value')\nplt.title('Number of race each day in week')","metadata":{"execution":{"iopub.status.busy":"2022-08-30T01:59:04.069015Z","iopub.execute_input":"2022-08-30T01:59:04.069251Z","iopub.status.idle":"2022-08-30T01:59:04.417261Z","shell.execute_reply.started":"2022-08-30T01:59:04.069228Z","shell.execute_reply":"2022-08-30T01:59:04.415849Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"data_1.groupby(['day_in_week'])['odds'].describe()","metadata":{"execution":{"iopub.status.busy":"2022-08-30T01:59:04.418601Z","iopub.execute_input":"2022-08-30T01:59:04.418871Z","iopub.status.idle":"2022-08-30T01:59:05.354386Z","shell.execute_reply.started":"2022-08-30T01:59:04.418849Z","shell.execute_reply":"2022-08-30T01:59:05.353287Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"* There are no differences about the amount of odds between (sunday and thusday), (friday and monday)\n* Mostly the number of odds will high in the weekend (friday, saturday and sunday)\n* While in monday have the highest odds (19100)","metadata":{}},{"cell_type":"markdown","source":"### Is there any differences between the amount of odds each month of year ?","metadata":{}},{"cell_type":"code","source":"odd_month = pd.DataFrame(columns = ['january', 'february', 'march', 'april', 'may', 'june', 'july',\n                                       'august', 'september', 'october', 'november', 'december'])\ncount = 1\nfor i in odd_month:\n    sample_ = data_1.odds.loc[data_1['month_in_year'] == str(count)]\n    sample_, lmbda = stats.yeojohnson(sample_)\n    sample_ = pd.Series(sample_)\n    sample_ = np.array([np.mean(sample_.sample(10).values) for j in range(500)])\n    odd_month[i] = sample_\n    count += 1\nodd_month","metadata":{"execution":{"iopub.status.busy":"2022-08-30T01:59:05.355376Z","iopub.execute_input":"2022-08-30T01:59:05.355665Z","iopub.status.idle":"2022-08-30T02:00:07.817638Z","shell.execute_reply.started":"2022-08-30T01:59:05.355624Z","shell.execute_reply":"2022-08-30T02:00:07.816887Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"alpha = .05\nfor i in odd_month:\n    shapiro, p = stats.shapiro(odd_month[i])\n    print(f'Critical value (Shapiro-based) ({i}) = {shapiro:4f}, p-value = {p:4f}')\n    if (p < alpha):\n        print('p-value < alpha reject H0 ==> Not Normal!!!')\n    else:\n        print('Normality Test passed!!!!')","metadata":{"execution":{"iopub.status.busy":"2022-08-30T02:00:07.818496Z","iopub.execute_input":"2022-08-30T02:00:07.818727Z","iopub.status.idle":"2022-08-30T02:00:07.827266Z","shell.execute_reply.started":"2022-08-30T02:00:07.818703Z","shell.execute_reply":"2022-08-30T02:00:07.826005Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"f, p = stats.f_oneway(odd_month.iloc[:,0], odd_month.iloc[:,1], odd_month.iloc[:,2], odd_month.iloc[:,3],\n                     odd_month.iloc[:,4], odd_month.iloc[:,5], odd_month.iloc[:,6])\n\n# conclusion base on p-value\nprint(f'p-value = {p:.2f}, alpha = {alpha:.2f}')\nif (p < alpha):\n    print('p-value < alpha reject H0')\n    print('There is significant difference between amount of odds of each month in year')\n    print('Turkey HSD is required')\nelse:\n    print('There is no difference between months')\n\n#--------------------------------------------------------------------------------------------------\n\nodd_month['index'] = odd_month.index\nodd_month_melt = pd.melt(odd_month.reset_index(), id_vars = ['index'], \n                  value_vars = ['january', 'february', 'march', 'april', 'may', 'june', 'july',\n                                       'august', 'september', 'october', 'november', 'december'])\nm_comp = pairwise_tukeyhsd(endog = odd_month_melt['value'], groups = odd_month_melt['variable'], \n                                           alpha = 0.05)\nprint(m_comp)","metadata":{"execution":{"iopub.status.busy":"2022-08-30T02:00:07.832285Z","iopub.execute_input":"2022-08-30T02:00:07.832612Z","iopub.status.idle":"2022-08-30T02:00:08.989325Z","shell.execute_reply.started":"2022-08-30T02:00:07.832590Z","shell.execute_reply":"2022-08-30T02:00:08.988461Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plt.figure(figsize=(12,8))\nsns.barplot(data = odd_month_melt, x = 'variable', y= 'value')","metadata":{"execution":{"iopub.status.busy":"2022-08-30T02:00:08.990408Z","iopub.execute_input":"2022-08-30T02:00:08.990631Z","iopub.status.idle":"2022-08-30T02:00:09.478094Z","shell.execute_reply.started":"2022-08-30T02:00:08.990609Z","shell.execute_reply":"2022-08-30T02:00:09.477160Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"data_1.groupby(['month_in_year'])['odds'].describe()","metadata":{"execution":{"iopub.status.busy":"2022-08-30T02:00:09.479127Z","iopub.execute_input":"2022-08-30T02:00:09.479357Z","iopub.status.idle":"2022-08-30T02:00:10.401930Z","shell.execute_reply.started":"2022-08-30T02:00:09.479334Z","shell.execute_reply":"2022-08-30T02:00:10.400901Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"* (april and may), (august and december), (august and july), (december and july), (january and march)\n* (july and november), (november and september) show no difference about the amount of odds\n* In the Q1 (january have no difference with march about the amount of odds) (distance:3)\n* In the Q2 (april have no difference with may about the amount of odds) (distance:1)\n* In the Q3 and Q4\n * july have no difference with august, november and december (distance:1/5/6)\n * august have no difference with december (distance:5)\n * september have no dirrence with november (distance:3)\n#### => we can conclude that the amount of odds will stable in Q3 and Q4","metadata":{}},{"cell_type":"markdown","source":"### Is there anything difference between amount of weight for each type of condition ?","metadata":{}},{"cell_type":"code","source":"data_1.head()","metadata":{"execution":{"iopub.status.busy":"2022-08-30T02:00:10.402941Z","iopub.execute_input":"2022-08-30T02:00:10.403163Z","iopub.status.idle":"2022-08-30T02:00:10.419883Z","shell.execute_reply.started":"2022-08-30T02:00:10.403141Z","shell.execute_reply":"2022-08-30T02:00:10.418980Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"cond_weight = pd.DataFrame(columns = ['Yielding', 'Firm', 'Sloppy',\n                                    'Good', 'Fast', 'Muddy', 'Soft'])\ncond = ['YL ', 'FM ', 'SY ', 'GD ', 'FT ', 'MY ', 'SF ']\n\ncount = 0\nfor i in cond_weight:\n    sample_ = data_1.weight_carried.loc[data_1['track_condition'] == cond[count]]\n    sample_ = np.log(sample_)\n    #sample_ = pd.Series(sample_)\n    sample_ = np.array([np.mean(sample_.sample(30).values) for j in range(100)])\n    cond_weight[i] = sample_\n    count += 1\n\ndel cond\ncond_weight.head()","metadata":{"execution":{"iopub.status.busy":"2022-08-30T02:00:10.420930Z","iopub.execute_input":"2022-08-30T02:00:10.421233Z","iopub.status.idle":"2022-08-30T02:00:30.059746Z","shell.execute_reply.started":"2022-08-30T02:00:10.421201Z","shell.execute_reply":"2022-08-30T02:00:30.058873Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"alpha = .05\nfor i in cond_weight:\n    shapiro, p = stats.shapiro(cond_weight[i])\n    print(f'Critical value (Shapiro-based) ({i}) = {shapiro:4f}, p-value = {p:4f}')\n    if (p < alpha):\n        print('p-value < alpha reject H0 ==> Not Normal!!!')\n    else:\n        print('Normality Test passed!!!!')","metadata":{"execution":{"iopub.status.busy":"2022-08-30T02:00:30.060826Z","iopub.execute_input":"2022-08-30T02:00:30.061401Z","iopub.status.idle":"2022-08-30T02:00:30.068954Z","shell.execute_reply.started":"2022-08-30T02:00:30.061376Z","shell.execute_reply":"2022-08-30T02:00:30.067925Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"* Yielding and Firm didn't pass the normality test because of high outliers\n#### => need to process","metadata":{}},{"cell_type":"code","source":"index_ = cond_weight.loc[(cond_weight['Yielding'] >= 4.8) | (cond_weight['Good'] >= 4.8)\n                        | (cond_weight['Soft'] <= 4.774)]\ncond_weight.drop(index = index_.index, inplace= True)","metadata":{"execution":{"iopub.status.busy":"2022-08-30T02:00:30.070143Z","iopub.execute_input":"2022-08-30T02:00:30.070541Z","iopub.status.idle":"2022-08-30T02:00:30.085808Z","shell.execute_reply.started":"2022-08-30T02:00:30.070516Z","shell.execute_reply":"2022-08-30T02:00:30.084245Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"cond_weight","metadata":{"execution":{"iopub.status.busy":"2022-08-30T02:00:30.088414Z","iopub.execute_input":"2022-08-30T02:00:30.088868Z","iopub.status.idle":"2022-08-30T02:00:30.103238Z","shell.execute_reply.started":"2022-08-30T02:00:30.088829Z","shell.execute_reply":"2022-08-30T02:00:30.101734Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"alpha = .05\nfor i in cond_weight:\n    shapiro, p = stats.shapiro(cond_weight[i])\n    print(f'Critical value (Shapiro-based) ({i}) = {shapiro:4f}, p-value = {p:4f}')\n    if (p < alpha):\n        print('p-value < alpha reject H0 ==> Not Normal!!!')\n    else:\n        print('Normality Test passed!!!!')","metadata":{"execution":{"iopub.status.busy":"2022-08-30T02:00:30.104358Z","iopub.execute_input":"2022-08-30T02:00:30.106968Z","iopub.status.idle":"2022-08-30T02:00:30.116437Z","shell.execute_reply.started":"2022-08-30T02:00:30.106935Z","shell.execute_reply":"2022-08-30T02:00:30.115364Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"* all columns passed the normality test","metadata":{}},{"cell_type":"code","source":"f, p = stats.f_oneway(cond_weight.iloc[:,0], cond_weight.iloc[:,1], cond_weight.iloc[:,2], cond_weight.iloc[:,3],\n                     cond_weight.iloc[:,4], cond_weight.iloc[:,5], cond_weight.iloc[:,6])\n\n# conclusion base on p-value\nprint(f'p-value = {p:.2f}, alpha = {alpha:.2f}')\nif (p < alpha):\n    print('p-value < alpha reject H0')\n    print('There is significant difference between amount of odds of each month in year')\n    print('Turkey HSD is required')\nelse:\n    print('There is no difference between months')\n    \n#--------------------------------------------------------------------------------------------------\n\ncond_weight['index'] = cond_weight.index\ncond_weight_melt = pd.melt(cond_weight.reset_index(), id_vars = ['index'], \n                  value_vars = ['Yielding', 'Firm', 'Sloppy',\n                                    'Good', 'Fast', 'Muddy', 'Soft'])\nm_comp = pairwise_tukeyhsd(endog = cond_weight_melt['value'], groups = cond_weight_melt['variable'], \n                                           alpha = 0.05)\nprint(m_comp)","metadata":{"execution":{"iopub.status.busy":"2022-08-30T02:00:30.117918Z","iopub.execute_input":"2022-08-30T02:00:30.118187Z","iopub.status.idle":"2022-08-30T02:00:30.522468Z","shell.execute_reply.started":"2022-08-30T02:00:30.118162Z","shell.execute_reply":"2022-08-30T02:00:30.521887Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"data_1.groupby(['track_condition'])['weight_carried'].describe()","metadata":{"execution":{"iopub.status.busy":"2022-08-30T02:00:30.523529Z","iopub.execute_input":"2022-08-30T02:00:30.523991Z","iopub.status.idle":"2022-08-30T02:00:31.107034Z","shell.execute_reply.started":"2022-08-30T02:00:30.523963Z","shell.execute_reply":"2022-08-30T02:00:31.106082Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"* All track conditions have differences in weight carried except for: \n    * With these track condition, the weight carried wont be different:\n      * Fast: [Good Muddy Sloppy]\n      * Firm: [Yielding]\n      * Good:  [Muddy Sloppy Yielding]\n      * Muddy: [Sloppy]","metadata":{}},{"cell_type":"markdown","source":"### Is there any difference in weight carried with each type of course ?","metadata":{}},{"cell_type":"code","source":"course_weight = pd.DataFrame(columns = ['Hurdle', 'Dirt', 'Outer_turf',\n                                    'Inner turf', 'turf'])\ncourse = ['M', 'D', 'O', 'I', 'T']\n\ncount = 0\nfor i in course_weight:\n    sample_ = data_1.weight_carried.loc[data_1['course_type'] == course[count]]\n    #sample_ = np.log(sample_)\n    #sample_ = pd.Series(sample_)\n    sample_ = np.array([np.mean(sample_.sample(30).values) for j in range(100)])\n    course_weight[i] = sample_\n    count += 1\n\ndel course\ncourse_weight.head()","metadata":{"execution":{"iopub.status.busy":"2022-08-30T02:00:31.108263Z","iopub.execute_input":"2022-08-30T02:00:31.108515Z","iopub.status.idle":"2022-08-30T02:00:54.079161Z","shell.execute_reply.started":"2022-08-30T02:00:31.108491Z","shell.execute_reply":"2022-08-30T02:00:54.078201Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"alpha = .05\nfor i in course_weight:\n    shapiro, p = stats.shapiro(course_weight[i])\n    print(f'Critical value (Shapiro-based) ({i}) = {shapiro:4f}, p-value = {p:4f}')\n    if (p < alpha):\n        print('p-value < alpha reject H0 ==> Not Normal!!!')\n    else:\n        print('Normality Test passed!!!!')","metadata":{"execution":{"iopub.status.busy":"2022-08-30T02:00:54.080673Z","iopub.execute_input":"2022-08-30T02:00:54.081420Z","iopub.status.idle":"2022-08-30T02:00:54.087538Z","shell.execute_reply.started":"2022-08-30T02:00:54.081393Z","shell.execute_reply":"2022-08-30T02:00:54.086524Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"f, p = stats.f_oneway(course_weight.iloc[:,0], course_weight.iloc[:,1], course_weight.iloc[:,2], \n                      course_weight.iloc[:,3], course_weight.iloc[:,4])\n\n# conclusion base on p-value\nprint(f'p-value = {p:.2f}, alpha = {alpha:.2f}')\nif (p < alpha):\n    print('p-value < alpha reject H0')\n    print('There is significant difference between amount of odds of each month in year')\n    print('Turkey HSD is required')\nelse:\n    print('There is no difference between months')\n\n#--------------------------------------------------------------------------------------------------\n\ncourse_weight['index'] = course_weight.index\ncourse_weight_melt = pd.melt(course_weight.reset_index(), id_vars = ['index'], \n                  value_vars = ['Hurdle', 'Dirt', 'Outer_turf',\n                                    'Inner turf', 'turf'])\nm_comp = pairwise_tukeyhsd(endog = course_weight_melt['value'], groups = course_weight_melt['variable'], \n                                           alpha = 0.05)\nprint(m_comp)","metadata":{"execution":{"iopub.status.busy":"2022-08-30T02:00:54.088631Z","iopub.execute_input":"2022-08-30T02:00:54.088883Z","iopub.status.idle":"2022-08-30T02:00:54.389933Z","shell.execute_reply.started":"2022-08-30T02:00:54.088861Z","shell.execute_reply":"2022-08-30T02:00:54.388632Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"data_1.groupby(['course_type'])['weight_carried'].describe()","metadata":{"execution":{"iopub.status.busy":"2022-08-30T02:00:54.391951Z","iopub.execute_input":"2022-08-30T02:00:54.392180Z","iopub.status.idle":"2022-08-30T02:00:54.982009Z","shell.execute_reply.started":"2022-08-30T02:00:54.392158Z","shell.execute_reply":"2022-08-30T02:00:54.981301Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"* All the course type have different weight carried except for:\n    * With these course type, the weight carried will not be different:\n      * Dirt: [Outer_turf, turf]\n      * Inner turf: [turf]\n      * Outer_turf: [turf]\n* If the course is hurdle, the weight carried is significant larger than other type of course","metadata":{}}]}