{"cells":[{"metadata":{},"cell_type":"markdown","source":"The \"Four Factors\" in basketball are designed to measure how good a team is at shooting, taking care of the ball, offensive rebounding, and getting to the foul line. Dean Oliver wrote about them in [Basketball on Paper](https://www.amazon.com/Basketball-Paper-Rules-Performance-Analysis/dp/1574886886), and [kenpom uses them extensively](https://kenpom.com/blog/four-factors/). This notebook will calculate and compile the four factor ratings for each team throughout the regular season and uses a difference between teams approach to generate a training file."},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"cell_type":"code","source":"# This Python 3 environment comes with many helpful analytics libraries installed\n# It is defined by the kaggle/python docker image: https://github.com/kaggle/docker-python\n# For example, here's several helpful packages to load in \n\nimport numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\nimport math\n\npd.set_option('display.float_format', lambda x: '%.3f' % x)\n# Input data files are available in the \"../input/\" directory.\n# For example, running this (by clicking run or pressing Shift+Enter) will list all files under the input directory\n\nimport os\nfor dirname, _, filenames in os.walk('/kaggle/input'):\n    for filename in filenames:\n        print(os.path.join(dirname, filename))\n\n# Any results you write to the current directory are saved as output.","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"First, let's randomize the order that the teams come into our training set and calculate the seeding differential."},{"metadata":{"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","trusted":true},"cell_type":"code","source":"#bring in data from 2003 on\ntourney_results = pd.read_csv('/kaggle/input/google-cloud-ncaa-march-madness-2020-division-1-mens-tournament/MDataFiles_Stage1/MNCAATourneyCompactResults.csv')\nsubmission = pd.read_csv('/kaggle/input/google-cloud-ncaa-march-madness-2020-division-1-mens-tournament/MSampleSubmissionStage1_2020.csv')\ntourney_results = tourney_results.loc[tourney_results['Season'] >= 2003]\n\ntraining_set = pd.DataFrame()\ntraining_set['Result'] = np.random.randint(0,2,len(tourney_results.index))\ntraining_set['Season'] = tourney_results['Season'].values\ntraining_set['Team1'] = training_set['Result'].values * tourney_results['WTeamID'].values + (1-training_set['Result'].values) * tourney_results['LTeamID'].values \ntraining_set['Team2'] = (1-training_set['Result'].values) * tourney_results['WTeamID'].values + training_set['Result'].values * tourney_results['LTeamID'].values \n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Calculate Delta Seeds\nseeds = pd.read_csv('/kaggle/input/google-cloud-ncaa-march-madness-2020-division-1-mens-tournament/MDataFiles_Stage1/MNCAATourneySeeds.csv')\nseeds['Seed'] =  pd.to_numeric(seeds['Seed'].str[1:3], downcast='integer',errors='coerce')\ntraining_set = training_set.merge(seeds, how='left', left_on=['Season','Team1'], right_on = ['Season','TeamID'])\ntraining_set = training_set.merge(seeds, how='left', left_on=['Season','Team2'], right_on = ['Season','TeamID'])\ntraining_set['deltaSeed'] = training_set['Seed_x'] - training_set['Seed_y']\ntraining_set.drop(columns = ['Seed_x','TeamID_x','Seed_y','TeamID_y'], inplace=True)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"To add another feature, I'm going to adjust Wins and Losses according to the NCAA's net methodology, which weights home games by 1.4 and away games by 0.6, and calculate an adjusted winning percentage for each team in the training set."},{"metadata":{"trusted":true},"cell_type":"code","source":"# Calculate weighted win pct based on location\n#home wins are weighted 0.6, away 1.4\nseason_results = pd.read_csv('/kaggle/input/google-cloud-ncaa-march-madness-2020-division-1-mens-tournament/MDataFiles_Stage1/MRegularSeasonDetailedResults.csv')\nseason_results['LLoc'] = season_results['WLoc']\nseason_results.loc[season_results['WLoc'] == 'H','LLoc'] = 'A'\nseason_results.loc[season_results['WLoc'] == 'A','LLoc'] = 'H'\nseason_results['WLocWeight'] = season_results['WLoc']\nseason_results.loc[season_results['WLoc'] == 'H','WLocWeight'] = 0.6\nseason_results.loc[season_results['WLoc'] == 'N','WLocWeight'] = 1\nseason_results.loc[season_results['WLoc'] == 'A','WLocWeight'] = 1.4\nseason_results['LLocWeight'] = season_results['LLoc']\nseason_results.loc[season_results['LLoc'] == 'H','LLocWeight'] = 1.4\nseason_results.loc[season_results['LLoc'] == 'N','LLocWeight'] = 1\nseason_results.loc[season_results['LLoc'] == 'A','LLocWeight'] = 0.6\n\nrecord = pd.DataFrame({'Adj. wins': season_results.groupby(['Season','WTeamID'])['WLocWeight'].sum(),\n                      'Adj. losses': season_results.groupby(['Season','LTeamID'])['LLocWeight'].sum(),\n                      'Wins': season_results.groupby(['Season','WTeamID']).size(),\n                      'Losses': season_results.groupby(['Season','LTeamID']).size()}).reset_index();\n\nrecord.rename(index=str,columns={'level_0':'Season',\n                                 'level_1':'TeamID'}, inplace=True)\nrecord.fillna(value=0 , inplace=True)\nrecord['games'] = record['Wins']+record['Losses']\nrecord = record.merge(seeds, how = 'left', left_on = ['Season','TeamID'], right_on = ['Season','TeamID'])","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Calculate [RPI.](https://www.nbastuffer.com/analytics101/relative-percentage-index-rpi/)\n\nRating Percentage Index (RPI) Formula = .25 \\* (Team’s Winning Percentage)+ .50 \\* (Opponents’  Average Winning Percentage) + 0.25 \\* (Opponents’ Opponents’  Average Winning Percentage)\n\nWe will use normal winning percentage, not adjusted."},{"metadata":{"trusted":true},"cell_type":"code","source":"season_results = season_results.merge(record[['Season','TeamID','Wins','games']], how = 'left', left_on = ['Season','WTeamID'], right_on=['Season','TeamID'])\nseason_results['WWins'] = season_results['Wins']\nseason_results['WGames'] = season_results['games']\nseason_results.drop(columns = ['TeamID','Wins','games'], inplace=True)\n\nseason_results = season_results.merge(record[['Season','TeamID','Wins','games']], how = 'left', left_on = ['Season','LTeamID'], right_on=['Season','TeamID'])\nseason_results['LWins'] = season_results['Wins']\nseason_results['LGames'] = season_results['games']\nseason_results.drop(columns = ['TeamID','Wins','games'], inplace=True)\nseason_results.fillna(value=0 , inplace=True)\n\nopprecord = pd.DataFrame({'WOppWins' : season_results.groupby(['Season','WTeamID'])['LWins'].sum(),\n                          'WOppGames' : season_results.groupby(['Season','WTeamID'])['LGames'].sum(),\n                          'LOppWins' : season_results.groupby(['Season','LTeamID'])['WWins'].sum(),\n                          'LOppGames' : season_results.groupby(['Season','LTeamID'])['WGames'].sum()}).reset_index();\n\nopprecord.rename(index=str,columns={'level_0':'Season',\n                                    'level_1':'TeamID'}, inplace=True)\nopprecord.fillna(value=0 , inplace=True)\nopprecord['OppWins'] = opprecord['WOppWins'] + opprecord['LOppWins']\nopprecord['OppGames'] = opprecord['WOppGames'] + opprecord['LOppGames']\nopprecord['OppWinPercent'] = (opprecord['OppWins']/opprecord['OppGames'])\n\n\nseason_results = season_results.merge(opprecord[['Season','TeamID','OppWins','OppGames']], how = 'left', left_on = ['Season','WTeamID'], right_on = ['Season','TeamID'])\nseason_results['WOppWins'] = season_results['OppWins']\nseason_results['WOppGames'] = season_results['OppGames']\nseason_results.drop(columns = ['TeamID','OppWins','OppGames'], inplace=True)\n\nseason_results = season_results.merge(opprecord[['Season','TeamID','OppWins','OppGames']], how = 'left', left_on = ['Season','LTeamID'], right_on = ['Season','TeamID'])\nseason_results['LOppWins'] = season_results['OppWins']\nseason_results['LOppGames'] = season_results['OppGames']\nseason_results.drop(columns = ['TeamID','OppWins','OppGames'], inplace=True)\n\nrecord = record.merge(opprecord[['Season','TeamID','OppWins','OppGames']],how='left',left_on=['Season','TeamID'],right_on=['Season','TeamID'])\n\noppopprecord = pd.DataFrame({'WOppOppWins' : season_results.groupby(['Season','WTeamID'])['LOppWins'].sum(),\n                             'WOppOppGames' : season_results.groupby(['Season','WTeamID'])['LOppGames'].sum(),\n                             'LOppOppWins' : season_results.groupby(['Season','LTeamID'])['WOppWins'].sum(),\n                             'LOppOppGames' : season_results.groupby(['Season','LTeamID'])['WOppGames'].sum()}).reset_index();\n\noppopprecord.rename(index=str,columns={'level_0':'Season',\n                                       'level_1':'TeamID'}, inplace=True)\noppopprecord.fillna(value=0 , inplace=True)\noppopprecord['OppOppWins'] = oppopprecord['WOppOppWins'] + oppopprecord['LOppOppWins']\noppopprecord['OppOppGames'] = oppopprecord['WOppOppGames'] + oppopprecord['LOppOppGames']\noppopprecord['OppOppWinPercent'] = (oppopprecord['OppOppWins']/oppopprecord['OppOppGames'])\n\n\nrecord = record.merge(oppopprecord[['Season','TeamID','OppOppWins','OppOppGames']],how='left',left_on=['Season','TeamID'],right_on=['Season','TeamID'])\n\nrecord['RPI'] = (.25*(record['Wins']/record['games'])) + (.50*(record['OppWins']/record['OppGames'])) + (.25*(record['OppOppWins']/record['OppOppGames']))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"training_set = training_set.merge(record[['Season','TeamID','RPI']], how='left', left_on = ['Season','Team1'], right_on = ['Season','TeamID'])\ntraining_set = training_set.merge(record[['Season','TeamID','RPI']], how='left', left_on = ['Season','Team2'], right_on = ['Season','TeamID'])\ntraining_set['deltaRPI'] = training_set['RPI_x'] - training_set['RPI_y']\ntraining_set.drop(columns=['TeamID_x','RPI_x','TeamID_y','RPI_y'],inplace=True)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Create O and D efficiency ratings for each team and game  \nORtg is roughly points scored per possession and DRtg is points allowed per possession  \nPossession formula comes from here: https://www.nbastuffer.com/analytics101/possession/  \neFG = effective fg %  \nTOP = Turnover %  \nOR% = offensive rebounding percentage  \nFTR = ability to get to the line  "},{"metadata":{"trusted":true},"cell_type":"code","source":"season_results['WPoss'] = 0.96*(season_results['WFGA'] + season_results['WTO']  - season_results['WOR'] + 0.475*season_results['WFTA'])\nseason_results['LOppPoss'] = season_results['WPoss']\n\nseason_results['WPace'] = season_results['WPoss']/(season_results['NumOT']*5 + 40)\nseason_results['LOppPace'] = season_results['WPace']\n\nseason_results['WORtg'] = season_results['WScore']/season_results['WPoss']\nseason_results['LDRtg'] = season_results['WORtg']\n\nseason_results['WOeFG'] = (season_results['WFGM'] + season_results['WFGM3']*0.5)/season_results['WFGA']\nseason_results['LOppOeFG'] = season_results['WOeFG']\n\nseason_results['WTOP'] = season_results['WTO']/season_results['WPoss']\nseason_results['LOppTOP'] = season_results['WTOP']\n\nseason_results['WOR%'] = season_results['WOR']/(season_results['WOR']+season_results['LDR'])\nseason_results['LOppOR%'] = season_results['WOR%']\n\nseason_results['WFTR'] = season_results['WFTM']/season_results['WFGA']\nseason_results['LOppFTR'] = season_results['WFTR']\n\n\n\n\nseason_results['LPoss'] = 0.96*(season_results['LFGA'] + season_results['LTO'] +  - season_results['LOR'] + 0.475*season_results['LFTA'])\nseason_results['WOppPoss'] = season_results['LPoss']\n\nseason_results['LPace'] = season_results['LPoss']/(season_results['NumOT']*5 + 40)\nseason_results['WOppPace'] = season_results['LPace']\n\nseason_results['LORtg'] = season_results['LScore']/season_results['LPoss']\nseason_results['WDRtg'] = season_results['LORtg']\n\nseason_results['LOeFG'] = (season_results['LFGM'] + season_results['LFGM3']*0.5)/season_results['LFGA']\nseason_results['WOppOeFG'] = season_results['LOeFG']\n\nseason_results['LTOP'] = season_results['LTO']/season_results['LPoss']\nseason_results['WOppTOP'] = season_results['LTOP']\n\nseason_results['LOR%'] = season_results['LOR']/(season_results['LOR']+season_results['WDR'])\nseason_results['WOppOR%'] = season_results['LOR%']\n\nseason_results['LFTR'] = season_results['LFTM']/season_results['LFGA']\nseason_results['WOppFTR'] = season_results['LFTR']\n\n\nseason_results['Poss'] = season_results['WPoss']+season_results['LPoss']","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Group Offensive and Defensive Efficiencies for each team and season"},{"metadata":{"trusted":true},"cell_type":"code","source":"ratings = pd.DataFrame({'WORtg' : season_results.groupby(['Season','WTeamID'])['WORtg'].mean(),\n                        'WDRtg' : season_results.groupby(['Season','WTeamID'])['WDRtg'].mean(),\n                        'LORtg' : season_results.groupby(['Season','LTeamID'])['LORtg'].mean(),\n                        'LDRtg' : season_results.groupby(['Season','LTeamID'])['LDRtg'].mean()}).reset_index();\n\nratings.rename(index=str,columns={'level_0':'Season',\n                                 'level_1':'TeamID'}, inplace=True)\n\nvalues = {'LORtg' : 0,\n         'LDRtg' : 0}\nratings = ratings.fillna(value = values)\nratings = ratings.merge(record, how='outer', left_on=['Season','TeamID'], right_on=['Season','TeamID'],copy=False)\n#create average ratings for the entire season since each team's games are split into wins and losses\n#AORtg = average offensive rating\n#ADRtg = average defensive rating\nratings['AORtg'] = (ratings['WORtg']*ratings['Wins'] + ratings['LORtg']*ratings['Losses'])/ratings['games']\nratings['ADRtg'] = (ratings['WDRtg']*ratings['Wins'] + ratings['LDRtg']*ratings['Losses'])/ratings['games']","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Create average ratings for entire NCAA by season to compare relative strength of each team.\n\nOStrength will give a multiplier > 1 if the team is an above average offensive team.\n\nDStrength will give a multiplier > 1 if the team is an above average defensive team.\n\nThese multipliers will be used to adjust each team's opponent's o rating and d rating for each game. The idea is to adjust for the opponent's level.\n\nScoring 70 points against a team that sucks at defense should be less valuable than scoring 70 on Texas Tech for instance."},{"metadata":{"trusted":true},"cell_type":"code","source":"season_ratings = pd.DataFrame({'NCAA AORtg' : ratings.groupby(['Season'])['AORtg'].mean(),\n                               'NCAA ADRtg' : ratings.groupby(['Season'])['ADRtg'].mean()}).reset_index();","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"ratings = ratings.merge(season_ratings[['Season','NCAA AORtg','NCAA ADRtg']],\n                        how = 'left',\n                        left_on = ['Season'],\n                        right_on = ['Season'])\n\nratings['OStrength'] = ratings['AORtg']/ratings['NCAA AORtg']\nratings['DStrength'] = ratings['NCAA ADRtg']/ratings['ADRtg']","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"#merge season strength ratings for losing team\nseason_results = season_results.merge(ratings[['Season','TeamID','OStrength','DStrength']], \n                                          how='left', \n                                          left_on = ['Season','LTeamID'], \n                                          right_on = ['Season','TeamID'],)\nseason_results.rename(index=str,columns={'OStrength':'LOStrength',\n                                         'DStrength':'LDStrength'}, inplace=True)\nseason_results.drop(columns=['TeamID'],inplace=True)\n\n#merge season strength ratings for winning team\nseason_results = season_results.merge(ratings[['Season','TeamID','OStrength','DStrength']], \n                                          how='left', \n                                          left_on = ['Season','WTeamID'], \n                                          right_on = ['Season','TeamID'],)\nseason_results.rename(index=str,columns={'OStrength':'WOStrength',\n                                         'DStrength':'WDStrength'}, inplace=True)\n\nseason_results.drop(columns=['TeamID'],inplace=True)\n\n\nseason_results.dropna(inplace=True)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"#Adjust each o rating and d rating by opponent's o and d strength\nseason_results['WAdjORtng'] = season_results['WORtg']*season_results['LDStrength']\nseason_results['WAdjDRtng'] = season_results['WDRtg']*season_results['LOStrength']\nseason_results['LAdjORtng'] = season_results['LORtg']*season_results['WDStrength']\nseason_results['LAdjDRtng'] = season_results['LORtg']*season_results['WOStrength']","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Now we will group stats for each teams' wins and losses to be summed up and divided by games played to get per game averages"},{"metadata":{"trusted":true},"cell_type":"code","source":"#create data from for winning team and losing team and merge into record dataframe\ndfW = season_results.groupby(['Season','WTeamID']).sum().reset_index()\ndfL = season_results.groupby(['Season','LTeamID']).sum().reset_index()\ncols = ['Pace','AdjORtng','AdjDRtng','OeFG','TOP','OR%','FTR','OppPace','OppOeFG','OppTOP','OppOR%','OppFTR']\n\nwcols = ['Season','WTeamID']\nlcols = ['Season','LTeamID']\nfor col in cols:\n    wname = 'W'+col\n    lname = 'L'+col\n    wcols.append(wname)\n    lcols.append(lname)\n    \ndf = record.merge(dfW[wcols], how='left', left_on = ['Season','TeamID'], right_on = ['Season','WTeamID'])\ndf = df.merge(dfL[lcols], how='left', left_on = ['Season','TeamID'], right_on = ['Season','LTeamID'])\ndf.fillna(0,inplace=True)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"for col in cols:\n    df[col] = (df['W'+col] + df['L'+col])/df['games']\n    \n\nwcols.remove('Season')\nlcols.remove('Season')\ndf.drop(columns = wcols+lcols, inplace=True)\n\nrecord = record.merge(df[['Season','TeamID']+cols],how='left',left_on=['Season','TeamID'],right_on=['Season','TeamID'])","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"training_set = training_set.merge(record[['Season','TeamID']+cols], how='left', left_on=['Season','Team1'], right_on = ['Season','TeamID'])\ntraining_set = training_set.merge(record[['Season','TeamID']+cols], how='left', left_on=['Season','Team2'], right_on = ['Season','TeamID'],suffixes = ['_1','_2'])\ntraining_set.drop(columns=['TeamID_1','TeamID_2'], inplace=True)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"for col in cols:\n    name = 'delta'+col\n    training_set[name] = training_set[col+'_1'] - training_set[col+'_2']\n    training_set.drop(columns=[col+'_1',col+'_2'],inplace=True)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Print training_set to be used to develop a model"},{"metadata":{"trusted":true},"cell_type":"code","source":"record.to_csv('record.csv',index=False)\ntraining_set.to_csv('training_set.csv', index=False)\npd.set_option('display.float_format', lambda x: '%.3f' % x)\ntraining_set.describe()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"By calculating each team's four factor stats and how effectively they limited their opponent's four factor stats, we should have a robust training set on which to develop a model."}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}