{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"**Investigation into the rating feature on the football probability prediction competition**\n\nThis is a derivative work based on these two excellent notebooks:\n\n[Investigation into rating feature (octosport)](https://www.kaggle.com/code/curiosityquotient/investigation-into-rating-feature-octosport) by @curiosityquotient\n\n[Ratings are Prediction of Goals Scored?](https://www.kaggle.com/code/szeyeung/ratings-are-prediction-of-goals-scored) by @szeyeung","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19"}},{"cell_type":"code","source":"# This Python 3 environment comes with many helpful analytics libraries installed\n# It is defined by the kaggle/python Docker image: https://github.com/kaggle/docker-python\n# For example, here's several helpful packages to load\n\nimport numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\nimport matplotlib.pyplot as plt\n\n# Input data files are available in the read-only \"../input/\" directory\n# For example, running this (by clicking run or pressing Shift+Enter) will list all files under the input directory\n\nimport os\nfor dirname, _, filenames in os.walk('/kaggle/input'):\n    for filename in filenames:\n        print(os.path.join(dirname, filename))\n\n# You can write up to 20GB to the current directory (/kaggle/working/) that gets preserved as output when you create a version using \"Save & Run All\" \n# You can also write temporary files to /kaggle/temp/, but they won't be saved outside of the current session\ntrain = pd.read_csv(\"/kaggle/input/football-match-probability-prediction/train.csv\")\ntest = pd.read_csv(\"/kaggle/input/football-match-probability-prediction/test.csv\")","metadata":{"execution":{"iopub.status.busy":"2022-04-11T17:27:55.338508Z","iopub.execute_input":"2022-04-11T17:27:55.338856Z","iopub.status.idle":"2022-04-11T17:28:04.479511Z","shell.execute_reply.started":"2022-04-11T17:27:55.338751Z","shell.execute_reply":"2022-04-11T17:28:04.478576Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"The basis of the code is @curisityquotient's model. However, I use @szeyeung's insights that a team's rating is a prediction of the goals they will score, and the opposition's rating is a prediction of the goals the opposition will score in a given match.\n\nThus I adapt @curiosityquotient's model by using both a\n\n**meanSelfRateHome(df, strt)**\n\nwhich measures the current home team's attacking strength (relative to the averaged defensive abilities of their opponents in the set of games considered), and also a\n\n**meanOppRateHome(df, strt)**\n\nwhich measures the current home team's defensive strength (relative to the averaged attacking abilities of their opponents in the set of games considered).\n\nSimilarly, I define the two quantities\n\n**meanSelfRateAway(df, strt)**\n\n**meanOppRateAway(df, strt)**\n\nfor the current away team.\n\nThe strength of each team is then measured by the difference between their 'Self' and 'Opp' ratings:\n\ndef meanRateHome(df, strt):\n    return meanSelfRateHome(df, strt) - meanOppRateHome(df, strt)\n\ndef meanRateAway(df, strt):\n    return meanSelfRateAway(df, strt) - meanOppRateAway(df, strt)","metadata":{}},{"cell_type":"code","source":"# function to calculate mean home rating. A start parameter is required as the initial investigation\n# uses the historical information only making in inappropriate to use the '1' suffixed data entries.\ndef meanSelfRateHome(df, strt):\n    HTSR = 'home_team_history_rating_'\n    HomeRateSelfHist = []\n    for i in range(strt, 11):\n        HomeRateSelfHist.append(df[HTSR+str(i)])\n    return np.nanmean(HomeRateSelfHist)\n\ndef meanOppRateHome(df, strt):\n    HTOR = 'home_team_history_opponent_rating_'\n    HomeRateOppHist = []\n    for i in range(strt, 11):\n        HomeRateOppHist.append(df[HTOR+str(i)])\n    return np.nanmean(HomeRateOppHist)\n\ndef meanSelfRateAway(df, strt):\n    ATSR = 'away_team_history_rating_'\n    AwayRateSelfHist = []\n    for i in range(strt, 11):        \n        AwayRateSelfHist.append(df[ATSR+str(i)])\n    return np.nanmean(AwayRateSelfHist)\n\ndef meanOppRateAway(df, strt):\n    ATOR = 'away_team_history_opponent_rating_'\n    AwayRateOppHist = []\n    for i in range(strt, 11):        \n        AwayRateOppHist.append(df[ATOR+str(i)])\n    return np.nanmean(AwayRateOppHist)\n\ndef meanRateHome(df, strt):\n    return meanSelfRateHome(df, strt) - meanOppRateHome(df, strt)\n\ndef meanRateAway(df, strt):\n    return meanSelfRateAway(df, strt) - meanOppRateAway(df, strt)\n            \nratingFrame = train.loc[:, ('id', 'target', 'home_team_history_goal_1', 'home_team_history_opponent_goal_1')].dropna()\nratingFrame['home_mean_rating'] = train.dropna().apply(meanRateHome, args=[1], axis=1)\nratingFrame['away_mean_rating'] = train.dropna().apply(meanRateAway, args=[1], axis=1)\nratingFrame['match_rating'] = ratingFrame['home_mean_rating']-ratingFrame['away_mean_rating']\nratingFrame['home_result'] = 0\n# comment out following line for specific match calculation (and uncomment following line)\nratingFrame['home_result'][ratingFrame['target']=='home'] = 1\n# comment out following line for prediction (and uncomment preceding line)\n#ratingFrame['home_result'][train['home_team_history_goal_1']>train['home_team_history_opponent_goal_1']] = 1\nratingFrame = ratingFrame.dropna()","metadata":{"execution":{"iopub.status.busy":"2022-04-11T17:28:12.0579Z","iopub.execute_input":"2022-04-11T17:28:12.058687Z","iopub.status.idle":"2022-04-11T17:28:25.291852Z","shell.execute_reply.started":"2022-04-11T17:28:12.058639Z","shell.execute_reply":"2022-04-11T17:28:25.291104Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"The next section of code is directly from @curisityquotient","metadata":{}},{"cell_type":"code","source":"bins = 15\nnoIts = 5\nhomeRatingResults = np.empty([2, bins*noIts])\nfor i in range(noIts):\n    dataSample = ratingFrame.sample(frac=0.33, random_state=np.random.randint(0, 100))\n    # split data into bins of equal size\n    dataSample['Hbins'] = pd.qcut(dataSample['match_rating'], bins, duplicates='drop')\n    binList = dataSample['Hbins'].unique() #create list of bins\n    winRate = [] #initialise before loop\n    rateBinMid = []\n    for j, binInt in enumerate(binList):\n        rateBinMid.append(binInt.mid) #  get mid point of interval\n        temp = dataSample.loc[dataSample['Hbins']==binInt] #slice data based on bin\n        winRate.append(temp['home_result'].sum()/temp['home_result'].count()) #calculate occurence of event\n        homeRatingResults[:, i*bins+j] = [rateBinMid[j], winRate[j]]\nfig, ax = plt.subplots()\nax.plot(homeRatingResults[0,:], homeRatingResults[1,:], 'o')","metadata":{"execution":{"iopub.status.busy":"2022-04-11T17:28:28.423716Z","iopub.execute_input":"2022-04-11T17:28:28.424208Z","iopub.status.idle":"2022-04-11T17:28:28.789422Z","shell.execute_reply.started":"2022-04-11T17:28:28.42417Z","shell.execute_reply":"2022-04-11T17:28:28.788385Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"from scipy.optimize import curve_fit\nfrom scipy.special import expit\nsigmoid = lambda rating, a, b, c, d : a*expit(b*rating-c)+d\npopt, pcov = curve_fit(sigmoid, homeRatingResults[0,:], homeRatingResults[1,:], [1, 1, 1, 0.1])\nfig, ax = plt.subplots()\nax.plot(homeRatingResults[0,:], homeRatingResults[1,:], 'o', np.arange(-20, 20), sigmoid(np.arange(-20, 20), popt[0], popt[1], popt[2], popt[3]))\nhomeFit = lambda rating : popt[0]*expit(popt[1]*rating-popt[2])+popt[3]","metadata":{"execution":{"iopub.status.busy":"2022-04-11T17:28:36.135822Z","iopub.execute_input":"2022-04-11T17:28:36.136742Z","iopub.status.idle":"2022-04-11T17:28:36.908121Z","shell.execute_reply.started":"2022-04-11T17:28:36.136699Z","shell.execute_reply":"2022-04-11T17:28:36.907107Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"ratingFrame['draw_result'] = 0\nratingFrame['away_result'] = 0\n# comment out following 2 lines for rating-match pairing (and uncomment following 2 lines)\nratingFrame['draw_result'][ratingFrame['target']=='draw'] = 1\nratingFrame['away_result'][ratingFrame['target']=='away'] = 1\n# comment out following 2 lines for prediction (and uncomment preceding 2 lines)\n#ratingFrame['draw_result'][train['home_team_history_goal_1']==train['home_team_history_opponent_goal_1']] = 1\n#ratingFrame['away_result'][train['home_team_history_goal_1']<train['home_team_history_opponent_goal_1']] = 1\nbins = 15\nnoIts = 5\ndrawRatingResults = np.empty([2, bins*noIts])\nawayRatingResults = np.empty([2, bins*noIts])\nfor i in range(noIts):\n    dataSample = ratingFrame.sample(frac=0.33, random_state=np.random.randint(0, 100))\n    # split data into bins of equal size\n    dataSample['Hbins'] = pd.qcut(dataSample['match_rating'], bins, duplicates='drop')\n    binList = dataSample['Hbins'].unique() #create list of bins\n    drawRate = [] #initialise before loop\n    awayRate = []\n    rateBinMid = []\n    for j, binInt in enumerate(binList):\n        rateBinMid.append(binInt.mid) #  get mid point of interval\n        temp = dataSample.loc[dataSample['Hbins']==binInt] #slice data based on bin\n        drawRate.append(temp['draw_result'].sum()/temp['draw_result'].count()) #calculate occurence of event\n        awayRate.append(temp['away_result'].sum()/temp['away_result'].count()) #calculate occurence of event\n        drawRatingResults[:, i*bins+j] = [rateBinMid[j], drawRate[j]]\n        awayRatingResults[:, i*bins+j] = [rateBinMid[j], awayRate[j]]\n\n        negSigmoid = lambda rating, a, b, c, d : -a*expit(b*rating-c)+d\ngauss = lambda rating, a, b, c, d : a*np.exp(-(rating-b)**2/(2*c**2))+d\n# remove draw extrema\n#drawPopt, pcov = curve_fit(gauss, drawRatingResults[0,:][(drawRatingResults[0,:]>-5)&(drawRatingResults[0,:]<5)], drawRatingResults[1,:][(drawRatingResults[0,:]>-5)&(drawRatingResults[0,:]<5)], [0.2, 0, 5, 0.1])\ndrawPopt, pcov = curve_fit(gauss, drawRatingResults[0,:], drawRatingResults[1,:], [0.2, 0, 5, 0.1])\ndrawFit = lambda rating : drawPopt[0]*np.exp(-(rating-drawPopt[1])**2/(2*drawPopt[2]**2))+drawPopt[3]\nfig, ax = plt.subplots()\nax.plot(drawRatingResults[0,:], drawRatingResults[1,:], 'o', np.arange(-20, 20), drawFit(np.arange(-20, 20)))\nawayPopt, pcov = curve_fit(negSigmoid, awayRatingResults[0,:],awayRatingResults[1,:], [1, 1, 1, 3])\nawayFit = lambda rating : -awayPopt[0]*expit(awayPopt[1]*rating-awayPopt[2])+awayPopt[3]\nfig, ax = plt.subplots()\nax.plot(awayRatingResults[0,:], awayRatingResults[1,:], 'o', np.arange(-20, 20), awayFit(np.arange(-20, 20)))","metadata":{"execution":{"iopub.status.busy":"2022-04-11T17:45:28.681295Z","iopub.execute_input":"2022-04-11T17:45:28.681585Z","iopub.status.idle":"2022-04-11T17:45:29.237187Z","shell.execute_reply.started":"2022-04-11T17:45:28.681549Z","shell.execute_reply":"2022-04-11T17:45:29.236221Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"For the predictions, I make a couple of changes to the code.\n\nFirstly, I clip the base home, away and draw probabilities to lie within the ranges given by:\n\n**prediction['home'] = np.clip(prediction['home'], 0.1, 0.95)**\n\n**prediction['away'] = np.clip(prediction['away'], 0.05, 0.9)**\n\n**prediction['draw'] = np.clip(prediction['draw'], 0.15, 0.3)**\n\nThese values are based on the plots by @curiosityquotient and the tendency of log loss to punish predictions that are both extreme and wrong.\n\nI also normalise the predictions such that the three final probabilities sum to one:\n\n**prediction['total'] = prediction['home'] + prediction['away'] + prediction['draw']**\n\n**prediction['home'] = prediction['home']/prediction['total']**\n\n**prediction['away'] = prediction['away']/prediction['total']**\n\n**prediction['draw'] = prediction['draw']/prediction['total']**","metadata":{}},{"cell_type":"code","source":"prediction = test.loc[:, ('id', 'home_team_history_rating_1', 'away_team_history_rating_1')]\nprediction['home_mean_rating'] = test.dropna().apply(meanRateHome, args=[1], axis=1)\nprediction['away_mean_rating'] = test.dropna().apply(meanRateAway, args=[1], axis=1)\nprediction['match_rating'] = prediction['home_mean_rating']-prediction['away_mean_rating']\n#prediction['match_rating'] = prediction['home_team_history_rating_1']-prediction['away_team_history_rating_1']\nprediction['match_rating'] = prediction['match_rating'].fillna(0)\nprediction['home'] = prediction['match_rating'].apply(homeFit)\nprediction['draw'] = prediction['match_rating'].apply(drawFit)\nprediction['away'] = prediction['match_rating'].apply(awayFit)\nprediction['home'] = np.clip(prediction['home'], 0.1, 0.95)\nprediction['away'] = np.clip(prediction['away'], 0.05, 0.9)\nprediction['draw'] = np.clip(prediction['draw'], 0.15, 0.3)\nprediction['total'] = prediction['home'] + prediction['away'] + prediction['draw']\nprediction['home'] = prediction['home']/prediction['total']\nprediction['away'] = prediction['away']/prediction['total']\nprediction['draw'] = prediction['draw']/prediction['total']","metadata":{"execution":{"iopub.status.busy":"2022-03-18T13:20:50.700909Z","iopub.execute_input":"2022-03-18T13:20:50.701612Z","iopub.status.idle":"2022-03-18T13:20:52.251579Z","shell.execute_reply.started":"2022-03-18T13:20:50.701551Z","shell.execute_reply":"2022-03-18T13:20:52.25081Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**Result**\n\nThis gives a score of 1.04849.\n","metadata":{}},{"cell_type":"code","source":"prediction[['id', 'home', 'draw', 'away']].to_csv('submission.csv', index=False)","metadata":{},"execution_count":null,"outputs":[]}]}