{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# FootBall\n\n![football](https://images.pexels.com/photos/47730/the-ball-stadion-football-the-pitch-47730.jpeg?auto=compress&cs=tinysrgb&h=750&w=1260)\n\nFootball, also called association football or soccer, is a game involving two teams of 11 players who try to maneuver the ball into the other team's goal without using their hands or arms. The team that scores more goals wins. Football is the world's most popular ball game in numbers of participants and spectators.","metadata":{"execution":{"iopub.status.busy":"2022-03-18T10:27:15.046942Z","iopub.execute_input":"2022-03-18T10:27:15.047266Z","iopub.status.idle":"2022-03-18T10:27:15.053422Z","shell.execute_reply.started":"2022-03-18T10:27:15.047231Z","shell.execute_reply":"2022-03-18T10:27:15.052661Z"}}},{"cell_type":"code","source":"# importing required libraries \n\nimport pandas as pd\nimport seaborn as sns\nimport matplotlib.pyplot as plt \nimport numpy as np\nfrom sklearn.model_selection import train_test_split\nimport warnings\nwarnings.simplefilter(\"ignore\")\npd.options.display.max_columns=200\npd.options.display.max_rows=200","metadata":{"execution":{"iopub.status.busy":"2022-03-19T06:03:06.716137Z","iopub.execute_input":"2022-03-19T06:03:06.716647Z","iopub.status.idle":"2022-03-19T06:03:07.958781Z","shell.execute_reply.started":"2022-03-19T06:03:06.716543Z","shell.execute_reply":"2022-03-19T06:03:07.957899Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Importing data\nImporting the given data train and test as df_train and df_test. Let´s make some assumptions about the relationships we might expect in our data. I have set max_columns and max_rows to 200 during importing the libraries so we can see the whole data with no skipping of columns/rows.","metadata":{}},{"cell_type":"code","source":"df_train = pd.read_csv(\"../input/football-match-probability-prediction/train.csv\")\ndf_train.head()","metadata":{"id":"6ce2773e","outputId":"a5d2f6ef-a9e6-424d-d52a-66b934ae511a","execution":{"iopub.status.busy":"2022-03-19T06:03:07.960771Z","iopub.execute_input":"2022-03-19T06:03:07.961229Z","iopub.status.idle":"2022-03-19T06:03:14.549982Z","shell.execute_reply.started":"2022-03-19T06:03:07.961185Z","shell.execute_reply":"2022-03-19T06:03:14.548936Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_test = pd.read_csv(\"../input/football-match-probability-prediction/test.csv\")\ndf_test.head()","metadata":{"id":"72db0a14","outputId":"4665f102-c784-4fa4-f3ea-aadfff30845b","execution":{"iopub.status.busy":"2022-03-19T06:03:14.551627Z","iopub.execute_input":"2022-03-19T06:03:14.552094Z","iopub.status.idle":"2022-03-19T06:03:18.754303Z","shell.execute_reply.started":"2022-03-19T06:03:14.552059Z","shell.execute_reply":"2022-03-19T06:03:18.753474Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Null value checking \n* It's very important to check the null values while starting the process, which is also called as pre-processing.\n* Using below filter can see features that are having null values","metadata":{}},{"cell_type":"code","source":"null_val = df_train.isna().sum()/len(df_train)*100\nnull_val[null_val>0]","metadata":{"id":"56f58092","outputId":"42dfb611-83be-42dc-cb65-0c9a38b16d97","execution":{"iopub.status.busy":"2022-03-19T06:03:18.756821Z","iopub.execute_input":"2022-03-19T06:03:18.757117Z","iopub.status.idle":"2022-03-19T06:03:19.106364Z","shell.execute_reply.started":"2022-03-19T06:03:18.757081Z","shell.execute_reply":"2022-03-19T06:03:19.105452Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"max(null_val)","metadata":{"id":"81455511","outputId":"a8aee470-f2a7-4632-a510-c908e9689b8c","execution":{"iopub.status.busy":"2022-03-19T06:03:19.107657Z","iopub.execute_input":"2022-03-19T06:03:19.107882Z","iopub.status.idle":"2022-03-19T06:03:19.114117Z","shell.execute_reply.started":"2022-03-19T06:03:19.107856Z","shell.execute_reply":"2022-03-19T06:03:19.113271Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Future Engineering\n* Since the columns are less than 30%, so we no need to drop it. instead we can fill using mean or mode. \n* Seperating  categorical and numerical columns for understading for better understanding the data.","metadata":{"id":"0601df29"}},{"cell_type":"code","source":"# Seperating categorical and numerical columns for train data\n\ncat = []\nnum = []\nfor i in df_train.columns:\n    if df_train[i].dtypes == 'O':\n        cat.append(i)\n    else:\n        num.append(i)","metadata":{"id":"317bd86e","execution":{"iopub.status.busy":"2022-03-19T06:03:19.115533Z","iopub.execute_input":"2022-03-19T06:03:19.116135Z","iopub.status.idle":"2022-03-19T06:03:19.136649Z","shell.execute_reply.started":"2022-03-19T06:03:19.116100Z","shell.execute_reply":"2022-03-19T06:03:19.135660Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# categorical columns\n\ndf_train[cat].head()","metadata":{"id":"4c1d4b1c","outputId":"2972b531-a1d0-4701-fd66-c156a6e3c6a6","execution":{"iopub.status.busy":"2022-03-19T06:03:19.137841Z","iopub.execute_input":"2022-03-19T06:03:19.138073Z","iopub.status.idle":"2022-03-19T06:03:19.195079Z","shell.execute_reply.started":"2022-03-19T06:03:19.138047Z","shell.execute_reply":"2022-03-19T06:03:19.194185Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# numerical columns\n\ndf_train[num].head()","metadata":{"id":"cb0ec6be","outputId":"9508537d-929d-4ac6-da9e-27acb48da05b","execution":{"iopub.status.busy":"2022-03-19T06:03:19.196664Z","iopub.execute_input":"2022-03-19T06:03:19.197330Z","iopub.status.idle":"2022-03-19T06:03:19.428711Z","shell.execute_reply.started":"2022-03-19T06:03:19.197285Z","shell.execute_reply":"2022-03-19T06:03:19.427869Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# checking of null values for test data\n\nnull_val = df_test.isna().sum()/len(df_test)*100\nnull_val[null_val>0]","metadata":{"execution":{"iopub.status.busy":"2022-03-19T06:03:19.429814Z","iopub.execute_input":"2022-03-19T06:03:19.430046Z","iopub.status.idle":"2022-03-19T06:03:19.649195Z","shell.execute_reply.started":"2022-03-19T06:03:19.430016Z","shell.execute_reply":"2022-03-19T06:03:19.648369Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Seperating categorical and numerical columns for test data\n\ncat_test = []\nnum_test = []\nfor i in df_test.columns:\n    if df_test[i].dtypes == 'O':\n        cat_test.append(i)\n    else:\n        num_test.append(i)","metadata":{"id":"1bbbf85d","execution":{"iopub.status.busy":"2022-03-19T06:03:19.653001Z","iopub.execute_input":"2022-03-19T06:03:19.653297Z","iopub.status.idle":"2022-03-19T06:03:19.670070Z","shell.execute_reply.started":"2022-03-19T06:03:19.653264Z","shell.execute_reply":"2022-03-19T06:03:19.669372Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# categorical columns\n\ndf_test[cat_test].head()","metadata":{"id":"1120ee1a","outputId":"dae463bd-a704-45df-8aae-7371226a4244","execution":{"iopub.status.busy":"2022-03-19T06:03:19.671267Z","iopub.execute_input":"2022-03-19T06:03:19.671970Z","iopub.status.idle":"2022-03-19T06:03:19.717586Z","shell.execute_reply.started":"2022-03-19T06:03:19.671926Z","shell.execute_reply":"2022-03-19T06:03:19.716663Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# numerical columns\n\ndf_test[num_test].head()","metadata":{"id":"07587f24","outputId":"f444de20-682f-465b-d1ef-e92a82994a78","execution":{"iopub.status.busy":"2022-03-19T06:03:19.719141Z","iopub.execute_input":"2022-03-19T06:03:19.720198Z","iopub.status.idle":"2022-03-19T06:03:19.942071Z","shell.execute_reply.started":"2022-03-19T06:03:19.719687Z","shell.execute_reply":"2022-03-19T06:03:19.941260Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"cat_test.append('is_cup')\ncat_test","metadata":{"id":"8eb7377f","outputId":"b79f5ee7-7295-4364-9b58-1b4aa765316e","execution":{"iopub.status.busy":"2022-03-19T06:03:19.943288Z","iopub.execute_input":"2022-03-19T06:03:19.943530Z","iopub.status.idle":"2022-03-19T06:03:19.949985Z","shell.execute_reply.started":"2022-03-19T06:03:19.943502Z","shell.execute_reply":"2022-03-19T06:03:19.949114Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Since the train and test data contrain null values: \n*     categorical columns are filled by mode.\n*     numerical columns are filled by mean ","metadata":{"execution":{"iopub.status.busy":"2022-03-18T09:42:06.257572Z","iopub.execute_input":"2022-03-18T09:42:06.258561Z","iopub.status.idle":"2022-03-18T09:42:06.265198Z","shell.execute_reply.started":"2022-03-18T09:42:06.258521Z","shell.execute_reply":"2022-03-18T09:42:06.263941Z"}}},{"cell_type":"code","source":"# filling null in train data\n\nfor i in df_train.columns:\n    if i in cat:\n        df_train[i] = df_train[i].fillna(df_train[i].mode())\n    else:\n        df_train[i] = df_train[i].fillna(df_train[i].mean())","metadata":{"id":"25899949","execution":{"iopub.status.busy":"2022-03-19T06:03:19.951135Z","iopub.execute_input":"2022-03-19T06:03:19.951370Z","iopub.status.idle":"2022-03-19T06:03:21.274809Z","shell.execute_reply.started":"2022-03-19T06:03:19.951335Z","shell.execute_reply":"2022-03-19T06:03:21.274084Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# filling null in test data\n\nfor i in df_test.columns:\n    if i in cat:\n        df_test[i] = df_test[i].fillna(df_test[i].mode())\n    else:\n        df_test[i] = df_test[i].fillna(df_test[i].mean())","metadata":{"id":"3bda8dcd","execution":{"iopub.status.busy":"2022-03-19T06:03:21.276012Z","iopub.execute_input":"2022-03-19T06:03:21.276259Z","iopub.status.idle":"2022-03-19T06:03:22.150533Z","shell.execute_reply.started":"2022-03-19T06:03:21.276229Z","shell.execute_reply":"2022-03-19T06:03:22.149611Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Checking for null after filling the null values, since I was in plan of droping some columns like date. So I didn't fill null for date columns ","metadata":{}},{"cell_type":"code","source":"df_train.isna().sum()/len(df_train)*100","metadata":{"id":"38548ab3","outputId":"882035ce-0df4-4a52-f459-880f03660f01","execution":{"iopub.status.busy":"2022-03-19T06:03:22.151803Z","iopub.execute_input":"2022-03-19T06:03:22.152026Z","iopub.status.idle":"2022-03-19T06:03:22.496306Z","shell.execute_reply.started":"2022-03-19T06:03:22.152000Z","shell.execute_reply":"2022-03-19T06:03:22.495727Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_test.isna().sum()/len(df_test)*100","metadata":{"id":"e17e4ac4","outputId":"9a4fed07-fb5a-4347-af60-956c3fcfb89f","execution":{"iopub.status.busy":"2022-03-19T06:03:22.497369Z","iopub.execute_input":"2022-03-19T06:03:22.497698Z","iopub.status.idle":"2022-03-19T06:03:22.714893Z","shell.execute_reply.started":"2022-03-19T06:03:22.497659Z","shell.execute_reply":"2022-03-19T06:03:22.714190Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"cat","metadata":{"id":"e9afa9e5","outputId":"472ffebe-41d3-4817-bc6d-1b44c2548f6e","execution":{"iopub.status.busy":"2022-03-19T06:03:22.715960Z","iopub.execute_input":"2022-03-19T06:03:22.716304Z","iopub.status.idle":"2022-03-19T06:03:22.722515Z","shell.execute_reply.started":"2022-03-19T06:03:22.716276Z","shell.execute_reply":"2022-03-19T06:03:22.721551Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Making a seperate variable for date columns and getting prepare to drop it.","metadata":{}},{"cell_type":"code","source":"date = [i for i in cat if 'date' in i]\ndate","metadata":{"id":"ce71744b","outputId":"76234a1a-853f-44a8-9788-816d9afc32fd","execution":{"iopub.status.busy":"2022-03-19T06:03:22.724122Z","iopub.execute_input":"2022-03-19T06:03:22.724399Z","iopub.status.idle":"2022-03-19T06:03:22.739250Z","shell.execute_reply.started":"2022-03-19T06:03:22.724370Z","shell.execute_reply":"2022-03-19T06:03:22.737749Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# removing the date columns in train\n\nfor i in date:\n    df_train.drop(i,1,inplace=True)\ndf_train.columns","metadata":{"id":"1eb16130","outputId":"e6131df4-e008-4b22-cb06-dd887a124125","execution":{"iopub.status.busy":"2022-03-19T06:03:22.740328Z","iopub.execute_input":"2022-03-19T06:03:22.740547Z","iopub.status.idle":"2022-03-19T06:03:24.132428Z","shell.execute_reply.started":"2022-03-19T06:03:22.740521Z","shell.execute_reply":"2022-03-19T06:03:24.131617Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# date is removed in train\n\nfor i in df_train.columns:\n    print(i)","metadata":{"scrolled":true,"id":"c88dfe12","outputId":"dd3069b7-5ea6-43cb-a735-f87030597f24","execution":{"iopub.status.busy":"2022-03-19T06:03:24.133613Z","iopub.execute_input":"2022-03-19T06:03:24.133880Z","iopub.status.idle":"2022-03-19T06:03:24.154133Z","shell.execute_reply.started":"2022-03-19T06:03:24.133851Z","shell.execute_reply":"2022-03-19T06:03:24.147920Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# removing the date columns in test\n\nfor i in date:\n    df_test.drop(i,1,inplace=True)\ndf_test.columns","metadata":{"id":"9ccc1887","outputId":"925cc742-f4d5-48b7-b2fb-35a12d655a9c","execution":{"iopub.status.busy":"2022-03-19T06:03:24.155984Z","iopub.execute_input":"2022-03-19T06:03:24.156238Z","iopub.status.idle":"2022-03-19T06:03:25.070703Z","shell.execute_reply.started":"2022-03-19T06:03:24.156202Z","shell.execute_reply":"2022-03-19T06:03:25.069759Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# date is removed in test\n\nfor i in df_test.columns:\n    print(i)","metadata":{"id":"a3f448c2","outputId":"27130a78-1c62-4ef2-d248-17881fb5b73f","execution":{"iopub.status.busy":"2022-03-19T06:03:25.071979Z","iopub.execute_input":"2022-03-19T06:03:25.072219Z","iopub.status.idle":"2022-03-19T06:03:25.097241Z","shell.execute_reply.started":"2022-03-19T06:03:25.072190Z","shell.execute_reply":"2022-03-19T06:03:25.085817Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_train.target.unique()","metadata":{"id":"01cef582","outputId":"40daef4a-baf2-4be7-ee0b-1ecdbc08d668","execution":{"iopub.status.busy":"2022-03-19T06:03:25.098439Z","iopub.execute_input":"2022-03-19T06:03:25.098910Z","iopub.status.idle":"2022-03-19T06:03:25.114345Z","shell.execute_reply.started":"2022-03-19T06:03:25.098873Z","shell.execute_reply":"2022-03-19T06:03:25.113599Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# removing date columns in categorical columns stored variable (cat)\n\nfor i in date:\n    cat.remove(i)\ncat","metadata":{"id":"0b938735","execution":{"iopub.status.busy":"2022-03-19T06:03:25.115267Z","iopub.execute_input":"2022-03-19T06:03:25.115476Z","iopub.status.idle":"2022-03-19T06:03:25.126235Z","shell.execute_reply.started":"2022-03-19T06:03:25.115451Z","shell.execute_reply":"2022-03-19T06:03:25.125378Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Encoding \nNote:\n* To represent a unicode string as a string of bytes is known as encoding.\n* To convert a string of bytes to a unicode string is known as decoding.\n\n\nI have choosen LabelEncoding instead of one hot encoding, because one hot will give you a various new columns based on the unique value. I will recommend LabelEncoding to use always, since it will replace in a single column for new value.  ","metadata":{}},{"cell_type":"code","source":"from sklearn.preprocessing import LabelEncoder\nle = LabelEncoder()","metadata":{"id":"98523571","execution":{"iopub.status.busy":"2022-03-19T06:03:25.127762Z","iopub.execute_input":"2022-03-19T06:03:25.128511Z","iopub.status.idle":"2022-03-19T06:03:25.138000Z","shell.execute_reply.started":"2022-03-19T06:03:25.128464Z","shell.execute_reply":"2022-03-19T06:03:25.136990Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# encoding train data\n\ndf_train['target'] = le.fit_transform(df_train['target'])\ndf_train['home_team_name'] = le.fit_transform(df_train['home_team_name'])\ndf_train['away_team_name'] = le.fit_transform(df_train['away_team_name'])\ndf_train['league_name'] = le.fit_transform(df_train['league_name'])\ndf_train['is_cup'] = df_train['is_cup'].replace({True:1,False:0})","metadata":{"id":"016affcc","execution":{"iopub.status.busy":"2022-03-19T06:03:25.138908Z","iopub.execute_input":"2022-03-19T06:03:25.139122Z","iopub.status.idle":"2022-03-19T06:03:25.456068Z","shell.execute_reply.started":"2022-03-19T06:03:25.139096Z","shell.execute_reply":"2022-03-19T06:03:25.455210Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Since after encoding manually for column is_cup we got 1 null values, and it's normal to get some times. So here I have filled with 0.","metadata":{}},{"cell_type":"code","source":"# filling null value for is_cup in train data.\n\ndf_train['is_cup'] = df_train['is_cup'].fillna(0)","metadata":{"id":"ade0ce68","execution":{"iopub.status.busy":"2022-03-19T06:03:25.460345Z","iopub.execute_input":"2022-03-19T06:03:25.460606Z","iopub.status.idle":"2022-03-19T06:03:25.466451Z","shell.execute_reply.started":"2022-03-19T06:03:25.460578Z","shell.execute_reply":"2022-03-19T06:03:25.465769Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_train.head()","metadata":{"id":"f433c226","outputId":"f96a985b-5551-4b27-cf6c-14e3e03d298c","execution":{"iopub.status.busy":"2022-03-19T06:03:25.467311Z","iopub.execute_input":"2022-03-19T06:03:25.467532Z","iopub.status.idle":"2022-03-19T06:03:25.646944Z","shell.execute_reply.started":"2022-03-19T06:03:25.467504Z","shell.execute_reply":"2022-03-19T06:03:25.646021Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"for i in date:\n    cat_test.remove(i)\nprint(cat_test)","metadata":{"id":"19c131b5","outputId":"f7626d9b-3936-4222-e023-56026edc61b6","execution":{"iopub.status.busy":"2022-03-19T06:03:25.648032Z","iopub.execute_input":"2022-03-19T06:03:25.648250Z","iopub.status.idle":"2022-03-19T06:03:25.653842Z","shell.execute_reply.started":"2022-03-19T06:03:25.648224Z","shell.execute_reply":"2022-03-19T06:03:25.652839Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# encoding test data\n\ndf_test['home_team_name'] = le.fit_transform(df_test['home_team_name'])\ndf_test['away_team_name'] = le.fit_transform(df_test['away_team_name'])\ndf_test['league_name'] = le.fit_transform(df_test['league_name'])\ndf_test['is_cup'] = df_test['is_cup'].replace({True:1,False:0})","metadata":{"id":"00111eac","execution":{"iopub.status.busy":"2022-03-19T06:03:25.655770Z","iopub.execute_input":"2022-03-19T06:03:25.656064Z","iopub.status.idle":"2022-03-19T06:03:25.843832Z","shell.execute_reply.started":"2022-03-19T06:03:25.656021Z","shell.execute_reply":"2022-03-19T06:03:25.842971Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# filling null value for is_cup in test data.\n\ndf_test['is_cup'] = df_test['is_cup'].fillna(0)","metadata":{"id":"62870e2b","execution":{"iopub.status.busy":"2022-03-19T06:03:25.844903Z","iopub.execute_input":"2022-03-19T06:03:25.845121Z","iopub.status.idle":"2022-03-19T06:03:25.849910Z","shell.execute_reply.started":"2022-03-19T06:03:25.845093Z","shell.execute_reply":"2022-03-19T06:03:25.849054Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_test.head()","metadata":{"id":"22045a48","outputId":"08335a36-d9de-4dc9-ffc4-c2c52385dd58","execution":{"iopub.status.busy":"2022-03-19T06:03:25.851306Z","iopub.execute_input":"2022-03-19T06:03:25.851531Z","iopub.status.idle":"2022-03-19T06:03:26.034802Z","shell.execute_reply.started":"2022-03-19T06:03:25.851504Z","shell.execute_reply":"2022-03-19T06:03:26.033755Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Splitting the data","metadata":{}},{"cell_type":"code","source":"# splitting train data\n\nxtrain = df_train.drop('target',1)\nytrain = df_train.target","metadata":{"id":"907395fe","execution":{"iopub.status.busy":"2022-03-19T06:03:26.036057Z","iopub.execute_input":"2022-03-19T06:03:26.036731Z","iopub.status.idle":"2022-03-19T06:03:26.211266Z","shell.execute_reply.started":"2022-03-19T06:03:26.036662Z","shell.execute_reply":"2022-03-19T06:03:26.210450Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# splitting test data\n\nxtest = df_test","metadata":{"id":"5c74e33d","execution":{"iopub.status.busy":"2022-03-19T06:03:26.212573Z","iopub.execute_input":"2022-03-19T06:03:26.212799Z","iopub.status.idle":"2022-03-19T06:03:26.217099Z","shell.execute_reply.started":"2022-03-19T06:03:26.212773Z","shell.execute_reply":"2022-03-19T06:03:26.216290Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_test.head()","metadata":{"id":"19d1cecb","outputId":"87fd34d7-fd82-4e4e-e345-2c838e0ea9f7","execution":{"iopub.status.busy":"2022-03-19T06:03:26.218652Z","iopub.execute_input":"2022-03-19T06:03:26.219183Z","iopub.status.idle":"2022-03-19T06:03:26.398924Z","shell.execute_reply.started":"2022-03-19T06:03:26.219126Z","shell.execute_reply":"2022-03-19T06:03:26.398003Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# ML algorithms \nDecisionTreeClassifier:\n\n        Decision tree classifiers are used successfully in many diverse areas. Their most important feature is the capability of capturing descriptive decisionmaking knowledge from the supplied data. Decision tree can be generated from training sets. The procedure for such generation based on the set of objects (S), each belonging to one of the classes C1, C2, …, Ck\n\nAdvantages:\n* A decision tree is easy to understand and interpret.\n* Expert opinion and preferences can be included, as well as hard data.\n* Can be used with other decision techniques.\n* New scenarios can easily be added.\n\nDisadvantages:\n* If a decision tree is used for categorical variables with multiple levels, those variables with more levels will have more information gain.\n* Calculations can quickly become very complex, although this is usually only a problem if the tree is being created by hand.","metadata":{}},{"cell_type":"code","source":"! pip install dtreeviz","metadata":{"execution":{"iopub.status.busy":"2022-03-19T06:03:26.400558Z","iopub.execute_input":"2022-03-19T06:03:26.401584Z","iopub.status.idle":"2022-03-19T06:03:40.148807Z","shell.execute_reply.started":"2022-03-19T06:03:26.401518Z","shell.execute_reply":"2022-03-19T06:03:40.147978Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"from sklearn.linear_model import LogisticRegression\nfrom sklearn.tree import DecisionTreeClassifier\nfrom sklearn.ensemble import RandomForestClassifier\nfrom sklearn.neighbors import KNeighborsClassifier\nfrom sklearn.naive_bayes import GaussianNB\nfrom sklearn.linear_model import SGDClassifier\nfrom lightgbm import LGBMClassifier\nfrom sklearn.ensemble import GradientBoostingClassifier\nfrom sklearn import tree\nfrom dtreeviz.trees import *\nimport graphviz","metadata":{"id":"461b3121","execution":{"iopub.status.busy":"2022-03-19T06:03:40.150284Z","iopub.execute_input":"2022-03-19T06:03:40.150873Z","iopub.status.idle":"2022-03-19T06:03:41.436074Z","shell.execute_reply.started":"2022-03-19T06:03:40.150829Z","shell.execute_reply":"2022-03-19T06:03:41.435101Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"clf = DecisionTreeClassifier(max_depth=4,random_state=1234)\nclf.fit(xtrain, ytrain)\ntree.export_text(clf)","metadata":{"execution":{"iopub.status.busy":"2022-03-19T06:03:41.437381Z","iopub.execute_input":"2022-03-19T06:03:41.437626Z","iopub.status.idle":"2022-03-19T06:03:46.519203Z","shell.execute_reply.started":"2022-03-19T06:03:41.437595Z","shell.execute_reply":"2022-03-19T06:03:46.518381Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# dtreeviz ","metadata":{}},{"cell_type":"markdown","source":"dtreeviz is used to explain the flow of DecisionTreeClassifier in a visual way.","metadata":{}},{"cell_type":"code","source":"fig = plt.figure(figsize=(25,20))\nviz = dtreeviz(clf,\n               xtrain,\n               ytrain,\n               target_name='Target',\n               feature_names=xtrain.columns,\n               title=\"Football Decisiontree Classification\",\n               class_names=['Home', 'Draw','Away'],\n               histtype='barstacked', \n               scale=1.2)\nviz","metadata":{"execution":{"iopub.status.busy":"2022-03-19T06:03:46.520503Z","iopub.execute_input":"2022-03-19T06:03:46.520768Z","iopub.status.idle":"2022-03-19T06:04:09.029776Z","shell.execute_reply.started":"2022-03-19T06:03:46.520729Z","shell.execute_reply":"2022-03-19T06:04:09.028814Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# CalibratedClassifier using Linear Regression","metadata":{}},{"cell_type":"code","source":"lr = LogisticRegression(solver='sag')","metadata":{"id":"ef2f0845","execution":{"iopub.status.busy":"2022-03-19T06:04:09.031532Z","iopub.execute_input":"2022-03-19T06:04:09.031814Z","iopub.status.idle":"2022-03-19T06:04:09.036347Z","shell.execute_reply.started":"2022-03-19T06:04:09.031782Z","shell.execute_reply":"2022-03-19T06:04:09.035384Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"lr.fit(xtrain,ytrain)\nprediction = lr.predict_proba(xtest)\nprediction","metadata":{"id":"Mvg41am28816","outputId":"ff4a06e9-fb16-4cac-b80c-bfc7b13314ef","execution":{"iopub.status.busy":"2022-03-19T06:04:09.037972Z","iopub.execute_input":"2022-03-19T06:04:09.038195Z","iopub.status.idle":"2022-03-19T06:04:50.780977Z","shell.execute_reply.started":"2022-03-19T06:04:09.038169Z","shell.execute_reply":"2022-03-19T06:04:50.779997Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"from sklearn.preprocessing import MinMaxScaler\nfrom sklearn.preprocessing import StandardScaler\nfrom sklearn.preprocessing import RobustScaler\n# fit scaler on training data\nnorm = MinMaxScaler().fit(xtrain)\n# transform training data\nxtrain_norm = norm.transform(xtrain)\n# transform testing data\nxtest_norm = norm.transform(xtest)","metadata":{"execution":{"iopub.status.busy":"2022-03-19T06:04:50.782752Z","iopub.execute_input":"2022-03-19T06:04:50.783313Z","iopub.status.idle":"2022-03-19T06:04:51.204199Z","shell.execute_reply.started":"2022-03-19T06:04:50.783264Z","shell.execute_reply":"2022-03-19T06:04:51.203260Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"from sklearn.calibration import CalibratedClassifierCV","metadata":{"execution":{"iopub.status.busy":"2022-03-19T06:04:51.205401Z","iopub.execute_input":"2022-03-19T06:04:51.205652Z","iopub.status.idle":"2022-03-19T06:04:51.219021Z","shell.execute_reply.started":"2022-03-19T06:04:51.205622Z","shell.execute_reply":"2022-03-19T06:04:51.218164Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"cc = CalibratedClassifierCV(lr,cv=3)\ncc.fit(xtrain_norm,ytrain)\nprediction_1 = cc.predict_proba(xtest_norm)\nprediction_1","metadata":{"execution":{"iopub.status.busy":"2022-03-19T06:04:51.220521Z","iopub.execute_input":"2022-03-19T06:04:51.220915Z","iopub.status.idle":"2022-03-19T06:05:27.780832Z","shell.execute_reply.started":"2022-03-19T06:04:51.220876Z","shell.execute_reply":"2022-03-19T06:05:27.779973Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Submission","metadata":{}},{"cell_type":"markdown","source":"Making a dataframe by including all required columns to submit. The unique values from the target column are created as a seperate column for final submission.","metadata":{}},{"cell_type":"code","source":"sub = pd.concat([pd.DataFrame(df_test.id),pd.DataFrame(prediction_1,\n                 columns=['home','draw','away'])[['home', 'draw','away']]],1)\nsub.set_index('id',inplace=True)\nsub.head()","metadata":{"id":"z608QYcI44RJ","outputId":"d509efec-2ecd-4c07-b291-ffd8507b7162","execution":{"iopub.status.busy":"2022-03-19T06:05:27.782525Z","iopub.execute_input":"2022-03-19T06:05:27.783074Z","iopub.status.idle":"2022-03-19T06:05:27.810587Z","shell.execute_reply.started":"2022-03-19T06:05:27.783007Z","shell.execute_reply":"2022-03-19T06:05:27.809736Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# submission file\n\nsub.to_csv('submission.csv')","metadata":{"execution":{"iopub.status.busy":"2022-03-19T06:05:27.812513Z","iopub.execute_input":"2022-03-19T06:05:27.813176Z","iopub.status.idle":"2022-03-19T06:05:28.411067Z","shell.execute_reply.started":"2022-03-19T06:05:27.813126Z","shell.execute_reply":"2022-03-19T06:05:28.409866Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"sub.head().style.highlight_max(color='lightgreen',axis =1)","metadata":{"execution":{"iopub.status.busy":"2022-03-19T06:05:28.412430Z","iopub.execute_input":"2022-03-19T06:05:28.412683Z","iopub.status.idle":"2022-03-19T06:05:28.490025Z","shell.execute_reply.started":"2022-03-19T06:05:28.412638Z","shell.execute_reply":"2022-03-19T06:05:28.489211Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# DecisionTree","metadata":{}},{"cell_type":"code","source":"dt = DecisionTreeClassifier()","metadata":{"execution":{"iopub.status.busy":"2022-03-19T06:05:28.491376Z","iopub.execute_input":"2022-03-19T06:05:28.491603Z","iopub.status.idle":"2022-03-19T06:05:28.495768Z","shell.execute_reply.started":"2022-03-19T06:05:28.491575Z","shell.execute_reply":"2022-03-19T06:05:28.494838Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"dt.fit(xtrain,ytrain)\nprediction_2 = dt.predict_proba(xtest)\nprediction_2","metadata":{"execution":{"iopub.status.busy":"2022-03-19T06:05:28.497237Z","iopub.execute_input":"2022-03-19T06:05:28.497469Z","iopub.status.idle":"2022-03-19T06:05:50.462007Z","shell.execute_reply.started":"2022-03-19T06:05:28.497442Z","shell.execute_reply":"2022-03-19T06:05:50.461028Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"sub1 = pd.concat([pd.DataFrame(df_test.id),pd.DataFrame(prediction_2,\n                 columns=['home','draw','away'])[['home', 'draw','away']]],1)\nsub1.set_index('id',inplace=True)\nsub1.head()","metadata":{"execution":{"iopub.status.busy":"2022-03-19T06:05:50.463451Z","iopub.execute_input":"2022-03-19T06:05:50.463708Z","iopub.status.idle":"2022-03-19T06:05:50.481807Z","shell.execute_reply.started":"2022-03-19T06:05:50.463679Z","shell.execute_reply":"2022-03-19T06:05:50.480940Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Hightlight\n\nSince the unique values from the target column are created as a seperate column for final submission and by using style.highlight_max can able to highlight the hihghest vaue of each column. Which will be easy to identify the max value of each column as you can see, that all the value are 1.0*100 = 100% accuracy by id x feature name. The sample is used to get the data in randomly.\n","metadata":{}},{"cell_type":"code","source":"sub1.sample(10).style.highlight_max(color='lightgreen',axis =1)","metadata":{"execution":{"iopub.status.busy":"2022-03-19T06:05:50.482951Z","iopub.execute_input":"2022-03-19T06:05:50.483183Z","iopub.status.idle":"2022-03-19T06:05:50.502020Z","shell.execute_reply.started":"2022-03-19T06:05:50.483155Z","shell.execute_reply":"2022-03-19T06:05:50.501197Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"If you enjoyed my work don't forget to upvote, which will really encourage me to work more like this. Do follow me on social meadia, for more coding and explanation of concept in Data Science and Big Data:\n\n\n* linkedin : https://www.linkedin.com/in/anandaram-g/\n* Github : https://github.com/anand-lab-172\n* Medium : https://medium.com/@anandlab\n","metadata":{}}]}