{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"## Football Match Probability Prediction\n\n**Competition Description**\n\nFootball has been at the heart of data science for more than a decade. If today's algorithms focus on event detection, player style, or team analysis, predicting the results of a match stays an open challenge.\n\nPredicting the outcomes of a match between two teams depends mostly (but not only) on their current form. The form of a team can be viewed as their recent sequence of results versus the other teams. So match probabilities between two teams can be different given their calendar.\n\nThis competition is about predicting the probabilities of more than 150000 match outcomes using the recent sequence of 10 matches of the teams.\n\n**Introduction**\n\nThe goal of this notebook is to develop a LSTM model.\nIt may be improved greatly by more feature engineering and other stuffs.\n\n> **In case you fork/copy/like this notebook:**\n> **Upvote it s'il te plaît/Per favore/Please**\n\nThis notebook was inspired (not copied) by the work of: \nhttps://www.kaggle.com/igorkf/football-match-probability-prediction-lstm-starter","metadata":{"_uuid":"7cd960c6-91d4-4ca0-a69a-9e4e520d184a","_cell_guid":"eab67375-a4a9-4e08-bb4a-3c099e21dfa9","trusted":true}},{"cell_type":"code","source":"!pip install scikit-learn==1.0.2","metadata":{"execution":{"iopub.status.busy":"2022-05-04T03:55:07.866323Z","iopub.execute_input":"2022-05-04T03:55:07.866644Z","iopub.status.idle":"2022-05-04T03:55:22.697602Z","shell.execute_reply.started":"2022-05-04T03:55:07.866559Z","shell.execute_reply":"2022-05-04T03:55:22.696756Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Import libraries\n# import gc\nimport numpy as np\nimport pandas as pd\nimport datetime as dt\n#\nfrom sklearn.model_selection import KFold\nfrom sklearn.preprocessing import RobustScaler, LabelEncoder\nfrom sklearn.metrics import accuracy_score, log_loss\n#\nimport tensorflow as tf\nfrom tensorflow.keras.utils import plot_model, to_categorical\nfrom tensorflow.keras.models import Model, load_model\nfrom tensorflow.keras.callbacks import EarlyStopping, ReduceLROnPlateau, LearningRateScheduler\nfrom tensorflow.keras.optimizers.schedules import ExponentialDecay\nfrom tensorflow.keras import layers\n# from tensorflow.keras.layers import Bidirectional, LSTM, MaxPooling2D, Conv2D\n# from tensorflow.keras.layers import Dense, Dropout, Input, Flatten, Masking\n# from tensorflow.keras.layers import Concatenate, Add, GRU\n# from tensorflow.keras.callbacks import ModelCheckpoint","metadata":{"_uuid":"920e6d1a-69cf-46e8-a78b-24f7fe780f73","_cell_guid":"c6c6ebe4-3bf5-4a40-8f8e-3f065ecd9e16","collapsed":false,"jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2022-05-04T03:55:22.700514Z","iopub.execute_input":"2022-05-04T03:55:22.700799Z","iopub.status.idle":"2022-05-04T03:55:28.524760Z","shell.execute_reply.started":"2022-05-04T03:55:22.700751Z","shell.execute_reply":"2022-05-04T03:55:28.524017Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#loading the data\ntrain = pd.read_csv('/kaggle/input/football-match-probability-prediction/train.csv')\ntest = pd.read_csv('/kaggle/input/football-match-probability-prediction/test.csv')\nsubmission = pd.read_csv('/kaggle/input/football-match-probability-prediction/sample_submission.csv')","metadata":{"_uuid":"f9cd6673-ccc8-4e4c-bce0-c4e7e1d8e98a","_cell_guid":"e5fbcac1-440c-4d84-8f6f-6ef0f1102c70","collapsed":false,"jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2022-05-04T03:55:28.526044Z","iopub.execute_input":"2022-05-04T03:55:28.526271Z","iopub.status.idle":"2022-05-04T03:55:37.325369Z","shell.execute_reply.started":"2022-05-04T03:55:28.526240Z","shell.execute_reply":"2022-05-04T03:55:37.324608Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train = train[train['home_team_name'] != train['away_team_name']]#.reset_index(drop=True)\ntrain = train[train['home_team_name'].notnull()]","metadata":{"execution":{"iopub.status.busy":"2022-05-04T03:55:37.327757Z","iopub.execute_input":"2022-05-04T03:55:37.328202Z","iopub.status.idle":"2022-05-04T03:55:37.541665Z","shell.execute_reply.started":"2022-05-04T03:55:37.328163Z","shell.execute_reply":"2022-05-04T03:55:37.540891Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# train = train[~train.league_name.str.lower().str.contains('friend')]","metadata":{"execution":{"iopub.status.busy":"2022-05-04T03:55:37.543904Z","iopub.execute_input":"2022-05-04T03:55:37.544102Z","iopub.status.idle":"2022-05-04T03:55:37.547087Z","shell.execute_reply.started":"2022-05-04T03:55:37.544077Z","shell.execute_reply":"2022-05-04T03:55:37.546433Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# test[test.league_name.str.lower().str.contains('friend')]","metadata":{"execution":{"iopub.status.busy":"2022-05-04T03:55:37.548637Z","iopub.execute_input":"2022-05-04T03:55:37.549144Z","iopub.status.idle":"2022-05-04T03:55:37.557215Z","shell.execute_reply.started":"2022-05-04T03:55:37.549106Z","shell.execute_reply":"2022-05-04T03:55:37.556573Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# train = train[train['home_team_name'] != train['away_team_name']]#.reset_index(drop=True)\n# train = train[train['home_team_name'].notnull()]","metadata":{"execution":{"iopub.status.busy":"2022-05-04T03:55:37.558339Z","iopub.execute_input":"2022-05-04T03:55:37.558751Z","iopub.status.idle":"2022-05-04T03:55:37.565739Z","shell.execute_reply.started":"2022-05-04T03:55:37.558713Z","shell.execute_reply":"2022-05-04T03:55:37.565063Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# exclude matches with no history at date 1 - full of NA (1159 rows excluded)\ntrain.dropna(subset=['home_team_history_match_date_1'], inplace = True)","metadata":{"execution":{"iopub.status.busy":"2022-05-04T03:55:37.567013Z","iopub.execute_input":"2022-05-04T03:55:37.567265Z","iopub.status.idle":"2022-05-04T03:55:37.672069Z","shell.execute_reply.started":"2022-05-04T03:55:37.567231Z","shell.execute_reply":"2022-05-04T03:55:37.671277Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Parameters\nSet some parameters that will be used later.","metadata":{"_uuid":"ac3c4d94-2456-491c-b634-480be3505386","_cell_guid":"62203296-73b6-416a-b8af-83919edc6721","trusted":true}},{"cell_type":"code","source":"# Set seed\nnp.random.seed(123)\ntf.random.set_seed(123)\n\n#Some parameters\nMASK = -666 # fill NA with -666 (the number of the beast)\nT_HIST = 10 # time history, last 10 games\nCLASS = 3 #number of classes (home, draw, away)\n\nDEBUG = False\n# Run on a small sample of the data\nif DEBUG:\n    train = train[:10000]","metadata":{"_uuid":"9bc24db5-b55a-4e16-87c3-ce6b4cd8a552","_cell_guid":"72e32380-622d-40ce-b35a-674120f981b1","collapsed":false,"jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2022-05-04T03:55:37.673628Z","iopub.execute_input":"2022-05-04T03:55:37.674129Z","iopub.status.idle":"2022-05-04T03:55:37.679638Z","shell.execute_reply.started":"2022-05-04T03:55:37.674091Z","shell.execute_reply":"2022-05-04T03:55:37.678987Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print(f\"Train: {train.shape} \\n Submission: {submission.shape}\")\ntrain.head()","metadata":{"_uuid":"f98d00e1-b8db-475f-926b-3f27fc07bc55","_cell_guid":"36d932ff-1cf7-4fa3-97c3-2657a83150a9","collapsed":false,"jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2022-05-04T03:55:37.681026Z","iopub.execute_input":"2022-05-04T03:55:37.681359Z","iopub.status.idle":"2022-05-04T03:55:37.721466Z","shell.execute_reply.started":"2022-05-04T03:55:37.681319Z","shell.execute_reply":"2022-05-04T03:55:37.720818Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Feature Engineering\nThe **most** important part of Machine Learning.","metadata":{"_uuid":"91794a39-6ab8-4fac-bf6a-833de1182b06","_cell_guid":"838bf7ec-34de-46d8-8de9-0c9be9cb1fea","trusted":true}},{"cell_type":"code","source":"# for cols \"date\", change to datatime \nfor col in train.filter(regex='date', axis=1).columns:\n    train[col] = pd.to_datetime(train[col])\n    test[col] = pd.to_datetime(test[col])\n\n# Some feature engineering\n\ndef add_features(df):\n    df['home_team_history_match_date_0'] = df['match_date']\n    df['away_team_history_match_date_0'] = df['match_date']\n    for i in range(1, 11): # range from 1 to 10\n        # Feat. difference of days\n        df[f'home_team_history_match_DIFF_days_{i}'] = (df['match_date'] - df[f'home_team_history_match_date_{i}']).dt.days\n        df[f'away_team_history_match_DIFF_days_{i}'] = (df['match_date'] - df[f'away_team_history_match_date_{i}']).dt.days\n        # date diff 2\n        # df[f'home_team_history_match_DIFF2_days_{i}'] = (df[f'home_team_history_match_date_{i}'] - df[f'home_team_history_match_date_{i-1}']).dt.days\n        # df[f'away_team_history_match_DIFF2_days_{i}'] = (df[f'away_team_history_match_date_{i}'] - df[f'away_team_history_match_date_{i-1}']).dt.days\n    # Feat. difference of scored goals\n        df[f'home_team_history_DIFF_goal_{i}'] = df[f'home_team_history_goal_{i}'] - df[f'home_team_history_opponent_goal_{i}']\n        df[f'away_team_history_DIFF_goal_{i}'] = df[f'away_team_history_goal_{i}'] - df[f'away_team_history_opponent_goal_{i}']\n    # Feat dummy winner x loser\n        df[f'home_winner_{i}'] = np.where(df[f'home_team_history_DIFF_goal_{i}'] > 0, 1., 0.) \n        df[f'home_loser_{i}'] = np.where(df[f'home_team_history_DIFF_goal_{i}'] < 0, 1., 0.)\n        df[f'away_winner_{i}'] = np.where(df[f'away_team_history_DIFF_goal_{i}'] > 0, 1., 0.)\n        df[f'away_loser_{i}'] = np.where(df[f'away_team_history_DIFF_goal_{i}'] < 0, 1., 0.)\n    # Results: multiple nested where # away:0, draw:1, home:2\n        df[f'home_team_result_{i}'] = np.where(df[f'home_team_history_DIFF_goal_{i}'] > 0., 2.,\n                         (np.where(df[f'home_team_history_DIFF_goal_{i}'] == 0., 1,\n                                   np.where(df[f'home_team_history_DIFF_goal_{i}'].isna(), np.nan, 0))))\n        df[f'away_team_result_{i}'] = np.where(df[f'away_team_history_DIFF_goal_{i}'] > 0., 2.,\n                         (np.where(df[f'away_team_history_DIFF_goal_{i}'] == 0., 1.,\n                                   np.where(df[f'away_team_history_DIFF_goal_{i}'].isna(), np.nan, 0.))))\n    # Feat. difference of rating (\"modified\" ELO RATING)\n        df[f'home_team_history_ELO_rating_{i}'] = 1/(1+10**((df[f'home_team_history_opponent_rating_{i}']-df[f'home_team_history_rating_{i}'])/10))\n        df[f'away_team_history_ELO_rating_{i}'] = 1/(1+10**((df[f'away_team_history_opponent_rating_{i}']-df[f'away_team_history_rating_{i}'])/10))\n        df[f'home_away_team_history_ELO_rating_{i}'] = 1/(1+10**((df[f'away_team_history_rating_{i}']-df[f'home_team_history_rating_{i}'])/10))\n    # Dif\n       #  df[f'home_team_history_DIF_rating_{i}'] = -df[f'home_team_history_opponent_rating_{i}']+df[f'home_team_history_rating_{i}']\n       #  df[f'away_team_history_DIF_rating_{i}'] = -df[f'away_team_history_opponent_rating_{i}']+df[f'away_team_history_rating_{i}']\n       #  df[f'home_away_team_history_DIF_rating_{i}'] = -df[f'away_team_history_rating_{i}']+df[f'home_team_history_rating_{i}']\n    # Feat. same coach id\n        df[f'home_team_history_SAME_coaX_{i}'] = np.where(df['home_team_coach_id']==df[f'home_team_history_coach_{i}'],1,0)\n        df[f'away_team_history_SAME_coaX_{i}'] = np.where(df['away_team_coach_id']==df[f'away_team_history_coach_{i}'],1,0) \n    # Feat. same league id\n        df[f'home_team_history_SAME_leaG_{i}'] = np.where(df['league_id']==df[f'home_team_history_league_id_{i}'],1,0)\n        df[f'away_team_history_SAME_leaG_{i}'] = np.where(df['league_id']==df[f'away_team_history_league_id_{i}'],1,0) \n    # Features...\n        df[f'feature_A_{i}'] = df[f'home_team_history_ELO_rating_{i}'] * df[f'home_team_history_is_play_home_{i}']\n        df[f'feature_B_{i}'] = df[f'away_team_history_ELO_rating_{i}'] * df[f'away_team_history_is_play_home_{i}']\n        df[f'feature_C_{i}'] = df[f'home_away_team_history_ELO_rating_{i}'] * df[f'home_team_history_is_play_home_{i}']\n        df[f'feature_X_{i}'] = df[f'home_team_history_ELO_rating_{i}'] * df[f'home_team_history_SAME_leaG_{i}']\n        df[f'feature_Y_{i}'] = df[f'away_team_history_ELO_rating_{i}'] * df[f'away_team_history_SAME_leaG_{i}']\n        df[f'feature_Z_{i}'] = df[f'home_away_team_history_ELO_rating_{i}'] * df[f'home_team_history_SAME_leaG_{i}']\n    # Fill NA with -666\n    # df.fillna(MASK, inplace = True)\n    return df\n\ntrain = add_features(train)\ntest = add_features(test)","metadata":{"_uuid":"daed24ca-8f78-4419-b19f-3b97d78f29f7","_cell_guid":"fa41a07c-c823-4716-8666-21f3f8cc12ed","collapsed":false,"jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2022-05-04T03:55:37.722891Z","iopub.execute_input":"2022-05-04T03:55:37.723343Z","iopub.status.idle":"2022-05-04T03:55:40.235988Z","shell.execute_reply.started":"2022-05-04T03:55:37.723307Z","shell.execute_reply":"2022-05-04T03:55:40.235228Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train","metadata":{"execution":{"iopub.status.busy":"2022-05-04T03:55:40.237148Z","iopub.execute_input":"2022-05-04T03:55:40.237409Z","iopub.status.idle":"2022-05-04T03:55:40.324044Z","shell.execute_reply.started":"2022-05-04T03:55:40.237375Z","shell.execute_reply":"2022-05-04T03:55:40.323348Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Scaling and Reshape\nThe input/output of the lstm is very trick. It is expected an array of dimensions (Matches/Batch, Time history, Features). I hope I did it right.","metadata":{"_uuid":"0fa67e85-50f7-43af-a9dc-b11c05bcbf8c","_cell_guid":"878d27a4-831a-4a9f-9c9b-275409afc69b","trusted":true}},{"cell_type":"code","source":"# save targets\n# train_id = train['id'].copy()\ntrain_y = train['target'].copy()\n#keep only some features\ntrain_x = train.drop(['target', 'home_team_name', 'away_team_name'], axis=1) #, inplace=True) # is_cup EXCLUDED\n# Exclude all date, league, coach columns\ntrain_x.drop(train.filter(regex='date').columns, axis=1, inplace = True)\ntrain_x.drop(train.filter(regex='league').columns, axis=1, inplace = True)\ntrain_x.drop(train.filter(regex='coach').columns, axis=1, inplace = True)\n#\n# Test set\n# test_id = test['id'].copy()\ntest_x = test.drop(['home_team_name', 'away_team_name'], axis=1)#, inplace=True) # is_cup EXCLUDED\n# Exclude all date, league, coach columns\ntest_x.drop(test.filter(regex='date').columns, axis=1, inplace = True)\ntest_x.drop(test.filter(regex='league').columns, axis=1, inplace = True)\ntest_x.drop(test.filter(regex='coach').columns, axis=1, inplace = True)","metadata":{"_uuid":"7b65ce25-5a33-4fb8-b04b-0f288ea3d9bb","_cell_guid":"fe6715ba-250e-4a67-bc76-f046ba57b793","collapsed":false,"jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2022-05-04T03:55:40.325161Z","iopub.execute_input":"2022-05-04T03:55:40.326014Z","iopub.status.idle":"2022-05-04T03:55:41.370009Z","shell.execute_reply.started":"2022-05-04T03:55:40.325973Z","shell.execute_reply":"2022-05-04T03:55:41.369232Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Target, train and test shape\nprint(f\"Target: {train_y.shape} \\n Train shape: {train_x.shape} \\n Test: {test_x.shape}\")\nprint(f\"Column names: {list(train_x.columns)}\")","metadata":{"_uuid":"9c748333-e838-48d8-88f0-33ef0051e57e","_cell_guid":"9ab58872-8c97-4c9a-a110-a55588e0f29f","collapsed":false,"jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2022-05-04T03:55:41.373968Z","iopub.execute_input":"2022-05-04T03:55:41.374179Z","iopub.status.idle":"2022-05-04T03:55:41.380226Z","shell.execute_reply.started":"2022-05-04T03:55:41.374152Z","shell.execute_reply":"2022-05-04T03:55:41.378548Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_x.head()","metadata":{"_uuid":"e45561e1-2c45-4bb8-8b20-70aa386fc82a","_cell_guid":"9dda4bc6-19b1-40f4-bab6-ca33dae90f12","collapsed":false,"jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2022-05-04T03:55:41.381679Z","iopub.execute_input":"2022-05-04T03:55:41.382177Z","iopub.status.idle":"2022-05-04T03:55:41.408585Z","shell.execute_reply.started":"2022-05-04T03:55:41.382136Z","shell.execute_reply":"2022-05-04T03:55:41.407723Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Store feature names\n# feature_names = list(train.columns)\n# Pivot dataframe to create an input array for the LSTM network\nfeature_groups = [\"home_team_history_is_play_home\", \"home_team_history_is_cup\",\n    \"home_team_history_goal\", \"home_team_history_opponent_goal\",\n    \"home_team_history_rating\", \"home_team_history_opponent_rating\",  \n    \"away_team_history_is_play_home\", \"away_team_history_is_cup\",\n    \"away_team_history_goal\", \"away_team_history_opponent_goal\",\n    \"away_team_history_rating\", \"away_team_history_opponent_rating\",  \n    \"home_team_history_match_DIFF_days\", \"away_team_history_match_DIFF_days\",\n    # \"home_team_history_match_DIFF2_days\", \"away_team_history_match_DIFF2_days\",\n    \"home_team_history_DIFF_goal\",\"away_team_history_DIFF_goal\",\n    \"home_team_history_ELO_rating\",\"away_team_history_ELO_rating\",\n    \"home_away_team_history_ELO_rating\",\n    \"home_team_history_SAME_coaX\", \"away_team_history_SAME_coaX\",\n    \"home_team_history_SAME_leaG\", \"away_team_history_SAME_leaG\",\n    \"home_team_result\", \"away_team_result\",\n    \"home_winner\", \"home_loser\", \"away_winner\", \"away_loser\",\n    \"feature_A\", \"feature_B\", \"feature_C\",\n    \"feature_X\", \"feature_Y\", \"feature_Z\",\n    # \"home_team_history_DIF_rating\",\"away_team_history_DIF_rating\",\"home_away_team_history_DIF_rating\"\n                 ]      \n\n# Pivot dimension (id*features) x time_history\ntrain_x_pivot = pd.wide_to_long(train_x, stubnames=feature_groups, i=['id','is_cup'], j='time', sep='_', suffix='\\d+')\ntest_x_pivot = pd.wide_to_long(test_x, stubnames=feature_groups, i=['id','is_cup'], j='time', sep='_', suffix='\\d+')\n#\nprint(f\"Train pivot shape: {train_x_pivot.shape}\")  \nprint(f\"Test pivot shape: {test_x_pivot.shape}\") ","metadata":{"_uuid":"c316690b-dbb6-45f2-882c-18bcb932af61","_cell_guid":"46437c51-e209-4f27-8d53-1bf0e8547fc4","collapsed":false,"jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2022-05-04T03:55:41.410315Z","iopub.execute_input":"2022-05-04T03:55:41.410676Z","iopub.status.idle":"2022-05-04T03:57:06.910582Z","shell.execute_reply.started":"2022-05-04T03:55:41.410642Z","shell.execute_reply":"2022-05-04T03:57:06.908981Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# train_x_pivot.head(10)","metadata":{"execution":{"iopub.status.busy":"2022-05-04T03:57:06.912201Z","iopub.execute_input":"2022-05-04T03:57:06.912464Z","iopub.status.idle":"2022-05-04T03:57:06.917986Z","shell.execute_reply.started":"2022-05-04T03:57:06.912426Z","shell.execute_reply":"2022-05-04T03:57:06.917195Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# create columns based on index\ntrain_x_pivot = train_x_pivot.reset_index()\ntest_x_pivot = test_x_pivot.reset_index()\n# Deal with the is_cup feature\n# There are NA in 'is_cup'\ntrain_x_pivot=train_x_pivot.fillna({'is_cup':False})\ntrain_x_pivot['is_cup'] = pd.get_dummies(train_x_pivot['is_cup'], drop_first=True)\n#\ntest_x_pivot=test_x_pivot.fillna({'is_cup':False})\ntest_x_pivot['is_cup']= pd.get_dummies(test_x_pivot['is_cup'], drop_first=True)","metadata":{"execution":{"iopub.status.busy":"2022-05-04T03:57:06.919481Z","iopub.execute_input":"2022-05-04T03:57:06.919975Z","iopub.status.idle":"2022-05-04T03:57:07.482176Z","shell.execute_reply.started":"2022-05-04T03:57:06.919931Z","shell.execute_reply":"2022-05-04T03:57:07.481430Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_x_pivot.tail(20)","metadata":{"execution":{"iopub.status.busy":"2022-05-04T03:57:07.483333Z","iopub.execute_input":"2022-05-04T03:57:07.484396Z","iopub.status.idle":"2022-05-04T03:57:07.528586Z","shell.execute_reply.started":"2022-05-04T03:57:07.484353Z","shell.execute_reply":"2022-05-04T03:57:07.527806Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"test_x_pivot.head(20)","metadata":{"execution":{"iopub.status.busy":"2022-05-04T03:57:07.529833Z","iopub.execute_input":"2022-05-04T03:57:07.530160Z","iopub.status.idle":"2022-05-04T03:57:07.573158Z","shell.execute_reply.started":"2022-05-04T03:57:07.530122Z","shell.execute_reply":"2022-05-04T03:57:07.572406Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Feature engineering again!!! Group by id, stats over time\ndef add_features_II(df):\n    # goals\n    df['home_team_history_DIFF_goal_csum'] = df.groupby('id')['home_team_history_DIFF_goal'].cumsum()\n    df['away_team_history_DIFF_goal_csum'] = df.groupby('id')['away_team_history_DIFF_goal'].cumsum()\n    # df['home_team_hist_goal_csum'] = df.groupby('id')['home_team_history_goal'].cumsum()\n    # df['home_team_hist_opp_goal_csum'] = df.groupby('id')['home_team_history_opponent_goal'].cumsum()\n    # df['away_team_hist_goal_csum'] = df.groupby('id')['away_team_history_goal'].cumsum()\n    # df['away_team_hist_opp_goal_csum'] = df.groupby('id')['away_team_history_opponent_goal'].cumsum()\n    # rating\n    df['home_team_hist_rat_mean'] = df.groupby('id')['home_team_history_rating'].transform('mean')\n    df['away_team_hist_rat_mean'] = df.groupby('id')['away_team_history_rating'].transform('mean')\n    # df['away_team_hist_rat_mean'] = df.groupby('id')['away_team_history_rating'].mean()\n    df['home_away_rat_elo'] = 1/(1+10**((df['away_team_hist_rat_mean']-df['home_team_hist_rat_mean'])/10))\n    # Result (%)\n    df['home_team_result_mean'] = df.groupby('id')['home_team_result'].transform('mean')\n    df['away_team_result_mean'] = df.groupby('id')['away_team_result'].transform('mean')\n    # df['home_team_result_perc'] = df.groupby('id')['home_team_result'].cumsum()\n    # df['away_team_result_perc'] = df.groupby('id')['away_team_result'].cumsum()\n    # df['home_team_result_perc'] = df['home_team_result_perc']/(df['time']*2)\n    # df['away_team_result_perc'] = df['away_team_result_perc']/(df['time']*2)\n    # Lags result if INV = True\n    df['home_team_result_lag1'] = df.groupby('id')['home_team_result'].shift(-1)\n    df['away_team_result_lag1'] = df.groupby('id')['away_team_result'].shift(-1)\n    df['home_team_result_lag2'] = df.groupby('id')['home_team_result'].shift(-2)\n    df['away_team_result_lag2'] = df.groupby('id')['away_team_result'].shift(-2)\n    # dummies results\n    # df = pd.get_dummies(df['home_team_result'])\n    # df = pd.get_dummies(df['away_team_result'])\n    # df['one'] = 1\n    # df['count'] = df.groupby('id').isna()\n    \n    return df\n\n# train_x_pivot = add_features_II(train_x_pivot)\n# test_x_pivot = add_features_II(test_x_pivot)\n","metadata":{"execution":{"iopub.status.busy":"2022-05-04T03:57:07.574486Z","iopub.execute_input":"2022-05-04T03:57:07.574735Z","iopub.status.idle":"2022-05-04T03:57:07.583957Z","shell.execute_reply.started":"2022-05-04T03:57:07.574701Z","shell.execute_reply":"2022-05-04T03:57:07.583298Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# train_x_pivot.head(20)\n# test_x_pivot.head(20)","metadata":{"execution":{"iopub.status.busy":"2022-05-04T03:57:07.585123Z","iopub.execute_input":"2022-05-04T03:57:07.585495Z","iopub.status.idle":"2022-05-04T03:57:07.596259Z","shell.execute_reply.started":"2022-05-04T03:57:07.585450Z","shell.execute_reply":"2022-05-04T03:57:07.595511Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Changing the sequence of time from 1...10 to 10...1 improve the model?\n# bidirectional LSTM is used\n\nINV = True\nif INV:\n    # Trying to keep the same id order\n    train_x_pivot.sort_values(by=['time'], inplace = True, ascending=False)\n    # Merge and drop columns\n    train_x_pivot = pd.merge(train_x['id'], train_x_pivot, on=\"id\").drop(['id', 'time'], axis = 1)\n    # Test\n    test_x_pivot.sort_values(by=['time'], inplace = True, ascending=False)\n    test_x_pivot = pd.merge(test_x['id'], test_x_pivot, on=\"id\").drop(['id', 'time'], axis = 1)\n    \n    # x_test_pivot = x_test_pivot.reset_index()\n    # x_test_pivot['time'] = (T_HIST + 1) - x_test_pivot['time']\n    # x_test_pivot.sort_values(by=['time'], inplace = True)\n    # x_test = pd.merge(test_id, x_test_pivot, on=\"id\").drop(['id', 'time'], axis = 1)\n    # x_test = x_test.to_numpy().reshape(-1, T_HIST, x_test.shape[-1])","metadata":{"execution":{"iopub.status.busy":"2022-05-04T03:57:07.597505Z","iopub.execute_input":"2022-05-04T03:57:07.598279Z","iopub.status.idle":"2022-05-04T03:57:09.833943Z","shell.execute_reply.started":"2022-05-04T03:57:07.598242Z","shell.execute_reply":"2022-05-04T03:57:09.833165Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# import sklearn\n# sklearn.__version__","metadata":{"execution":{"iopub.status.busy":"2022-05-04T03:57:09.835262Z","iopub.execute_input":"2022-05-04T03:57:09.835517Z","iopub.status.idle":"2022-05-04T03:57:09.838379Z","shell.execute_reply.started":"2022-05-04T03:57:09.835483Z","shell.execute_reply":"2022-05-04T03:57:09.837729Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"x_train = train_x_pivot.copy() #drop(['id', 'time'], axis=1)\nx_test = test_x_pivot.copy() #drop(['id', 'time'], axis=1)\n# Fill NA with median\nfill_median = True\nif fill_median:\n    x_train = np.where(np.isnan(x_train), np.nanmedian(x_train, axis=0), x_train)\n    x_test = np.where(np.isnan(x_test), np.nanmedian(x_test, axis=0), x_test)\n\n# Scale features using statistics that are robust to outliers\nRS = RobustScaler(quantile_range=(20, 80), unit_variance=True) # quantile_range=(10, 90)\nx_train = RS.fit_transform(x_train)\nx_test = RS.transform(x_test)\n# Reshape \nx_train = x_train.reshape(-1, T_HIST, x_train.shape[-1])\nx_test = x_test.reshape(-1, T_HIST, x_test.shape[-1])\n\nif False:\n    # Fill NA with MASK\n    x_train = np.nan_to_num(x_train, nan=MASK)\n    x_test = np.nan_to_num(x_test, nan=MASK)\n\n# Back to pandas.dataframe\n# x_train = pd.DataFrame(train, columns=feature_names)\n# x_train = pd.concat([train_id, x_train], axis = 1)\n#\n# x_test = pd.DataFrame(test, columns=feature_names)\n# x_test = pd.concat([test_id, x_test], axis = 1)","metadata":{"execution":{"iopub.status.busy":"2022-05-04T03:57:09.839690Z","iopub.execute_input":"2022-05-04T03:57:09.840086Z","iopub.status.idle":"2022-05-04T03:57:12.475387Z","shell.execute_reply.started":"2022-05-04T03:57:09.840050Z","shell.execute_reply":"2022-05-04T03:57:12.474654Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print(f\"Train array shape: {x_train.shape} \\nTest array shape: {x_test.shape}\")","metadata":{"_uuid":"9adbb500-7f82-41be-aa8f-f19c9291c4e9","_cell_guid":"bedd04b7-cf54-4289-b3c9-b6086f90a580","collapsed":false,"jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2022-05-04T03:57:12.476862Z","iopub.execute_input":"2022-05-04T03:57:12.477141Z","iopub.status.idle":"2022-05-04T03:57:12.482168Z","shell.execute_reply.started":"2022-05-04T03:57:12.477105Z","shell.execute_reply":"2022-05-04T03:57:12.481473Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Deal with targets\n# encode class values as integers\nencoder = LabelEncoder()\nencoder.fit(train_y)\nencoded_y = encoder.transform(train_y)\n# convert integers to dummy variables (i.e. one hot encoded)\ndummy_y = to_categorical(encoded_y)\n# \nprint(encoded_y.shape)\nprint(dummy_y.shape)","metadata":{"_uuid":"129f90f7-7744-488c-9e61-b49bb2c90992","_cell_guid":"40d047c0-ca6f-4bdc-af0f-17e0d1f843f5","collapsed":false,"jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2022-05-04T03:57:12.483569Z","iopub.execute_input":"2022-05-04T03:57:12.484282Z","iopub.status.idle":"2022-05-04T03:57:12.524071Z","shell.execute_reply.started":"2022-05-04T03:57:12.484241Z","shell.execute_reply":"2022-05-04T03:57:12.523197Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# encoding away: 0 draw: 1 home: 2 \nprint(encoded_y[:10,])\n# Order: away, draw, home\nprint(dummy_y[:10,])","metadata":{"_uuid":"f66ba6cb-16e0-4560-a345-e5a38b5ed82c","_cell_guid":"6a9dcfac-a3a7-40a2-b7c1-c515319b7e0c","collapsed":false,"jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2022-05-04T03:57:12.525324Z","iopub.execute_input":"2022-05-04T03:57:12.525634Z","iopub.status.idle":"2022-05-04T03:57:12.533605Z","shell.execute_reply.started":"2022-05-04T03:57:12.525597Z","shell.execute_reply":"2022-05-04T03:57:12.532829Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Setup GPU\nBelow declare whether to use 1 GPU or multiple GPU. (Change CUDA_VISIBLE_DEVICES to use more GPUs). I am not aware of the impact on speed.","metadata":{}},{"cell_type":"code","source":"import os\nos.environ[\"CUDA_VISIBLE_DEVICES\"]=\"0\"\n\n# USE MULTIPLE GPUS\nif os.environ[\"CUDA_VISIBLE_DEVICES\"].count(',') == 0:\n    gpu_strategy = tf.distribute.get_strategy()\n    print('single strategy')\nelse:\n    gpu_strategy = tf.distribute.MirroredStrategy()\n    print('multiple strategy')","metadata":{"execution":{"iopub.status.busy":"2022-05-04T03:57:12.535266Z","iopub.execute_input":"2022-05-04T03:57:12.536140Z","iopub.status.idle":"2022-05-04T03:57:12.550537Z","shell.execute_reply.started":"2022-05-04T03:57:12.536101Z","shell.execute_reply":"2022-05-04T03:57:12.549835Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Make a model\nThe model is a very simple LSTM. Why LSTM? Because we have the time history of the football team. I'm not sure whether it has any influence on the result at least for my football team, losing and winning at random. \nI would try to make a XGBOOST model with aggregated time features and compare the results.","metadata":{"_uuid":"f0fd77a2-0d9c-4fba-ac6d-f879a553bf4c","_cell_guid":"51235a13-fea9-47fd-a2b9-caeccbd0b67f","trusted":true}},{"cell_type":"code","source":"# This is similar to the model of \"igorkf\" (Good, loss ~ 0.999)\ndef model_2():\n    x_input = layers.Input(shape=x_train.shape[1:])\n    # x = layers.Masking(mask_value=MASK, input_shape=(x_train.shape[1:]))(x_input)\n    x = layers.Bidirectional(tf.compat.v1.keras.layers.CuDNNLSTM(16, return_sequences=True))(x_input) #(x)\n    x = layers.MaxPooling1D(pool_size=2)(x)\n#     x = layers.BatchNormalization()(x)\n    x = layers.Dropout(0.5)(x)  \n    x = layers.Bidirectional(tf.compat.v1.keras.layers.CuDNNLSTM(8, return_sequences=True))(x)\n    x = layers.MaxPooling1D(pool_size=2)(x)\n#     x = layers.BatchNormalization()(x)\n    x = layers.Dropout(0.5)(x)\n    x = layers.Flatten()(x)\n    # output\n    output = layers.Dense(CLASS, activation='softmax')(x)\n    model = Model(inputs=[x_input],outputs=[output])\n\n    return model","metadata":{"_uuid":"8f733748-8539-4dff-9db7-88edd1e41bdf","_cell_guid":"da442908-1692-4121-b108-ebd4141c4d4f","collapsed":false,"jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2022-05-04T03:57:12.551747Z","iopub.execute_input":"2022-05-04T03:57:12.552243Z","iopub.status.idle":"2022-05-04T03:57:12.560487Z","shell.execute_reply.started":"2022-05-04T03:57:12.552205Z","shell.execute_reply":"2022-05-04T03:57:12.559683Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# x_train.shape","metadata":{"execution":{"iopub.status.busy":"2022-05-04T03:57:12.561800Z","iopub.execute_input":"2022-05-04T03:57:12.562301Z","iopub.status.idle":"2022-05-04T03:57:12.572129Z","shell.execute_reply.started":"2022-05-04T03:57:12.562262Z","shell.execute_reply":"2022-05-04T03:57:12.571389Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Choose your model\nmodel = model_2()\nmodel.summary()","metadata":{"_uuid":"43445b36-d540-4621-b921-efbff25a5a48","_cell_guid":"d48d09c0-c867-4d6c-9edb-9174b243cf4f","collapsed":false,"jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2022-05-04T03:57:12.573321Z","iopub.execute_input":"2022-05-04T03:57:12.573761Z","iopub.status.idle":"2022-05-04T03:57:15.643283Z","shell.execute_reply.started":"2022-05-04T03:57:12.573724Z","shell.execute_reply":"2022-05-04T03:57:15.642450Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# plot_model(\n#     model, \n#     to_file='Football_Prob_Model.png', \n#     show_shapes=True,\n#     show_layer_names=True\n# )","metadata":{"_uuid":"8a324a9c-628d-4173-b87f-c425a194d954","_cell_guid":"efb6e5f5-4590-4ea3-bf8b-afe761843ee4","collapsed":false,"jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2022-05-04T03:57:15.644582Z","iopub.execute_input":"2022-05-04T03:57:15.644842Z","iopub.status.idle":"2022-05-04T03:57:15.651024Z","shell.execute_reply.started":"2022-05-04T03:57:15.644807Z","shell.execute_reply":"2022-05-04T03:57:15.649746Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"sample_weight = (train.match_date.astype(np.int64) / train.match_date.astype(np.int64).max()).values","metadata":{"execution":{"iopub.status.busy":"2022-05-04T03:57:15.652099Z","iopub.execute_input":"2022-05-04T03:57:15.652489Z","iopub.status.idle":"2022-05-04T03:57:15.721137Z","shell.execute_reply.started":"2022-05-04T03:57:15.652427Z","shell.execute_reply":"2022-05-04T03:57:15.720433Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"sample_weight = np.array([i ** 3 for i in sample_weight])","metadata":{"execution":{"iopub.status.busy":"2022-05-04T03:57:15.722307Z","iopub.execute_input":"2022-05-04T03:57:15.722796Z","iopub.status.idle":"2022-05-04T03:57:15.789595Z","shell.execute_reply.started":"2022-05-04T03:57:15.722737Z","shell.execute_reply":"2022-05-04T03:57:15.788623Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"len(sample_weight)","metadata":{"execution":{"iopub.status.busy":"2022-05-04T03:57:15.791822Z","iopub.execute_input":"2022-05-04T03:57:15.792331Z","iopub.status.idle":"2022-05-04T03:57:15.798565Z","shell.execute_reply.started":"2022-05-04T03:57:15.792290Z","shell.execute_reply":"2022-05-04T03:57:15.797732Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Fit the model and a make submission\nOnce the model is fitted we make the **probabilities prediction**. Then we make the submission dataframe with **4 columns**. The columns home, away and draw contain probability while the column id contains the match id.","metadata":{"_uuid":"ea635cb3-5380-4259-ad5f-fd94e9606f26","_cell_guid":"0c123082-ce45-4f81-841d-eb90adea1783","trusted":true}},{"cell_type":"code","source":" # Just one fold, validation split = 20%\nif False:\n    EPOCH = 200\n    BATCH_SIZE = 512 \n    # N_SPLITS = 5\n    SEED = 123\n    VERBOSE = 1\n    PATIENCE = EPOCH // 10\n    VAL_SPLIT = 0.2\n    # Model\n    # It is a multi-class classification problem, categorical_crossentropy is used as the loss function.\n    model.compile(optimizer=\"adam\", loss=\"categorical_crossentropy\",\n                     metrics=[\"accuracy\"])\n    #\n    es = EarlyStopping(monitor='val_loss', patience=PATIENCE, verbose=0, mode='min',\n                           restore_best_weights=True)\n    lr = ReduceLROnPlateau(monitor=\"val_loss\", factor=0.5, patience=10, verbose=0)\n    #\n    model.fit(x_train, dummy_y, \n                  # validation_data=(x_train, y_train), \n                  validation_split=VAL_SPLIT,\n                  epochs=EPOCH,\n                  verbose=VERBOSE,\n                  batch_size=BATCH_SIZE,\n                  callbacks=[lr, es])\n","metadata":{"_uuid":"e20e5330-09de-4723-b0c4-6a1b07ab6927","_cell_guid":"19806374-e259-453a-b033-e80a15e0f03e","collapsed":false,"jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2022-05-04T03:57:15.800229Z","iopub.execute_input":"2022-05-04T03:57:15.800552Z","iopub.status.idle":"2022-05-04T03:57:15.809165Z","shell.execute_reply.started":"2022-05-04T03:57:15.800513Z","shell.execute_reply":"2022-05-04T03:57:15.808411Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# N_SPLITS of the traning set for validation using KFold\n# Parameters\nEPOCH = 200\nBATCH_SIZE = 512\nN_SPLITS = 20\nSEED = 123\nVERBOSE = 1\nPATIENCE = EPOCH // 10\n\ntest_preds = []\n\nwith gpu_strategy.scope():\n    kf = KFold(n_splits=N_SPLITS) # shuffle=True , random_state=SEED\n\n    for fold, (train_idx, test_idx) in enumerate(kf.split(x_train, dummy_y)):\n        print('-'*15, '>', f'Fold {fold+1}/{N_SPLITS}', '<', '-'*15)\n        X_train, X_valid = x_train[train_idx], x_train[test_idx]\n        Y_train, Y_valid = dummy_y[train_idx], dummy_y[test_idx]\n        w_train, w_valid = sample_weight[train_idx], sample_weight[test_idx]\n        \n        ######### Model: CHANGE HERE TOO ################\n        model = model_2()\n        # It is a multi-class classification problem, categorical_crossentropy is used as the loss function.\n        model.compile(optimizer=\"adam\", loss=\"categorical_crossentropy\",\n                     metrics=[\"accuracy\"])\n        #\n        es = EarlyStopping(monitor='val_loss', patience=PATIENCE, verbose=0, mode='min',\n                           restore_best_weights=True)\n        \n        lr = ReduceLROnPlateau(monitor=\"val_loss\", factor=0.5, patience=10, verbose=0)\n        #\n        model.fit(X_train, Y_train, \n#                   sample_weight=w_train,\n                  validation_data=(X_valid, Y_valid), \n                  epochs=EPOCH,\n                  verbose=VERBOSE,\n                  batch_size=BATCH_SIZE,  \n                  callbacks=[lr, es])\n        \n        # Model validation    \n        y_true = Y_valid.squeeze()\n        y_pred = model.predict(X_valid, batch_size=BATCH_SIZE).squeeze()\n        score1 = log_loss(y_true, y_pred)\n        print(f\"Fold-{fold+1} | OOF LogLoss Score: {score1}\")\n        # Predictions\n        test_preds.append(model.predict(x_test).squeeze())\n        # test_preds.append(model.predict(x_test, batch_size=BATCH_SIZE).squeeze().reshape(-1, 1).squeeze())","metadata":{"execution":{"iopub.status.busy":"2022-05-04T03:58:53.564674Z","iopub.execute_input":"2022-05-04T03:58:53.565242Z","iopub.status.idle":"2022-05-04T04:12:08.778249Z","shell.execute_reply.started":"2022-05-04T03:58:53.565199Z","shell.execute_reply":"2022-05-04T04:12:08.777484Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"predictions = sum(test_preds)/N_SPLITS \n\n# away, draw, home\nsubmission = pd.DataFrame(predictions,columns=['away', 'draw', 'home'])\n\n# Round\nround_num = False\nif round_num:\n    submission = submission.round(2)\n    submission['draw'] = 1 - (submission['home'] + submission['away'])  \n    \n#do not forget the id column\nsubmission['id'] = test[['id']]\n\n#submit!\nsubmission[['id', 'home', 'away', 'draw']].to_csv('submission.csv', index=False)","metadata":{"_uuid":"fbc22907-d44b-4d98-a63e-7308371f2575","_cell_guid":"daf73b19-9b2b-426f-a75a-eb1842f15261","collapsed":false,"jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2022-05-04T03:57:15.847879Z","iopub.status.idle":"2022-05-04T03:57:15.848323Z","shell.execute_reply.started":"2022-05-04T03:57:15.848089Z","shell.execute_reply":"2022-05-04T03:57:15.848113Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"submission[['id', 'home', 'away', 'draw']].head()","metadata":{"_uuid":"a5aeb080-c652-4e2c-9476-e662c51ff731","_cell_guid":"a7b164c3-f7d6-47a8-b6a6-b84ebb382b4d","collapsed":false,"jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2022-05-04T03:57:15.849876Z","iopub.status.idle":"2022-05-04T03:57:15.850305Z","shell.execute_reply.started":"2022-05-04T03:57:15.850065Z","shell.execute_reply":"2022-05-04T03:57:15.850087Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Conclusion\n**Good luck!**\n\nThe best accuracy of the validation set was 0.5015 so far, it is pretty low. You won't make money using this model.\n\nA huge LSTM makes worse predictions than a small LSTM. Need to work on this!\n\nReport any error that you will probably find.","metadata":{"_uuid":"002cd78f-9689-47ca-a8fe-2e28eba99b1d","_cell_guid":"4a755058-46c1-4ab7-9693-2381814c1bc0","trusted":true}}]}