{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"code","source":"# This Python 3 environment comes with many helpful analytics libraries installed\n# It is defined by the kaggle/python Docker image: https://github.com/kaggle/docker-python\n# For example, here's several helpful packages to load\n\nimport numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\nfrom datetime import datetime # Convert date and time into a timestep\n\n# Input data files are available in the read-only \"../input/\" directory\n# For example, running this (by clicking run or pressing Shift+Enter) will list all files under the input directory\n\nimport os\nfor dirname, _, filenames in os.walk('/kaggle/input'):\n    for filename in filenames:\n        print(os.path.join(dirname, filename))\n\n# You can write up to 20GB to the current directory (/kaggle/working/) that gets preserved as output when you create a version using \"Save & Run All\" \n# You can also write temporary files to /kaggle/temp/, but they won't be saved outside of the current session","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","execution":{"iopub.status.busy":"2021-11-08T05:21:16.498520Z","iopub.execute_input":"2021-11-08T05:21:16.499176Z","iopub.status.idle":"2021-11-08T05:21:16.514807Z","shell.execute_reply.started":"2021-11-08T05:21:16.499150Z","shell.execute_reply":"2021-11-08T05:21:16.513591Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Loading Data","metadata":{}},{"cell_type":"code","source":"data_file = \"../input/g-research-crypto-forecasting/\"\n!ls $data_file","metadata":{"execution":{"iopub.status.busy":"2021-11-08T05:21:16.516551Z","iopub.execute_input":"2021-11-08T05:21:16.517056Z","iopub.status.idle":"2021-11-08T05:21:16.816822Z","shell.execute_reply.started":"2021-11-08T05:21:16.517030Z","shell.execute_reply":"2021-11-08T05:21:16.816030Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Stored In Variable","metadata":{}},{"cell_type":"code","source":"cf_df = pd.read_csv(data_file + 'train.csv')","metadata":{"execution":{"iopub.status.busy":"2021-11-08T05:21:16.818189Z","iopub.execute_input":"2021-11-08T05:21:16.818396Z","iopub.status.idle":"2021-11-08T05:21:42.568113Z","shell.execute_reply.started":"2021-11-08T05:21:16.818373Z","shell.execute_reply":"2021-11-08T05:21:42.567242Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"cf_df.head(8)","metadata":{"execution":{"iopub.status.busy":"2021-11-08T05:21:42.569511Z","iopub.execute_input":"2021-11-08T05:21:42.569800Z","iopub.status.idle":"2021-11-08T05:21:42.591935Z","shell.execute_reply.started":"2021-11-08T05:21:42.569760Z","shell.execute_reply":"2021-11-08T05:21:42.591067Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Data features\n-  **timestamp**: All timestamps are returned as second Unix timestamps (the number of seconds elapsed since 1970-01-01 00:00:00.000 UTC). Timestamps in this dataset are multiple of 60, indicating minute-by-minute data.","metadata":{}},{"cell_type":"code","source":"asset_details = pd.read_csv(data_file + 'asset_details.csv')\nasset_details","metadata":{"execution":{"iopub.status.busy":"2021-11-08T05:21:42.593980Z","iopub.execute_input":"2021-11-08T05:21:42.594450Z","iopub.status.idle":"2021-11-08T05:21:42.610493Z","shell.execute_reply.started":"2021-11-08T05:21:42.594420Z","shell.execute_reply":"2021-11-08T05:21:42.609875Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"btcoin = cf_df[cf_df[\"Asset_ID\"]==1].set_index(\"timestamp\") # Asset_ID = 1 for Bitcoin\nbtcoin_min = btcoin.iloc[-200:] # Select recent data rows","metadata":{"execution":{"iopub.status.busy":"2021-11-08T05:21:42.612011Z","iopub.execute_input":"2021-11-08T05:21:42.612411Z","iopub.status.idle":"2021-11-08T05:21:42.854460Z","shell.execute_reply.started":"2021-11-08T05:21:42.612375Z","shell.execute_reply":"2021-11-08T05:21:42.853669Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import plotly.graph_objects as go\n\nfig = go.Figure(data=[go.Candlestick(x=btcoin_min.index, open=btcoin_min['Open'], high=btcoin_min['High'], low=btcoin_min['Low'], close=btcoin_min['Close'])])\nfig.show()","metadata":{"execution":{"iopub.status.busy":"2021-11-08T05:21:42.855653Z","iopub.execute_input":"2021-11-08T05:21:42.855935Z","iopub.status.idle":"2021-11-08T05:21:42.871040Z","shell.execute_reply.started":"2021-11-08T05:21:42.855899Z","shell.execute_reply":"2021-11-08T05:21:42.869649Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Preprocessing \n- ","metadata":{}},{"cell_type":"markdown","source":"## Dealing with missing data ","metadata":{}},{"cell_type":"code","source":"miss = cf_df[cf_df[\"Asset_ID\"]==6].set_index(\"timestamp\") # Asset_ID = 6 for Ethereum\nmiss.info(show_counts =True)","metadata":{"execution":{"iopub.status.busy":"2021-11-08T05:21:42.872410Z","iopub.execute_input":"2021-11-08T05:21:42.872972Z","iopub.status.idle":"2021-11-08T05:21:43.092798Z","shell.execute_reply.started":"2021-11-08T05:21:42.872940Z","shell.execute_reply":"2021-11-08T05:21:43.091823Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"miss.isna().sum()","metadata":{"execution":{"iopub.status.busy":"2021-11-08T05:21:43.093702Z","iopub.execute_input":"2021-11-08T05:21:43.093868Z","iopub.status.idle":"2021-11-08T05:21:43.123668Z","shell.execute_reply.started":"2021-11-08T05:21:43.093845Z","shell.execute_reply":"2021-11-08T05:21:43.122647Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"btcoin.head()","metadata":{"execution":{"iopub.status.busy":"2021-11-08T05:21:43.125036Z","iopub.execute_input":"2021-11-08T05:21:43.125228Z","iopub.status.idle":"2021-11-08T05:21:43.141642Z","shell.execute_reply.started":"2021-11-08T05:21:43.125201Z","shell.execute_reply":"2021-11-08T05:21:43.140927Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"leg_btcoin = btcoin.index[0].astype('datetime64[s]')\nend_btcoin = btcoin.index[-1].astype('datetime64[s]')\nleg_miss = miss.index[0].astype('datetime64[s]')\nend_miss = miss.index[-1].astype('datetime64[s]')\n\nprint('BTCOIN data goes from ', leg_btcoin, 'to ', end_btcoin)\nprint('Ethereum data goes from ', leg_miss, 'to ', end_miss)","metadata":{"execution":{"iopub.status.busy":"2021-11-08T05:21:43.143160Z","iopub.execute_input":"2021-11-08T05:21:43.143505Z","iopub.status.idle":"2021-11-08T05:21:43.151261Z","shell.execute_reply.started":"2021-11-08T05:21:43.143473Z","shell.execute_reply":"2021-11-08T05:21:43.150734Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"- Missing asset data, for a given minute, is not represented by NaN's, but instead by the absence of those rows. We can check the timestamp difference between consecutive rows to see if there is missing data.","metadata":{}},{"cell_type":"code","source":"(miss.index[1:]-miss.index[:-1]).value_counts().head()","metadata":{"execution":{"iopub.status.busy":"2021-11-08T05:21:43.152264Z","iopub.execute_input":"2021-11-08T05:21:43.152481Z","iopub.status.idle":"2021-11-08T05:21:43.192002Z","shell.execute_reply.started":"2021-11-08T05:21:43.152451Z","shell.execute_reply":"2021-11-08T05:21:43.190821Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"- Notice that there are many gaps in the data. To work with most time series models, we should preprocess our data into a format without time gaps. To fill the gaps, we can use the `.reindex()` method for forward filling, filling gaps with the previous valid value. \n","metadata":{}},{"cell_type":"code","source":"miss = miss.reindex(range(miss.index[0],miss.index[-1]+60,60),method='pad')","metadata":{"execution":{"iopub.status.busy":"2021-11-08T05:21:43.193413Z","iopub.execute_input":"2021-11-08T05:21:43.193629Z","iopub.status.idle":"2021-11-08T05:21:43.554664Z","shell.execute_reply.started":"2021-11-08T05:21:43.193600Z","shell.execute_reply":"2021-11-08T05:21:43.553505Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"(miss.index[1:]-miss.index[:-1]).value_counts().head()","metadata":{"execution":{"iopub.status.busy":"2021-11-08T05:21:43.558826Z","iopub.execute_input":"2021-11-08T05:21:43.559055Z","iopub.status.idle":"2021-11-08T05:21:43.589577Z","shell.execute_reply.started":"2021-11-08T05:21:43.559028Z","shell.execute_reply":"2021-11-08T05:21:43.588981Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Data visualisation\n- We  will start by visualising the Close prices for the two assets we have selected.","metadata":{}},{"cell_type":"code","source":"import matplotlib.pyplot as plt\n\n# plot vwap time series for both chosen assets\nf = plt.figure(figsize=(15,4))\n\n# fill missing values for BTCOIN\nbtcoin = btcoin.reindex(range(btcoin.index[0],btcoin.index[-1]+60,60),method='pad')\n\nax = f.add_subplot(121)\nplt.plot(btcoin['Close'], label='BTCOIN')\nplt.legend()\nplt.xlabel('Time')\nplt.ylabel('Bitcoin')\n\nax2 = f.add_subplot(122)\nax2.plot(miss['Close'], color='red', label='ETH')\nplt.legend()\nplt.xlabel('Time')\nplt.ylabel('Ethereum') # Ethereum is a blockchain computer program similar to Bitcoin. It can be used to create automated contracts or circulate a digital currency called Ether.\n\nplt.tight_layout()\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2021-11-08T05:21:43.590758Z","iopub.execute_input":"2021-11-08T05:21:43.591003Z","iopub.status.idle":"2021-11-08T05:21:46.829919Z","shell.execute_reply.started":"2021-11-08T05:21:43.590969Z","shell.execute_reply":"2021-11-08T05:21:46.828590Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import time\n\n# auxiliary function, from datetime to timestamp\ntotimestamp = lambda s: np.int32(time.mktime(datetime.strptime(s, \"%d/%m/%Y\").timetuple()))\n\n# create intervals\nbtcoin_min_2021 = btcoin.loc[totimestamp('01/06/2021'):totimestamp('01/07/2021')]\nmiss_min_2021 = miss.loc[totimestamp('01/06/2021'):totimestamp('01/07/2021')]","metadata":{"execution":{"iopub.status.busy":"2021-11-08T05:21:46.832138Z","iopub.execute_input":"2021-11-08T05:21:46.832426Z","iopub.status.idle":"2021-11-08T05:21:46.839644Z","shell.execute_reply.started":"2021-11-08T05:21:46.832395Z","shell.execute_reply":"2021-11-08T05:21:46.838053Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# plot time series for both chosen assets\nf = plt.figure(figsize=(7,8))\n\nax = f.add_subplot(211)\nplt.plot(btcoin_min_2021['Close'], label='btc')\nplt.legend()\nplt.xlabel('Time')\nplt.ylabel('Bitcoin Close')\n\nax2 = f.add_subplot(212)\nax2.plot(miss_min_2021['Close'], color='red', label='eth')\nplt.legend()\nplt.xlabel('Time')\nplt.ylabel('Ethereum Close') \n\nplt.tight_layout()\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2021-11-08T05:21:46.841127Z","iopub.execute_input":"2021-11-08T05:21:46.841351Z","iopub.status.idle":"2021-11-08T05:21:47.324272Z","shell.execute_reply.started":"2021-11-08T05:21:46.841321Z","shell.execute_reply":"2021-11-08T05:21:47.323636Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"- On shorter intervals we can visually see some potential correlation between both assets, with some simultaneous ups and downs. A better format for analyzing such movements is by calculating asset returns. \n","metadata":{}},{"cell_type":"markdown","source":"## Log returns","metadata":{}},{"cell_type":"markdown","source":"## In order to analyze price changes for an asset we can deal with the price difference. However, different assets exhibit different price scales, so that the their returns are not readily comparable. We can solve this problem by computing the percentage change in price instead, also known as the return. This return coincides with the percentage change in our invested capital.\n\n## Returns are widely used in finance, however log returns are preferred for mathematical modelling of time series, as they are additive across time. Also, while regular returns cannot go below -100%, log returns are not bounded.\n\n## To compute the log return, we can simply take the logarithm of the ratio between two consecutive prices. The first row will have an empty return as the previous value is unknown, therefore the empty return data point will be dropped.","metadata":{}},{"cell_type":"code","source":"# define function to compute log returns\ndef log_return(series, periods=1):\n    return np.log(series).diff(periods=periods)","metadata":{"execution":{"iopub.status.busy":"2021-11-08T05:21:47.325509Z","iopub.execute_input":"2021-11-08T05:21:47.325799Z","iopub.status.idle":"2021-11-08T05:21:47.331798Z","shell.execute_reply.started":"2021-11-08T05:21:47.325762Z","shell.execute_reply":"2021-11-08T05:21:47.330603Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## We can visualize the log return for our two assets. See how the signal now looks more like white noise, with less drift than the time series for prices.\n","metadata":{}},{"cell_type":"code","source":"import scipy.stats as stats\n\nlret_btc = log_return(btcoin_min_2021.Close)[1:]\nlret_eth = log_return(miss_min_2021.Close)[1:]\nlret_btc.rename('lret_btc', inplace=True)\nlret_eth.rename('lret_eth', inplace=True)\n\nplt.figure(figsize=(8,4))\nplt.plot(lret_btc);\nplt.plot(lret_eth);\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2021-11-08T05:21:47.333381Z","iopub.execute_input":"2021-11-08T05:21:47.335903Z","iopub.status.idle":"2021-11-08T05:21:47.575048Z","shell.execute_reply.started":"2021-11-08T05:21:47.335851Z","shell.execute_reply":"2021-11-08T05:21:47.574496Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Correlation between assets\n\n- We hypothesized before that crypto asset returns may exhibit some correlation. Let's check this in more detail now.\n- We can check how the correlation between Bitcoin and Ethereum change over time for the 2021 period we selected. \n\n\n\n","metadata":{}},{"cell_type":"code","source":"# join two asset in single DataFrame\n\nlret_btc_long = log_return(btcoin.Close)[1:]\nlret_eth_long = log_return(miss.Close)[1:]\nlret_btc_long.rename('lret_btc', inplace=True)\nlret_eth_long.rename('lret_eth', inplace=True)\ntwo_assets = pd.concat([lret_btc_long, lret_eth_long], axis=1)\n\n# group consecutive rows and use .corr() for correlation between columns\ncorr_time = two_assets.groupby(two_assets.index//(10000*60)).corr().loc[:,\"lret_btc\"].loc[:,\"lret_eth\"]\n\ncorr_time.plot();\nplt.xticks([])\nplt.ylabel(\"Correlation\")\nplt.title(\"Correlation between BTC and ETH over time\");","metadata":{"execution":{"iopub.status.busy":"2021-11-08T05:21:47.576269Z","iopub.execute_input":"2021-11-08T05:21:47.576620Z","iopub.status.idle":"2021-11-08T05:21:48.145866Z","shell.execute_reply.started":"2021-11-08T05:21:47.576584Z","shell.execute_reply":"2021-11-08T05:21:48.144777Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Note the high but variable correlation between the assets. Here we can see that there is some changing dynamics over time, and this would be critical for this time series challenge, that is, how to perform forecasts in a highly non-stationary environment.\n","metadata":{}},{"cell_type":"markdown","source":"## A stationary behaviour of a system or a process is characterized by non-changing statistical properties over time such as the mean, variance and autocorrelation. On the other hand, a non-stationary behaviour is characterized by a continuous change of statistical properties over time. Stationarity is important because many useful analytical tools and statistical tests and models rely on it.\n","metadata":{}},{"cell_type":"markdown","source":"## We can also check the correlation between all assets visualizing the correlation matrix. Note how some assets have much higher pairwise correlation than others.","metadata":{}},{"cell_type":"code","source":"# create dataframe with returns for all assets\nall_assets_2021 = pd.DataFrame([])\nfor asset_id, asset_name in zip(asset_details.Asset_ID, asset_details.Asset_Name):\n  asset = cf_df[cf_df[\"Asset_ID\"]==asset_id].set_index(\"timestamp\")\n  asset = asset.loc[totimestamp('01/01/2021'):totimestamp('01/05/2021')]\n  asset = asset.reindex(range(asset.index[0],asset.index[-1]+60,60),method='pad')\n  lret = log_return(asset.Close.fillna(0))[1:]\n  all_assets_2021 = all_assets_2021.join(lret, rsuffix=asset_name, how=\"outer\")","metadata":{"execution":{"iopub.status.busy":"2021-11-08T05:21:48.147278Z","iopub.execute_input":"2021-11-08T05:21:48.147917Z","iopub.status.idle":"2021-11-08T05:21:50.974266Z","shell.execute_reply.started":"2021-11-08T05:21:48.147872Z","shell.execute_reply":"2021-11-08T05:21:50.973171Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plt.imshow(all_assets_2021.corr());\nplt.yticks(asset_details.Asset_ID.values, asset_details.Asset_Name.values);\nplt.xticks(asset_details.Asset_ID.values, asset_details.Asset_Name.values, rotation='vertical');\nplt.colorbar();","metadata":{"execution":{"iopub.status.busy":"2021-11-08T05:21:50.975887Z","iopub.execute_input":"2021-11-08T05:21:50.976644Z","iopub.status.idle":"2021-11-08T05:21:51.429347Z","shell.execute_reply.started":"2021-11-08T05:21:50.976575Z","shell.execute_reply":"2021-11-08T05:21:51.428561Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"- We encourage participants to perform additional statistical analyses to have a stronger grasp on the dataset, including autocorrelation, time-series decomposition and stationarity tests.\n","metadata":{}},{"cell_type":"markdown","source":"# Building your prediction model","metadata":{}},{"cell_type":"code","source":"# Select some input features from the trading data: \n# 5 min log return, abs(5 min log return), upper shadow, and lower shadow.\nupper_shadow = lambda asset: asset.High - np.maximum(asset.Close,asset.Open)\nlower_shadow = lambda asset: np.minimum(asset.Close,asset.Open)- asset.Low\n\nX_btc = pd.concat([log_return(btcoin.VWAP,periods=5), log_return(btcoin.VWAP,periods=1).abs(), \n               upper_shadow(btcoin), lower_shadow(btcoin)], axis=1)\ny_btc = btcoin.Target\n\nX_eth = pd.concat([log_return(miss.VWAP,periods=5), log_return(miss.VWAP,periods=1).abs(), \n               upper_shadow(miss), lower_shadow(miss)], axis=1)\ny_eth = miss.Target","metadata":{"execution":{"iopub.status.busy":"2021-11-08T05:21:51.430516Z","iopub.execute_input":"2021-11-08T05:21:51.430761Z","iopub.status.idle":"2021-11-08T05:21:51.807761Z","shell.execute_reply.started":"2021-11-08T05:21:51.430732Z","shell.execute_reply":"2021-11-08T05:21:51.806825Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# select training and test periods\ntrain_window = [totimestamp(\"01/05/2021\"), totimestamp(\"30/05/2021\")]\ntest_window = [totimestamp(\"01/06/2021\"), totimestamp(\"30/06/2021\")]\n\n# divide data into train and test, compute X and y\n# we aim to build simple regression models using a window_size of 1\nX_btc_train = X_btc.loc[train_window[0]:train_window[1]].fillna(0).to_numpy()  # filling NaN's with zeros\ny_btc_train = y_btc.loc[train_window[0]:train_window[1]].fillna(0).to_numpy()  \n\nX_btc_test = X_btc.loc[test_window[0]:test_window[1]].fillna(0).to_numpy() \ny_btc_test = y_btc.loc[test_window[0]:test_window[1]].fillna(0).to_numpy() \n\nX_eth_train = X_eth.loc[train_window[0]:train_window[1]].fillna(0).to_numpy()  \ny_eth_train = y_eth.loc[train_window[0]:train_window[1]].fillna(0).to_numpy()  \n\nX_eth_test = X_eth.loc[test_window[0]:test_window[1]].fillna(0).to_numpy() \ny_eth_test = y_eth.loc[test_window[0]:test_window[1]].fillna(0).to_numpy() ","metadata":{"execution":{"iopub.status.busy":"2021-11-08T05:21:51.809033Z","iopub.execute_input":"2021-11-08T05:21:51.809219Z","iopub.status.idle":"2021-11-08T05:21:51.824398Z","shell.execute_reply.started":"2021-11-08T05:21:51.809195Z","shell.execute_reply":"2021-11-08T05:21:51.823815Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## We now standardize the input data. Standardization is the process of putting different variables on the same scale. In regression analysis, it is often crucial to standardize your independent variables or you may risk obtaining misleading results.\n","metadata":{}},{"cell_type":"code","source":"from sklearn.preprocessing import StandardScaler\n# simple preprocessing of the data \nscaler = StandardScaler()\n\nX_btc_train_scaled = scaler.fit_transform(X_btc_train)\nX_btc_test_scaled = scaler.transform(X_btc_test)\n\nX_eth_train_scaled = scaler.fit_transform(X_eth_train)\nX_eth_test_scaled = scaler.transform(X_eth_test)","metadata":{"execution":{"iopub.status.busy":"2021-11-08T05:21:51.825588Z","iopub.execute_input":"2021-11-08T05:21:51.825853Z","iopub.status.idle":"2021-11-08T05:21:51.838520Z","shell.execute_reply.started":"2021-11-08T05:21:51.825821Z","shell.execute_reply":"2021-11-08T05:21:51.837704Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Baseline model: Linear Regression\n- We will try a simple Linear Regression model on the features we designed. Note that Linear Regression is not commonly used in time series analysis, specially with only one time step! \n\n- We compare two Linear Regression baselines, one that considers each asset independently and one multiple inputs that models all assets together.\n","metadata":{}},{"cell_type":"code","source":"from sklearn.linear_model import LinearRegression\n\n# implement basic ML baseline (one per asset)\nlr = LinearRegression()\nlr.fit(X_btc_train_scaled,y_btc_train)\ny_pred_lr_btc = lr.predict(X_btc_test_scaled)\n\nlr.fit(X_eth_train_scaled,y_eth_train)\ny_pred_lr_eth = lr.predict(X_eth_test_scaled)","metadata":{"execution":{"iopub.status.busy":"2021-11-08T05:21:51.839811Z","iopub.execute_input":"2021-11-08T05:21:51.839992Z","iopub.status.idle":"2021-11-08T05:21:51.861707Z","shell.execute_reply.started":"2021-11-08T05:21:51.839970Z","shell.execute_reply":"2021-11-08T05:21:51.860553Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# implement more complex baseline (multiple output regression model)\nfrom sklearn.multioutput import MultiOutputRegressor\n\n# we concatenate X and y for both assets\nX_both_train = np.concatenate((X_btc_train_scaled, X_eth_train_scaled), axis=1)\nX_both_test = np.concatenate((X_btc_test_scaled, X_eth_test_scaled), axis=1)\ny_both_train = np.column_stack((y_btc_train, y_eth_train))\ny_both_test = np.column_stack((y_btc_test, y_eth_test))\n\n# define the direct multioutput model and fit it\nmlr = MultiOutputRegressor(LinearRegression())\nlr.fit(X_both_train,y_both_train)\ny_pred_lr_both = lr.predict(X_both_test)","metadata":{"execution":{"iopub.status.busy":"2021-11-08T05:21:51.863298Z","iopub.execute_input":"2021-11-08T05:21:51.863529Z","iopub.status.idle":"2021-11-08T05:21:51.881011Z","shell.execute_reply.started":"2021-11-08T05:21:51.863505Z","shell.execute_reply":"2021-11-08T05:21:51.880119Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Evaluate baselines\n- The competition performance metric is weighted correlation. However, for now we will use simple correlation to evaluate the two baseline models built.","metadata":{}},{"cell_type":"code","source":"print('Test score for LR baseline: BITCOIN', f\"{np.corrcoef(y_pred_lr_btc, y_btc_test)[0,1]:.2f}\", \n                                ', ETHEREUM', f\"{np.corrcoef(y_pred_lr_eth, y_eth_test)[0,1]:.2f}\")\nprint('Test score for multiple output LR baseline: BITCOIN', f\"{np.corrcoef(y_pred_lr_both[:,0], y_btc_test)[0,1]:.2f}\", \n                                                ', ETHEREUM', f\"{np.corrcoef(y_pred_lr_both[:,1], y_eth_test)[0,1]:.2f}\")","metadata":{"execution":{"iopub.status.busy":"2021-11-08T05:21:51.882269Z","iopub.execute_input":"2021-11-08T05:21:51.882478Z","iopub.status.idle":"2021-11-08T05:21:51.892830Z","shell.execute_reply.started":"2021-11-08T05:21:51.882451Z","shell.execute_reply":"2021-11-08T05:21:51.892122Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## We can see that, for the training and test periods selected, the multiple asset LR model performs better than simply modelling each asset separately. Note that because the data is highly non-stationary, these results might vary a lot for different periods.\n","metadata":{}},{"cell_type":"markdown","source":"## Submission \n\nNote that this is a Code Competition, in which you must submit your notebook to be run against the hidden private data. Your notebook should use the provided python time-series API, which ensures that models do not peek forward in time. To use the API, follow the instructions and template in [Code Competition Detailed API instructions](https://www.kaggle.com/eranuragsingh/anurag-detailed-api-introduction/edit) and [Basic Submission Template](https://www.kaggle.com/eranuragsingh/anurag-basic-submission-template/edit).","metadata":{}}]}