{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"### Granger causality testing for 1 day\nIn this short notebook we shall perform the [Granger causality test](https://en.wikipedia.org/wiki/Granger_causality) for each of the 14 cryptocurrencies on the very last day of the dataset associated with the kaggle [G-Research Crypto Forecasting competition](https://www.kaggle.com/c/g-research-crypto-forecasting). The objective of the Granger causality test is to asses whether one time series can be used to forecast another time series. To do this we shall make use of the statsmodels [grangercausalitytests](https://www.statsmodels.org/stable/generated/statsmodels.tsa.stattools.grangercausalitytests.html) routine.","metadata":{}},{"cell_type":"code","source":"import numpy as np \nimport pandas as pd\nimport matplotlib.pyplot as plt\n#plt.style.use('fivethirtyeight')\nplt.rcParams.update({'font.size': 12})\nplt.rcParams[\"figure.figsize\"] = (7, 7)\nimport seaborn as sns\nimport plotly.express as px\nfrom datetime import date\n\n# read in the data\ntrain = pd.read_csv(\"../input/g-research-crypto-forecasting/train.csv\")\nasset_details = pd.read_csv(\"../input/g-research-crypto-forecasting/asset_details.csv\")\ntrain['timestamp'] = pd.to_datetime(train['timestamp'], unit='s')\ntrain = train.set_index('timestamp')\nmapping = dict(asset_details[['Asset_ID', 'Asset_Name']].values)\ntrain[\"Asset_name\"] = train[\"Asset_ID\"].map(mapping)\ncryptocurrencies = asset_details['Asset_Name'].unique()","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","_kg_hide-input":true,"_kg_hide-output":true,"execution":{"iopub.status.busy":"2021-11-14T08:48:08.235011Z","iopub.execute_input":"2021-11-14T08:48:08.235375Z","iopub.status.idle":"2021-11-14T08:49:17.478568Z","shell.execute_reply.started":"2021-11-14T08:48:08.235287Z","shell.execute_reply":"2021-11-14T08:49:17.477324Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Here is where we select the date range to be tested (this can very easily be changed by the interested reader)","metadata":{}},{"cell_type":"code","source":"# extract the last day of data\nlast_day = train.loc['2021-09-20 00:00:00':'2021-09-21 00:00:00']\n# create columns of the Close values\nclose_columns = pd.pivot_table(last_day,index='timestamp',columns='Asset_ID',values='Close')","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Let us perform the Granger causality test for each pair of cryptocurrencies for lags of 1 up to 30 minutes, testing whether the time series in the second column (`j`) causes the time series in the first column (`i`). Note that the Granger test requires that the data be [stationary](https://en.wikipedia.org/wiki/Stationary_process), so we actually look at the [percentage changes](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.pct_change.html) in each of these two time series, rather than the absolute values:","metadata":{}},{"cell_type":"code","source":"import statsmodels.api as sm\nfrom statsmodels.tsa.stattools import grangercausalitytests\n\ngranger_matrix = np.zeros((14, 14), float)\nfor lag in range(1,31):  # look at lags from 1 to 30 minutes\n    for i in range(14):\n        for j in range(14):\n            results = grangercausalitytests(close_columns[[i, j]].pct_change().dropna(), [lag], verbose=False)\n            granger_matrix[i,j] = results[lag][0]['ssr_ftest'][0]\n    print(\"For a lag of\",lag,\"minutes the global maximum value is\",np.amax(granger_matrix))","metadata":{"execution":{"iopub.status.busy":"2021-11-14T08:49:17.484646Z","iopub.execute_input":"2021-11-14T08:49:17.485065Z","iopub.status.idle":"2021-11-14T08:51:01.582316Z","shell.execute_reply.started":"2021-11-14T08:49:17.48501Z","shell.execute_reply":"2021-11-14T08:51:01.581185Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"We can see that the strongest value is seen at a lag of 1 minute. Let us go back and take a closer look at the values for this particular lag","metadata":{}},{"cell_type":"code","source":"lag = 1\nfor i in range(14):\n    for j in range(14):\n        results = grangercausalitytests(close_columns[[i, j]].pct_change().dropna(), [lag], verbose=False)\n        granger_matrix[i,j] = results[lag][0]['ssr_ftest'][0]\n\nplt.rcParams[\"figure.figsize\"] = (12, 12)\nsns.heatmap(granger_matrix,cmap='YlOrBr',annot=True, fmt='.1f',);\nplt.yticks(asset_details.Asset_ID.values +0.5, asset_details.Asset_Name.values, rotation='horizontal');\nplt.xticks(asset_details.Asset_ID.values +0.5, asset_details.Asset_Name.values, rotation='vertical');","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2021-11-14T08:51:01.583956Z","iopub.execute_input":"2021-11-14T08:51:01.584605Z","iopub.status.idle":"2021-11-14T08:51:04.344005Z","shell.execute_reply.started":"2021-11-14T08:51:01.584553Z","shell.execute_reply":"2021-11-14T08:51:04.343119Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"We can see that the strongest signal corresponds to Ethereum (Asset_ID=6) seemingly causing changes in IOTA (Asset_ID=8):","metadata":{}},{"cell_type":"code","source":"resutls = grangercausalitytests(close_columns[[8, 6]].pct_change().dropna(), [1])","metadata":{"execution":{"iopub.status.busy":"2021-11-14T08:51:04.3464Z","iopub.execute_input":"2021-11-14T08:51:04.347292Z","iopub.status.idle":"2021-11-14T08:51:04.364029Z","shell.execute_reply.started":"2021-11-14T08:51:04.347245Z","shell.execute_reply":"2021-11-14T08:51:04.363248Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Let us make an interactive plot of the [percentage change](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.pct_change.html) in these two time series (without lag) together","metadata":{}},{"cell_type":"code","source":"IOTA_pct_change = pd.Series(close_columns[8]).pct_change().dropna()\nEthereum_pct_change = pd.Series(close_columns[6]).pct_change().dropna()\n\n# https://matplotlib.org/stable/gallery/color/named_colors.html\ncolor_1 = 'blue'\ncolor_2 = 'olive'\ncolor_3 = 'orange'\ncolor_4 = 'crimson'\ncolor_5 = 'limegreen'\ncolor_6 = 'red'\ncolor_7 = 'teal'\ncolor_8 = 'yellowgreen'\n\nfig  = px.line(y=IOTA_pct_change)\nfig.update_traces(line_color=color_2)\n\nfig_2  = px.line(y=Ethereum_pct_change)\nfig_2.update_traces(line_color=color_4)\nfig.add_trace(fig_2.data[0])\n\nfig.update_layout(\n    title=\"IOTA in olive green and Ethereum in crimson\",\n    xaxis_title=\"Time\",\n    yaxis_title=\"% change\",)\nfig.show();","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2021-11-14T08:51:04.364988Z","iopub.execute_input":"2021-11-14T08:51:04.365526Z","iopub.status.idle":"2021-11-14T08:51:05.434885Z","shell.execute_reply.started":"2021-11-14T08:51:04.365491Z","shell.execute_reply":"2021-11-14T08:51:05.434049Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Conclusion \nIt looks like *all* of the other 13 cryptocurrencies cause strong changes in the IOTA cryptocurrency (Asset_ID=8) one minute later. \n\nI am not at all sure whether to believe/trust this conclusion at the moment.\n\n### See also\n\n* [\"*Granger causality Part II: The Movie*\"](https://www.kaggle.com/carlmcbrideellis/granger-causality-part-ii-the-movie)  - where one can find a heatmap of the 1 minute lag that has been animated on a daily basis from the 1st of May 2021 up to the 20th of September 2021.\n\n### Related reading\n\n* [C. W. J. Granger \"*Investigating Causal Relations by Econometric Models and Cross-spectral Methods*\",  Econometrica **Vol. 37** pp. 424-438 (1969)](https://doi.org/10.2307/1912791)\n\n### Appendix\nThe asset details, mapping the `Asset_ID` with the `Asset_Name`","metadata":{}},{"cell_type":"code","source":"asset_details","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2021-11-14T08:51:05.436462Z","iopub.execute_input":"2021-11-14T08:51:05.437304Z","iopub.status.idle":"2021-11-14T08:51:05.452725Z","shell.execute_reply.started":"2021-11-14T08:51:05.437264Z","shell.execute_reply":"2021-11-14T08:51:05.45215Z"},"trusted":true},"execution_count":null,"outputs":[]}]}