{
  "id": 287499,
  "title": "Granger causality test, IOTA, and strange results?",
  "url": "/competitions/g-research-crypto-forecasting/discussion/287499",
  "author_name": "",
  "post_date": "2021-11-14T09:11:38.609233800Z",
  "votes": 19,
  "comment_count": 12,
  "views": 0,
  "content": "<p>I have just written a short notebook <a href=\"https://www.kaggle.com/carlmcbrideellis/granger-causality-testing-for-1-day\" target=\"_blank\">\"Granger causality testing for 1 day\"</a> where a <a href=\"https://en.wikipedia.org/wiki/Granger_causality\" target=\"_blank\">Granger causality test</a> is performed, which is designed to test whether one time series can be used to forecast another time series, on all of the cryptocurrencies with each other (using the last day of our dataset).</p>\n<p>The results indicate that <em>all</em> of the other 13 cryptocurrencies seem to have a strong influence on the IOTA cryptocurrency, something that does not seem quite right to me. As I am by no means an expert in time series, and less still when it comes to the Granger causality test, I would appreciate your opinion as to whether the results are valid, or even make sense!</p>\n<p>Aside: We have seen in the notebook <a href=\"https://www.kaggle.com/carlmcbrideellis/correlograms-of-14-cryptocurrencies-1-day\" target=\"_blank\">\"Correlograms of 14 cryptocurrencies (1 day)\"</a> that each of the 14 cryptocurrencies on their own appear to be akin to random walks, so it is difficult to forecast the next move of each cryptocurrency individually. However, if some of the cryptocurrencies do indeed follow the lead of others then we have a great advantage at our disposal; knowing the value of a <em>leader</em> cryptocurrency will give a good idea of the future value of a <em>follower</em> cryptocurrency, and it is there that we can potentially make money.</p>\n<p>All the best,<br>\ncarl</p>\n<p><strong>Update:</strong> <a href=\"https://www.kaggle.com/carlmcbrideellis/granger-causality-part-ii-the-movie\" target=\"_blank\">\"<em>Granger causality Part II: The Movie</em>\"</a></p>",
  "messages": [
    {
      "id": "1581817",
      "postDate": "11/14/2021 09:11:38",
      "content": "<p>I have just written a short notebook <a href=\"https://www.kaggle.com/carlmcbrideellis/granger-causality-testing-for-1-day\" target=\"_blank\">\"Granger causality testing for 1 day\"</a> where a <a href=\"https://en.wikipedia.org/wiki/Granger_causality\" target=\"_blank\">Granger causality test</a> is performed, which is designed to test whether one time series can be used to forecast another time series, on all of the cryptocurrencies with each other (using the last day of our dataset).</p>\n<p>The results indicate that <em>all</em> of the other 13 cryptocurrencies seem to have a strong influence on the IOTA cryptocurrency, something that does not seem quite right to me. As I am by no means an expert in time series, and less still when it comes to the Granger causality test, I would appreciate your opinion as to whether the results are valid, or even make sense!</p>\n<p>Aside: We have seen in the notebook <a href=\"https://www.kaggle.com/carlmcbrideellis/correlograms-of-14-cryptocurrencies-1-day\" target=\"_blank\">\"Correlograms of 14 cryptocurrencies (1 day)\"</a> that each of the 14 cryptocurrencies on their own appear to be akin to random walks, so it is difficult to forecast the next move of each cryptocurrency individually. However, if some of the cryptocurrencies do indeed follow the lead of others then we have a great advantage at our disposal; knowing the value of a <em>leader</em> cryptocurrency will give a good idea of the future value of a <em>follower</em> cryptocurrency, and it is there that we can potentially make money.</p>\n<p>All the best,<br>\ncarl</p>\n<p><strong>Update:</strong> <a href=\"https://www.kaggle.com/carlmcbrideellis/granger-causality-part-ii-the-movie\" target=\"_blank\">\"<em>Granger causality Part II: The Movie</em>\"</a></p>",
      "rawMarkdown": "I have just written a short notebook [\"Granger causality testing for 1 day\"](https://www.kaggle.com/carlmcbrideellis/granger-causality-testing-for-1-day) where a [Granger causality test](https://en.wikipedia.org/wiki/Granger_causality) is performed, which is designed to test whether one time series can be used to forecast another time series, on all of the cryptocurrencies with each other (using the last day of our dataset).\n\nThe results indicate that *all* of the other 13 cryptocurrencies seem to have a strong influence on the IOTA cryptocurrency, something that does not seem quite right to me. As I am by no means an expert in time series, and less still when it comes to the Granger causality test, I would appreciate your opinion as to whether the results are valid, or even make sense!\n\nAside: We have seen in the notebook [\"Correlograms of 14 cryptocurrencies (1 day)\"](https://www.kaggle.com/carlmcbrideellis/correlograms-of-14-cryptocurrencies-1-day) that each of the 14 cryptocurrencies on their own appear to be akin to random walks, so it is difficult to forecast the next move of each cryptocurrency individually. However, if some of the cryptocurrencies do indeed follow the lead of others then we have a great advantage at our disposal; knowing the value of a *leader* cryptocurrency will give a good idea of the future value of a *follower* cryptocurrency, and it is there that we can potentially make money.\n\nAll the best,\ncarl\n\n**Update:** [\"*Granger causality Part II: The Movie*\"](https://www.kaggle.com/carlmcbrideellis/granger-causality-part-ii-the-movie)",
      "votes": null
    },
    {
      "id": "1581968",
      "postDate": "11/14/2021 11:56:11",
      "content": "<p>Nice catch ! I went trough their website and some of their paper. No simple explanation for what you observe. Maybe a general correlation with market (used as liquidity due to the feeless nature ? adoption increasing with crypto adoption due to their interaction with IoT ?). </p>\n<p>An interesting finding though: the IOTA DLT was relying on a coordinator (and thus wasn't really decentralized). They are currently in the process of removing that coordinator. With a 'coordicide' deployed on a devellopment branch as of october 4. Possibly a nice source of concept drift (and shake-up).</p>",
      "rawMarkdown": "Nice catch ! I went trough their website and some of their paper. No simple explanation for what you observe. Maybe a general correlation with market (used as liquidity due to the feeless nature ? adoption increasing with crypto adoption due to their interaction with IoT ?). \n\nAn interesting finding though: the IOTA DLT was relying on a coordinator (and thus wasn't really decentralized). They are currently in the process of removing that coordinator. With a 'coordicide' deployed on a devellopment branch as of october 4. Possibly a nice source of concept drift (and shake-up).",
      "votes": null
    },
    {
      "id": "1581981",
      "postDate": "11/14/2021 12:07:19",
      "content": "<p>Dear <a href=\"https://www.kaggle.com/lucasmorin\" target=\"_blank\">@lucasmorin</a> </p>\n<p>Since posting I have been playing with the date ranges in the abovementioned notebook,  (See my new comment and notebook below).</p>\n<p>I would be interested to learn more of this  <em>coordinator</em> and the role played.</p>\n<p>All the best,<br>\ncarl</p>",
      "rawMarkdown": "Dear @lucasmorin \n\nSince posting I have been playing with the date ranges in the abovementioned notebook, ~~and I see the same findings for other date ranges; using all of May of 2021 as the range. However, before April 2021 the \"linkage\" seems to break and the results look different: for example, from 1 Jan to 1 April it is Etherium Classic that is influenced by many of the other cryptocurrencies.~~ (See my new comment and notebook below).\n\nI would be interested to learn more of this  *coordinator* and the role played.\n\nAll the best,\ncarl",
      "votes": null
    },
    {
      "id": "1582008",
      "postDate": "11/14/2021 12:53:17",
      "content": "<p>From what I've understood by skimming trough their website: </p>\n<ul>\n<li>Iota is a proof of work decentralized ledger technology</li>\n<li>What is unusual is that the work required for proof is not mainly a cryptographic problem but validating 2 previous transactions. (It seems that there is still a cryptographic problem left to avoid spam).</li>\n<li>This allow many things such as reducing the computing power needed by avoiding mining, and in turn time for validation, fees… etc. So much that transactions are free. It was tought to improve the scalability of currently existing blockchains.</li>\n<li>Validation relying on previous validation create  a very complex graph (Direct Acyclic Graph to be precise) called 'The Tangle'. </li>\n<li>The graph of who is validating what has to be maintened externally to avoid some type of attacks (someone validating himself). It was done trough a centralised algo called 'The coordinator'.</li>\n<li>This coordinator was (and has been) a point of attack.</li>\n<li>They have found a way to not rely on that coordinator and are in the process of removing it entirely. (an on-going event called 'The Coordicide').  </li>\n</ul>",
      "rawMarkdown": "From what I've understood by skimming trough their website: \n- Iota is a proof of work decentralized ledger technology\n- What is unusual is that the work required for proof is not mainly a cryptographic problem but validating 2 previous transactions. (It seems that there is still a cryptographic problem left to avoid spam).\n- This allow many things such as reducing the computing power needed by avoiding mining, and in turn time for validation, fees... etc. So much that transactions are free. It was tought to improve the scalability of currently existing blockchains.\n- Validation relying on previous validation create  a very complex graph (Direct Acyclic Graph to be precise) called 'The Tangle'. \n- The graph of who is validating what has to be maintened externally to avoid some type of attacks (someone validating himself). It was done trough a centralised algo called 'The coordinator'.\n- This coordinator was (and has been) a point of attack.\n- They have found a way to not rely on that coordinator and are in the process of removing it entirely. (an on-going event called 'The Coordicide').",
      "votes": null
    },
    {
      "id": "1582836",
      "postDate": "11/15/2021 10:23:13",
      "content": "<blockquote>\n  <p>As I am by no means an expert in time series</p>\n</blockquote>\n<p><a href=\"https://www.kaggle.com/carlmcbrideellis\" target=\"_blank\">@carlmcbrideellis</a> regarding time series I don't think a lifetime is enough to become an \"expert\", maybe Rob Hyndman being the exception ;-) anyway, I have not begun coding here yet but I can remind a couple of things regarding causality  tests in finance I have found relevant this years:</p>\n<ul>\n<li>Causality test should be made on stationary series. (You are using price changes so that condition probably checked)</li>\n<li>If there is both correlation among series and autocorrelation to some extent in each series then test can be positive, and indeed be positive… for the time tested. This might be the case here, not 100% sure.</li>\n<li>Apparent causality, if usable, is an inefficiency and as such unstable as hell, and usually ephemeral in markets.</li>\n<li>One last big point imo to remember when testing causality is that it will not really detect \"latent counfounders\",<br>\nthis is, real cause can be hidden and what the test reports is just a different time of reaction to that latent variable. What, again, can be usable/stable but most frecuently not.</li>\n</ul>\n<p>Anyway, cool path you are opening here looking for causality. Deceiving as hell, but definitely interesting!</p>",
      "rawMarkdown": "> As I am by no means an expert in time series\n\n@carlmcbrideellis regarding time series I don't think a lifetime is enough to become an \"expert\", maybe Rob Hyndman being the exception ;-) anyway, I have not begun coding here yet but I can remind a couple of things regarding causality  tests in finance I have found relevant this years:\n\n- Causality test should be made on stationary series. (You are using price changes so that condition probably checked)\n- If there is both correlation among series and autocorrelation to some extent in each series then test can be positive, and indeed be positive... for the time tested. This might be the case here, not 100% sure.\n- Apparent causality, if usable, is an inefficiency and as such unstable as hell, and usually ephemeral in markets.\n- One last big point imo to remember when testing causality is that it will not really detect \"latent counfounders\",\nthis is, real cause can be hidden and what the test reports is just a different time of reaction to that latent variable. What, again, can be usable/stable but most frecuently not.\n\nAnyway, cool path you are opening here looking for causality. Deceiving as hell, but definitely interesting!",
      "votes": null
    },
    {
      "id": "1582865",
      "postDate": "11/15/2021 11:03:44",
      "content": "<p>Gracias <a href=\"https://www.kaggle.com/miguelpm\" target=\"_blank\">@miguelpm</a> </p>\n<p>Un saludo muy cordial,<br>\ncarl</p>",
      "rawMarkdown": "Gracias @miguelpm \n\nUn saludo muy cordial,\ncarl",
      "votes": null
    },
    {
      "id": "1583107",
      "postDate": "11/15/2021 14:54:45",
      "content": "<p>Dear <a href=\"https://www.kaggle.com/lucasmorin\" target=\"_blank\">@lucasmorin</a> </p>\n<p>Ok, I have made a follow-up notebook <a href=\"https://www.kaggle.com/carlmcbrideellis/granger-causality-part-ii-the-movie\" target=\"_blank\">\"<em>Granger causality Part II: The Movie</em>\"</a> looking in particular at the 1 minute lag for the Granger causality test on a daily basis from the 1st of May 2021 up to the 20th of September 2021.</p>\n<p>Aside: Although not pertinent to this competition as the data we have been provided with is the data we use, given that HFT is the <em>de rigueur</em> these days it would be interesting to perform the same calculations on the 1 second timescale rather than the 1 minute time scale, looking at any diffusion of changes throughout the market.</p>\n<p>All the best,<br>\ncarl</p>",
      "rawMarkdown": "Dear @lucasmorin \n\nOk, I have made a follow-up notebook [\"*Granger causality Part II: The Movie*\"](https://www.kaggle.com/carlmcbrideellis/granger-causality-part-ii-the-movie) looking in particular at the 1 minute lag for the Granger causality test on a daily basis from the 1st of May 2021 up to the 20th of September 2021.\n\nAside: Although not pertinent to this competition as the data we have been provided with is the data we use, given that HFT is the *de rigueur* these days it would be interesting to perform the same calculations on the 1 second timescale rather than the 1 minute time scale, looking at any diffusion of changes throughout the market.\n\nAll the best,\ncarl",
      "votes": null
    },
    {
      "id": "1584261",
      "postDate": "11/16/2021 11:55:15",
      "content": "<p>Dear <a href=\"https://www.kaggle.com/carlmcbrideellis\" target=\"_blank\">@carlmcbrideellis</a> </p>\n<p>Do you recommend any reference to understand the Granger causality test?</p>\n<p>Thanks in advance</p>",
      "rawMarkdown": "Dear @carlmcbrideellis \n\nDo you recommend any reference to understand the Granger causality test?\n\nThanks in advance",
      "votes": null
    },
    {
      "id": "1584269",
      "postDate": "11/16/2021 12:02:05",
      "content": "<p>Dear <a href=\"https://www.kaggle.com/maherelouahabi\" target=\"_blank\">@maherelouahabi</a> </p>\n<p>A good place to start would be with the Wikipedia page on <a href=\"https://en.wikipedia.org/wiki/Granger_causality\" target=\"_blank\">Granger causality</a> along with the original paper <a href=\"https://doi.org/10.2307/1912791\" target=\"_blank\">C. W. J. Granger \"<em>Investigating Causal Relations by Econometric Models and Cross-spectral Methods</em>\", Econometrica <strong>Vol. 37</strong> pp. 424-438 (1969)</a>. However, in terms of a very loose outline we shall make use of this image from Wikipedia:</p>\n<p><img src=\"https://raw.githubusercontent.com/Carl-McBride-Ellis/images_for_kaggle/main/Granger_causality_small.png\" alt=\"\"></p>\n<p>We see the two impulses in X appearing later in Y, however the impulses in Y do not later appear in X. One can imagine here that X is the <em>leader</em>, and Y is the <em>follower</em>. What the the Granger causality test asks is whether one can better predict, or forecast, future values of Y using the past values of both Y and X than one can using the past values of Y alone.<br>\nWhy do we need this mixing? Imagine X and Y were sine waves, with X peaking just before Y. One could say that X causes Y. However, being a simple sine wave, we could just as well predict the future value of Y without any help at all from X. By mixing we are looking at how much does X bring to the table when it comes to predicting Y. The null hypothesis being that X is of no help in predicting the future of Y.</p>\n<p>All the best,<br>\ncarl</p>",
      "rawMarkdown": "Dear @maherelouahabi \n\nA good place to start would be with the Wikipedia page on [Granger causality](https://en.wikipedia.org/wiki/Granger_causality) along with the original paper [C. W. J. Granger \"*Investigating Causal Relations by Econometric Models and Cross-spectral Methods*\", Econometrica **Vol. 37** pp. 424-438 (1969)](https://doi.org/10.2307/1912791). However, in terms of a very loose outline we shall make use of this image from Wikipedia:\n\n![](https://raw.githubusercontent.com/Carl-McBride-Ellis/images_for_kaggle/main/Granger_causality_small.png)\n\nWe see the two impulses in X appearing later in Y, however the impulses in Y do not later appear in X. One can imagine here that X is the *leader*, and Y is the *follower*. What the the Granger causality test asks is whether one can better predict, or forecast, future values of Y using the past values of both Y and X than one can using the past values of Y alone.\nWhy do we need this mixing? Imagine X and Y were sine waves, with X peaking just before Y. One could say that X causes Y. However, being a simple sine wave, we could just as well predict the future value of Y without any help at all from X. By mixing we are looking at how much does X bring to the table when it comes to predicting Y. The null hypothesis being that X is of no help in predicting the future of Y.\n\nAll the best,\ncarl",
      "votes": null
    },
    {
      "id": "1584319",
      "postDate": "11/16/2021 13:14:10",
      "content": "<p>Muchas gracias <a href=\"https://www.kaggle.com/carlmcbrideellis\" target=\"_blank\">@carlmcbrideellis</a> lo revisaré.</p>\n<p>Un saludo</p>",
      "rawMarkdown": "Muchas gracias @carlmcbrideellis lo revisaré.\n\nUn saludo",
      "votes": null
    },
    {
      "id": "1584470",
      "postDate": "11/16/2021 14:58:37",
      "content": "<p>Very useful notebook, thanks <a href=\"https://www.kaggle.com/carlmcbrideellis\" target=\"_blank\">@carlmcbrideellis</a> for sharing </p>",
      "rawMarkdown": "Very useful notebook, thanks @carlmcbrideellis for sharing",
      "votes": null
    },
    {
      "id": "1607270",
      "postDate": "12/05/2021 19:24:11",
      "content": "<p>Read firstly <a href=\"https://www.researchgate.net/publication/329166243_Toda-Yamamoto_Granger_No-causality_Analysis_of_Stock_Market_Growth_and_Economic_Growth_in_Ghana\" target=\"_blank\">https://www.researchgate.net/publication/329166243_Toda-Yamamoto_Granger_No-causality_Analysis_of_Stock_Market_Growth_and_Economic_Growth_in_Ghana</a> </p>",
      "rawMarkdown": "Read firstly https://www.researchgate.net/publication/329166243_Toda-Yamamoto_Granger_No-causality_Analysis_of_Stock_Market_Growth_and_Economic_Growth_in_Ghana",
      "votes": null
    },
    {
      "id": "1607281",
      "postDate": "12/05/2021 19:42:07",
      "content": "<p>Thank you <a href=\"https://www.kaggle.com/vladimircape\" target=\"_blank\">@vladimircape</a> </p>\n<p>¿Is there a section in particular to focus on?</p>\n<p>All the best,<br>\ncarl</p>",
      "rawMarkdown": "Thank you @vladimircape \n\n¿Is there a section in particular to focus on?\n\nAll the best,\ncarl",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 1581968,
      "author_name": "lucasmorin",
      "author_url": "",
      "post_date": "11/14/2021 11:56:11",
      "content": "<p>Nice catch ! I went trough their website and some of their paper. No simple explanation for what you observe. Maybe a general correlation with market (used as liquidity due to the feeless nature ? adoption increasing with crypto adoption due to their interaction with IoT ?). </p>\n<p>An interesting finding though: the IOTA DLT was relying on a coordinator (and thus wasn't really decentralized). They are currently in the process of removing that coordinator. With a 'coordicide' deployed on a devellopment branch as of october 4. Possibly a nice source of concept drift (and shake-up).</p>",
      "votes": null,
      "replies": [
        {
          "id": 1581981,
          "author_name": "carlmcbrideellis",
          "author_url": "",
          "post_date": "11/14/2021 12:07:19",
          "content": "<p>Dear <a href=\"https://www.kaggle.com/lucasmorin\" target=\"_blank\">@lucasmorin</a> </p>\n<p>Since posting I have been playing with the date ranges in the abovementioned notebook,  (See my new comment and notebook below).</p>\n<p>I would be interested to learn more of this  <em>coordinator</em> and the role played.</p>\n<p>All the best,<br>\ncarl</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1582008,
          "author_name": "lucasmorin",
          "author_url": "",
          "post_date": "11/14/2021 12:53:17",
          "content": "<p>From what I've understood by skimming trough their website: </p>\n<ul>\n<li>Iota is a proof of work decentralized ledger technology</li>\n<li>What is unusual is that the work required for proof is not mainly a cryptographic problem but validating 2 previous transactions. (It seems that there is still a cryptographic problem left to avoid spam).</li>\n<li>This allow many things such as reducing the computing power needed by avoiding mining, and in turn time for validation, fees… etc. So much that transactions are free. It was tought to improve the scalability of currently existing blockchains.</li>\n<li>Validation relying on previous validation create  a very complex graph (Direct Acyclic Graph to be precise) called 'The Tangle'. </li>\n<li>The graph of who is validating what has to be maintened externally to avoid some type of attacks (someone validating himself). It was done trough a centralised algo called 'The coordinator'.</li>\n<li>This coordinator was (and has been) a point of attack.</li>\n<li>They have found a way to not rely on that coordinator and are in the process of removing it entirely. (an on-going event called 'The Coordicide').  </li>\n</ul>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1583107,
          "author_name": "carlmcbrideellis",
          "author_url": "",
          "post_date": "11/15/2021 14:54:45",
          "content": "<p>Dear <a href=\"https://www.kaggle.com/lucasmorin\" target=\"_blank\">@lucasmorin</a> </p>\n<p>Ok, I have made a follow-up notebook <a href=\"https://www.kaggle.com/carlmcbrideellis/granger-causality-part-ii-the-movie\" target=\"_blank\">\"<em>Granger causality Part II: The Movie</em>\"</a> looking in particular at the 1 minute lag for the Granger causality test on a daily basis from the 1st of May 2021 up to the 20th of September 2021.</p>\n<p>Aside: Although not pertinent to this competition as the data we have been provided with is the data we use, given that HFT is the <em>de rigueur</em> these days it would be interesting to perform the same calculations on the 1 second timescale rather than the 1 minute time scale, looking at any diffusion of changes throughout the market.</p>\n<p>All the best,<br>\ncarl</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 1582836,
      "author_name": "miguelpm",
      "author_url": "",
      "post_date": "11/15/2021 10:23:13",
      "content": "<blockquote>\n  <p>As I am by no means an expert in time series</p>\n</blockquote>\n<p><a href=\"https://www.kaggle.com/carlmcbrideellis\" target=\"_blank\">@carlmcbrideellis</a> regarding time series I don't think a lifetime is enough to become an \"expert\", maybe Rob Hyndman being the exception ;-) anyway, I have not begun coding here yet but I can remind a couple of things regarding causality  tests in finance I have found relevant this years:</p>\n<ul>\n<li>Causality test should be made on stationary series. (You are using price changes so that condition probably checked)</li>\n<li>If there is both correlation among series and autocorrelation to some extent in each series then test can be positive, and indeed be positive… for the time tested. This might be the case here, not 100% sure.</li>\n<li>Apparent causality, if usable, is an inefficiency and as such unstable as hell, and usually ephemeral in markets.</li>\n<li>One last big point imo to remember when testing causality is that it will not really detect \"latent counfounders\",<br>\nthis is, real cause can be hidden and what the test reports is just a different time of reaction to that latent variable. What, again, can be usable/stable but most frecuently not.</li>\n</ul>\n<p>Anyway, cool path you are opening here looking for causality. Deceiving as hell, but definitely interesting!</p>",
      "votes": null,
      "replies": [
        {
          "id": 1582865,
          "author_name": "carlmcbrideellis",
          "author_url": "",
          "post_date": "11/15/2021 11:03:44",
          "content": "<p>Gracias <a href=\"https://www.kaggle.com/miguelpm\" target=\"_blank\">@miguelpm</a> </p>\n<p>Un saludo muy cordial,<br>\ncarl</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 1584261,
      "author_name": "maherelouahabi",
      "author_url": "",
      "post_date": "11/16/2021 11:55:15",
      "content": "<p>Dear <a href=\"https://www.kaggle.com/carlmcbrideellis\" target=\"_blank\">@carlmcbrideellis</a> </p>\n<p>Do you recommend any reference to understand the Granger causality test?</p>\n<p>Thanks in advance</p>",
      "votes": null,
      "replies": [
        {
          "id": 1584269,
          "author_name": "carlmcbrideellis",
          "author_url": "",
          "post_date": "11/16/2021 12:02:05",
          "content": "<p>Dear <a href=\"https://www.kaggle.com/maherelouahabi\" target=\"_blank\">@maherelouahabi</a> </p>\n<p>A good place to start would be with the Wikipedia page on <a href=\"https://en.wikipedia.org/wiki/Granger_causality\" target=\"_blank\">Granger causality</a> along with the original paper <a href=\"https://doi.org/10.2307/1912791\" target=\"_blank\">C. W. J. Granger \"<em>Investigating Causal Relations by Econometric Models and Cross-spectral Methods</em>\", Econometrica <strong>Vol. 37</strong> pp. 424-438 (1969)</a>. However, in terms of a very loose outline we shall make use of this image from Wikipedia:</p>\n<p><img src=\"https://raw.githubusercontent.com/Carl-McBride-Ellis/images_for_kaggle/main/Granger_causality_small.png\" alt=\"\"></p>\n<p>We see the two impulses in X appearing later in Y, however the impulses in Y do not later appear in X. One can imagine here that X is the <em>leader</em>, and Y is the <em>follower</em>. What the the Granger causality test asks is whether one can better predict, or forecast, future values of Y using the past values of both Y and X than one can using the past values of Y alone.<br>\nWhy do we need this mixing? Imagine X and Y were sine waves, with X peaking just before Y. One could say that X causes Y. However, being a simple sine wave, we could just as well predict the future value of Y without any help at all from X. By mixing we are looking at how much does X bring to the table when it comes to predicting Y. The null hypothesis being that X is of no help in predicting the future of Y.</p>\n<p>All the best,<br>\ncarl</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1584319,
          "author_name": "maherelouahabi",
          "author_url": "",
          "post_date": "11/16/2021 13:14:10",
          "content": "<p>Muchas gracias <a href=\"https://www.kaggle.com/carlmcbrideellis\" target=\"_blank\">@carlmcbrideellis</a> lo revisaré.</p>\n<p>Un saludo</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 1584470,
      "author_name": "ayush0510",
      "author_url": "",
      "post_date": "11/16/2021 14:58:37",
      "content": "<p>Very useful notebook, thanks <a href=\"https://www.kaggle.com/carlmcbrideellis\" target=\"_blank\">@carlmcbrideellis</a> for sharing </p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 1607270,
      "author_name": "vladimircape",
      "author_url": "",
      "post_date": "12/05/2021 19:24:11",
      "content": "<p>Read firstly <a href=\"https://www.researchgate.net/publication/329166243_Toda-Yamamoto_Granger_No-causality_Analysis_of_Stock_Market_Growth_and_Economic_Growth_in_Ghana\" target=\"_blank\">https://www.researchgate.net/publication/329166243_Toda-Yamamoto_Granger_No-causality_Analysis_of_Stock_Market_Growth_and_Economic_Growth_in_Ghana</a> </p>",
      "votes": null,
      "replies": [
        {
          "id": 1607281,
          "author_name": "carlmcbrideellis",
          "author_url": "",
          "post_date": "12/05/2021 19:42:07",
          "content": "<p>Thank you <a href=\"https://www.kaggle.com/vladimircape\" target=\"_blank\">@vladimircape</a> </p>\n<p>¿Is there a section in particular to focus on?</p>\n<p>All the best,<br>\ncarl</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "1581817": "I have just written a short notebook [\"Granger causality testing for 1 day\"](https://www.kaggle.com/carlmcbrideellis/granger-causality-testing-for-1-day) where a [Granger causality test](https://en.wikipedia.org/wiki/Granger_causality) is performed, which is designed to test whether one time series can be used to forecast another time series, on all of the cryptocurrencies with each other (using the last day of our dataset).\n\nThe results indicate that *all* of the other 13 cryptocurrencies seem to have a strong influence on the IOTA cryptocurrency, something that does not seem quite right to me. As I am by no means an expert in time series, and less still when it comes to the Granger causality test, I would appreciate your opinion as to whether the results are valid, or even make sense!\n\nAside: We have seen in the notebook [\"Correlograms of 14 cryptocurrencies (1 day)\"](https://www.kaggle.com/carlmcbrideellis/correlograms-of-14-cryptocurrencies-1-day) that each of the 14 cryptocurrencies on their own appear to be akin to random walks, so it is difficult to forecast the next move of each cryptocurrency individually. However, if some of the cryptocurrencies do indeed follow the lead of others then we have a great advantage at our disposal; knowing the value of a *leader* cryptocurrency will give a good idea of the future value of a *follower* cryptocurrency, and it is there that we can potentially make money.\n\nAll the best,\ncarl\n\n**Update:** [\"*Granger causality Part II: The Movie*\"](https://www.kaggle.com/carlmcbrideellis/granger-causality-part-ii-the-movie)",
    "1581968": "Nice catch ! I went trough their website and some of their paper. No simple explanation for what you observe. Maybe a general correlation with market (used as liquidity due to the feeless nature ? adoption increasing with crypto adoption due to their interaction with IoT ?). \n\nAn interesting finding though: the IOTA DLT was relying on a coordinator (and thus wasn't really decentralized). They are currently in the process of removing that coordinator. With a 'coordicide' deployed on a devellopment branch as of october 4. Possibly a nice source of concept drift (and shake-up).",
    "1581981": "Dear @lucasmorin \n\nSince posting I have been playing with the date ranges in the abovementioned notebook, ~~and I see the same findings for other date ranges; using all of May of 2021 as the range. However, before April 2021 the \"linkage\" seems to break and the results look different: for example, from 1 Jan to 1 April it is Etherium Classic that is influenced by many of the other cryptocurrencies.~~ (See my new comment and notebook below).\n\nI would be interested to learn more of this  *coordinator* and the role played.\n\nAll the best,\ncarl",
    "1582008": "From what I've understood by skimming trough their website: \n- Iota is a proof of work decentralized ledger technology\n- What is unusual is that the work required for proof is not mainly a cryptographic problem but validating 2 previous transactions. (It seems that there is still a cryptographic problem left to avoid spam).\n- This allow many things such as reducing the computing power needed by avoiding mining, and in turn time for validation, fees... etc. So much that transactions are free. It was tought to improve the scalability of currently existing blockchains.\n- Validation relying on previous validation create  a very complex graph (Direct Acyclic Graph to be precise) called 'The Tangle'. \n- The graph of who is validating what has to be maintened externally to avoid some type of attacks (someone validating himself). It was done trough a centralised algo called 'The coordinator'.\n- This coordinator was (and has been) a point of attack.\n- They have found a way to not rely on that coordinator and are in the process of removing it entirely. (an on-going event called 'The Coordicide').",
    "1582836": "> As I am by no means an expert in time series\n\n@carlmcbrideellis regarding time series I don't think a lifetime is enough to become an \"expert\", maybe Rob Hyndman being the exception ;-) anyway, I have not begun coding here yet but I can remind a couple of things regarding causality  tests in finance I have found relevant this years:\n\n- Causality test should be made on stationary series. (You are using price changes so that condition probably checked)\n- If there is both correlation among series and autocorrelation to some extent in each series then test can be positive, and indeed be positive... for the time tested. This might be the case here, not 100% sure.\n- Apparent causality, if usable, is an inefficiency and as such unstable as hell, and usually ephemeral in markets.\n- One last big point imo to remember when testing causality is that it will not really detect \"latent counfounders\",\nthis is, real cause can be hidden and what the test reports is just a different time of reaction to that latent variable. What, again, can be usable/stable but most frecuently not.\n\nAnyway, cool path you are opening here looking for causality. Deceiving as hell, but definitely interesting!",
    "1582865": "Gracias @miguelpm \n\nUn saludo muy cordial,\ncarl",
    "1583107": "Dear @lucasmorin \n\nOk, I have made a follow-up notebook [\"*Granger causality Part II: The Movie*\"](https://www.kaggle.com/carlmcbrideellis/granger-causality-part-ii-the-movie) looking in particular at the 1 minute lag for the Granger causality test on a daily basis from the 1st of May 2021 up to the 20th of September 2021.\n\nAside: Although not pertinent to this competition as the data we have been provided with is the data we use, given that HFT is the *de rigueur* these days it would be interesting to perform the same calculations on the 1 second timescale rather than the 1 minute time scale, looking at any diffusion of changes throughout the market.\n\nAll the best,\ncarl",
    "1584261": "Dear @carlmcbrideellis \n\nDo you recommend any reference to understand the Granger causality test?\n\nThanks in advance",
    "1584269": "Dear @maherelouahabi \n\nA good place to start would be with the Wikipedia page on [Granger causality](https://en.wikipedia.org/wiki/Granger_causality) along with the original paper [C. W. J. Granger \"*Investigating Causal Relations by Econometric Models and Cross-spectral Methods*\", Econometrica **Vol. 37** pp. 424-438 (1969)](https://doi.org/10.2307/1912791). However, in terms of a very loose outline we shall make use of this image from Wikipedia:\n\n![](https://raw.githubusercontent.com/Carl-McBride-Ellis/images_for_kaggle/main/Granger_causality_small.png)\n\nWe see the two impulses in X appearing later in Y, however the impulses in Y do not later appear in X. One can imagine here that X is the *leader*, and Y is the *follower*. What the the Granger causality test asks is whether one can better predict, or forecast, future values of Y using the past values of both Y and X than one can using the past values of Y alone.\nWhy do we need this mixing? Imagine X and Y were sine waves, with X peaking just before Y. One could say that X causes Y. However, being a simple sine wave, we could just as well predict the future value of Y without any help at all from X. By mixing we are looking at how much does X bring to the table when it comes to predicting Y. The null hypothesis being that X is of no help in predicting the future of Y.\n\nAll the best,\ncarl",
    "1584319": "Muchas gracias @carlmcbrideellis lo revisaré.\n\nUn saludo",
    "1584470": "Very useful notebook, thanks @carlmcbrideellis for sharing",
    "1607270": "Read firstly https://www.researchgate.net/publication/329166243_Toda-Yamamoto_Granger_No-causality_Analysis_of_Stock_Market_Growth_and_Economic_Growth_in_Ghana",
    "1607281": "Thank you @vladimircape \n\n¿Is there a section in particular to focus on?\n\nAll the best,\ncarl"
  },
  "source": "meta"
}