{ "id": 7480, "title": "GTE and connection", "url": "/competitions/connectomics/discussion/7480", "author_name": "", "post_date": "2014-03-21T04:28:22.387Z", "votes": null, "comment_count": 10, "views": 1945, "content": "

Hello,

\n

In the simulation, the connection is either 0 or 1. The connection probability is from zero to one. The first two benchmarks in leaderboard use correlation, which is also from 0 to 1, to evaluate the connection.

\n

For GTE, how neural scientists scales those value? Are they simply scaling the maximum GTE as 1 and the lower bound threshold GTE as 0?

\n

GTE is evaluated at t time. Are we assume that the connection is constant all the time, so GTE should converge to constant after a long time?

\n

By the way, how can we understand the g threshold? Why it should be smaller than one value?

\n

Thanks,

\n

Li

", "messages": [ { "id": "40771", "postDate": "03/21/2014 04:28:22", "content": "

Hello,

\n

In the simulation, the connection is either 0 or 1. The connection probability is from zero to one. The first two benchmarks in leaderboard use correlation, which is also from 0 to 1, to evaluate the connection.

\n

For GTE, how neural scientists scales those value? Are they simply scaling the maximum GTE as 1 and the lower bound threshold GTE as 0?

\n

GTE is evaluated at t time. Are we assume that the connection is constant all the time, so GTE should converge to constant after a long time?

\n

By the way, how can we understand the g threshold? Why it should be smaller than one value?

\n

Thanks,

\n

Li

", "rawMarkdown": "", "votes": null }, { "id": "40785", "postDate": "03/21/2014 12:00:59", "content": "

Hi,

\n

GTE is an information measure and is quantified in bits and the values are not scaled. The benchmark used to test the goodness of the reconstruction is the AUC (Area Under the Curve) of the ROC (Receiver Operating Characteristic). To obtain the ROC one sorts the scores from higher to lower, so any global scaling becomes irrelevant.

\n

GTE computes probabilities for all times t, but the final result is summed over all t, so there's no time dependence in the GTE measure. And yes, the connection is constant, although as long as it was stationary it would be enough.

\n

The system under study has two distinctive dynamical regimes, one with very low activity and another with very high activity (bursts). Due to the slow acquisition time of the signal, information measures might yield different results of influence (or causality) depending on which regime the system is at. The G threshold divides the two regimes based on the global fluorescence signal and is later applied to GTE to only calculate it in the low activity regime. For a better description of the threshold you can check:

\n

http://www.ploscompbiol.org/article/info%3Adoi%2F10.1371%2Fjournal.pcbi.1002653

", "rawMarkdown": "", "votes": null }, { "id": "40807", "postDate": "03/21/2014 18:26:24", "content": "

Thanks, Javier!

\n

Just few follow-ups.

\n

1. So the benchmark GTE is using the average value over time?

\n

2. The reason benchmark GTE do not consider high activity region is because the causality is hard to see when all the neurons fires?

\n

Thanks,

\n

Li

", "rawMarkdown": "", "votes": null }, { "id": "40808", "postDate": "03/21/2014 18:51:08", "content": "

Hi,

\n

1. Sorry, I didn't explain it well. Given two signals X and Y, GTE for X->Y has to calculate transition probabilitie of the form P(X_present | X_past), P(X_present | X_past, Y_past) etc... Since you don't know their functional form you estimate them from the data, and to do so you iterate over the whole time series. These P don't depend on time.

\n

2. Yes. During the periods of high activity most causality measures would tell you that either all neurons are connected to each other or none are, due to the fact that the activity appears to be synchronized (which in fact is not). This is a direct consequence of the poor time resolution of the signals.

", "rawMarkdown": "", "votes": null }, { "id": "41026", "postDate": "03/26/2014 03:29:17", "content": "

Dear admin,

\n

Thank you very much for your patient reply.

\n

For a tradition definition of transfer entropy, which does not consider y_{n+1} in the conditional probability distribution, if x series equals to y series, the transfer entropy will be zero because the part in log function is 1.

\n

However, by considering y_{n+1} in the condition, if x series equals to y series, the transfer entropy will be much bigger (over 10 times) than the situation where x series does not equal to y series.

\n

Anyone find similar case?

\n

Thanks,

\n

Li

", "rawMarkdown": "", "votes": null }, { "id": "41030", "postDate": "03/26/2014 05:15:54", "content": "

For what it's worth, Li, I have not seen the behavior you describe. Taking

\n

A = (1,1,3,3,3,2,2,2,1,1,1,1) and
B = (1,1,1,3,3,3,2,1,1,1,1,1),

\n

for example, my (Markov order 2) code gives GTE(A -> B) = 0.4854753 and GTE(B -> A) = 0.160964, suggesting that A may indeed be the cause of B. For GTE(A -> A) I get only 0.360964, not something of order 10x larger.

\n

My code has not been properly validated, however, and you should therefore not take my word for it.

\n

Note: If someone who has the Matlab code working would kindly run the above example (or some other simple examples) and post the results, that would be an extremely useful reference point to have for all who are writing their own GTE code. TIA!

", "rawMarkdown": "", "votes": null }, { "id": "41078", "postDate": "03/27/2014 05:36:41", "content": "

Hello, Algoasaurus,

\n

I have tested the sample A = (1,1,3,3,3,2,2,2,1,1,1,1) and B = (1,1,1,3,3,3,2,1,1,1,1,1) on my python code with 2nd order Markov chain. 

\n

GTE(A -> B) = 0.0922305553953

\n

GTE(B -> A) = 0.111111111111

\n

GTE(A -> A) = 0.333333333333

\n

It is different from your result... I just start to try GTE, so I am not sure whether I am right. I really appreciate if someone can post their result on this sample test.

\n

And also, I made up another long test sample:

\n

A = [0,0,0,0,0,0,1,0,0,1,0,0,1,0,0,1,0,0,1,0,0,1,0,0,1,0,0,1,0,0,1,0,0,1,0,0,1,0,0,1]
B = [0,0,0,0,0,0,0,0,1,0,0,1,0,0,1,0,0,1,0,0,1,0,0,1,0,0,1,0,0,1,0,0,1,0,0,1,0,0,1,0]

\n

B is just 2 steps later than A.

\n

My code shows:

\n

GTE(A -> B) = 0.430360688971

\n

GTE(B -> A) = 0.241620828505

\n

GTE(A -> A) = 0.339178679165

\n

Best,

\n

Li

", "rawMarkdown": "", "votes": null }, { "id": "41083", "postDate": "03/27/2014 08:52:28", "content": "

Li li, that appears to be right, but I'll have to get a closer look.

\n

It seems that by allowing "instantaneous" interactions you lose some of the nice properties of TE, like 0 score on perfect synchronisation. However, due to the poor time resolution of the experimental signals, the inclusion of that term is needed to greatly improve TE performance on the dataset. This is also related to the fact that GTE overestimates bidirectional links.

", "rawMarkdown": "", "votes": null }, { "id": "41087", "postDate": "03/27/2014 10:07:03", "content": "

Li, just for the sake of completeness I wanted to mention that for the two length=40 sequences you listed, my code gave GTE(A -> B) = 0.09630953 and GTE(B -->A) = 0.390863. Thanks for posting your results. The fact that our results differ shows the need for someone with access to the "official" GTE code to post some sample values.

", "rawMarkdown": "", "votes": null }, { "id": "41127", "postDate": "03/28/2014 16:14:05", "content": "

Algoasaurus, the length=40 sequences is designed that B is caused by A. So I think A->B should be larger than B->A.

", "rawMarkdown": "", "votes": null }, { "id": "41131", "postDate": "03/28/2014 18:30:53", "content": "

I understand what you're saying, Li, but I feel it's more complicated. In terms of cross-correlation there is indeed a peak at lag +2, which is consistent with A --> B. But there is also a secondary peak at lag -1 (for the original ordering of A and B) that becomes a peak at lag +1 if the roles of A and B are reversed. So, I'm inclined to think that B --> A is indicated at least as strongly as A --> B. Putting the argument another way: what if the first occurrence of 1 in sequence A is just a noise spike. Replace that 1 by 0 and what you are left with is a sequence that I'm sure you would call evidence of B --> A rather than vice versa.

", "rawMarkdown": "", "votes": null } ], "comments": [ { "id": 40785, "author_name": "orlandi", "author_url": "", "post_date": "03/21/2014 12:00:59", "content": "

Hi,

\n

GTE is an information measure and is quantified in bits and the values are not scaled. The benchmark used to test the goodness of the reconstruction is the AUC (Area Under the Curve) of the ROC (Receiver Operating Characteristic). To obtain the ROC one sorts the scores from higher to lower, so any global scaling becomes irrelevant.

\n

GTE computes probabilities for all times t, but the final result is summed over all t, so there's no time dependence in the GTE measure. And yes, the connection is constant, although as long as it was stationary it would be enough.

\n

The system under study has two distinctive dynamical regimes, one with very low activity and another with very high activity (bursts). Due to the slow acquisition time of the signal, information measures might yield different results of influence (or causality) depending on which regime the system is at. The G threshold divides the two regimes based on the global fluorescence signal and is later applied to GTE to only calculate it in the low activity regime. For a better description of the threshold you can check:

\n

http://www.ploscompbiol.org/article/info%3Adoi%2F10.1371%2Fjournal.pcbi.1002653

", "votes": null, "replies": [] }, { "id": 40807, "author_name": "aikinogard", "author_url": "", "post_date": "03/21/2014 18:26:24", "content": "

Thanks, Javier!

\n

Just few follow-ups.

\n

1. So the benchmark GTE is using the average value over time?

\n

2. The reason benchmark GTE do not consider high activity region is because the causality is hard to see when all the neurons fires?

\n

Thanks,

\n

Li

", "votes": null, "replies": [] }, { "id": 40808, "author_name": "orlandi", "author_url": "", "post_date": "03/21/2014 18:51:08", "content": "

Hi,

\n

1. Sorry, I didn't explain it well. Given two signals X and Y, GTE for X->Y has to calculate transition probabilitie of the form P(X_present | X_past), P(X_present | X_past, Y_past) etc... Since you don't know their functional form you estimate them from the data, and to do so you iterate over the whole time series. These P don't depend on time.

\n

2. Yes. During the periods of high activity most causality measures would tell you that either all neurons are connected to each other or none are, due to the fact that the activity appears to be synchronized (which in fact is not). This is a direct consequence of the poor time resolution of the signals.

", "votes": null, "replies": [] }, { "id": 41026, "author_name": "aikinogard", "author_url": "", "post_date": "03/26/2014 03:29:17", "content": "

Dear admin,

\n

Thank you very much for your patient reply.

\n

For a tradition definition of transfer entropy, which does not consider y_{n+1} in the conditional probability distribution, if x series equals to y series, the transfer entropy will be zero because the part in log function is 1.

\n

However, by considering y_{n+1} in the condition, if x series equals to y series, the transfer entropy will be much bigger (over 10 times) than the situation where x series does not equal to y series.

\n

Anyone find similar case?

\n

Thanks,

\n

Li

", "votes": null, "replies": [] }, { "id": 41030, "author_name": "bcragin", "author_url": "", "post_date": "03/26/2014 05:15:54", "content": "

For what it's worth, Li, I have not seen the behavior you describe. Taking

\n

A = (1,1,3,3,3,2,2,2,1,1,1,1) and
B = (1,1,1,3,3,3,2,1,1,1,1,1),

\n

for example, my (Markov order 2) code gives GTE(A -> B) = 0.4854753 and GTE(B -> A) = 0.160964, suggesting that A may indeed be the cause of B. For GTE(A -> A) I get only 0.360964, not something of order 10x larger.

\n

My code has not been properly validated, however, and you should therefore not take my word for it.

\n

Note: If someone who has the Matlab code working would kindly run the above example (or some other simple examples) and post the results, that would be an extremely useful reference point to have for all who are writing their own GTE code. TIA!

", "votes": null, "replies": [] }, { "id": 41078, "author_name": "aikinogard", "author_url": "", "post_date": "03/27/2014 05:36:41", "content": "

Hello, Algoasaurus,

\n

I have tested the sample A = (1,1,3,3,3,2,2,2,1,1,1,1) and B = (1,1,1,3,3,3,2,1,1,1,1,1) on my python code with 2nd order Markov chain. 

\n

GTE(A -> B) = 0.0922305553953

\n

GTE(B -> A) = 0.111111111111

\n

GTE(A -> A) = 0.333333333333

\n

It is different from your result... I just start to try GTE, so I am not sure whether I am right. I really appreciate if someone can post their result on this sample test.

\n

And also, I made up another long test sample:

\n

A = [0,0,0,0,0,0,1,0,0,1,0,0,1,0,0,1,0,0,1,0,0,1,0,0,1,0,0,1,0,0,1,0,0,1,0,0,1,0,0,1]
B = [0,0,0,0,0,0,0,0,1,0,0,1,0,0,1,0,0,1,0,0,1,0,0,1,0,0,1,0,0,1,0,0,1,0,0,1,0,0,1,0]

\n

B is just 2 steps later than A.

\n

My code shows:

\n

GTE(A -> B) = 0.430360688971

\n

GTE(B -> A) = 0.241620828505

\n

GTE(A -> A) = 0.339178679165

\n

Best,

\n

Li

", "votes": null, "replies": [] }, { "id": 41083, "author_name": "orlandi", "author_url": "", "post_date": "03/27/2014 08:52:28", "content": "

Li li, that appears to be right, but I'll have to get a closer look.

\n

It seems that by allowing "instantaneous" interactions you lose some of the nice properties of TE, like 0 score on perfect synchronisation. However, due to the poor time resolution of the experimental signals, the inclusion of that term is needed to greatly improve TE performance on the dataset. This is also related to the fact that GTE overestimates bidirectional links.

", "votes": null, "replies": [] }, { "id": 41087, "author_name": "bcragin", "author_url": "", "post_date": "03/27/2014 10:07:03", "content": "

Li, just for the sake of completeness I wanted to mention that for the two length=40 sequences you listed, my code gave GTE(A -> B) = 0.09630953 and GTE(B -->A) = 0.390863. Thanks for posting your results. The fact that our results differ shows the need for someone with access to the "official" GTE code to post some sample values.

", "votes": null, "replies": [] }, { "id": 41127, "author_name": "aikinogard", "author_url": "", "post_date": "03/28/2014 16:14:05", "content": "

Algoasaurus, the length=40 sequences is designed that B is caused by A. So I think A->B should be larger than B->A.

", "votes": null, "replies": [] }, { "id": 41131, "author_name": "bcragin", "author_url": "", "post_date": "03/28/2014 18:30:53", "content": "

I understand what you're saying, Li, but I feel it's more complicated. In terms of cross-correlation there is indeed a peak at lag +2, which is consistent with A --> B. But there is also a secondary peak at lag -1 (for the original ordering of A and B) that becomes a peak at lag +1 if the roles of A and B are reversed. So, I'm inclined to think that B --> A is indicated at least as strongly as A --> B. Putting the argument another way: what if the first occurrence of 1 in sequence A is just a noise spike. Replace that 1 by 0 and what you are left with is a sequence that I'm sure you would call evidence of B --> A rather than vice versa.

", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "40771": "", "40785": "", "40807": "", "40808": "", "41026": "", "41030": "", "41078": "", "41083": "", "41087": "", "41127": "", "41131": "" }, "source": "meta" }