{
  "id": 7029,
  "title": "Misc Data/Code Questions",
  "url": "/competitions/connectomics/discussion/7029",
  "author_name": "",
  "post_date": "2014-02-09T19:02:58.643Z",
  "votes": null,
  "comment_count": 14,
  "views": 2630,
  "content": "<p>A couple of (hopefully) quick data questions:</p>\n<p>1) what do negative fluorescence values mean? See e.g. neuron 3 at time step 2 below.</p>\n<p>&gt; fv=read.csv(&quot;fluorescence_valid.txt&quot;,header=F)<br>&gt; fv[1:5,1:5]<br> V1 V2 V3 V4 V5<br>1 0.026 0.172 0.063 0.031 0.010<br>2 0.012 0.159 -0.001 0.031 -0.011<br>3 0.030 0.200 0.068 0.049 0.027<br>4 0.013 0.142 0.051 0.038 0.040<br>5 0.005 0.200 0.027 0.024 0.042</p>\n<p>2) For the correlation between neurons 1 and 2, I obtain 0.9267311, while the Correlation_valid_test benchmark gives 0.353957 as the &quot;strength&quot; for this pair. Based on the program description I had expected these numbers to be the same. Can someone tell me what I'm missing?</p>\n<p>&gt; cor(fv[,1],fv[,2])<br>[1] 0.9267311<br>&gt; bm=read.csv(&quot;Correlation_valid_test_20140131T194629.csv&quot;,header=T)<br>&gt; bm[2,]<br> NET_neuronI_neuronJ Strength<br>2 valid_1_2 0.353957</p>\n<p>Thanks!!</p>",
  "messages": [
    {
      "id": "38535",
      "postDate": "02/09/2014 19:02:58",
      "content": "<p>A couple of (hopefully) quick data questions:</p>\n<p>1) what do negative fluorescence values mean? See e.g. neuron 3 at time step 2 below.</p>\n<p>&gt; fv=read.csv(&quot;fluorescence_valid.txt&quot;,header=F)<br>&gt; fv[1:5,1:5]<br> V1 V2 V3 V4 V5<br>1 0.026 0.172 0.063 0.031 0.010<br>2 0.012 0.159 -0.001 0.031 -0.011<br>3 0.030 0.200 0.068 0.049 0.027<br>4 0.013 0.142 0.051 0.038 0.040<br>5 0.005 0.200 0.027 0.024 0.042</p>\n<p>2) For the correlation between neurons 1 and 2, I obtain 0.9267311, while the Correlation_valid_test benchmark gives 0.353957 as the &quot;strength&quot; for this pair. Based on the program description I had expected these numbers to be the same. Can someone tell me what I'm missing?</p>\n<p>&gt; cor(fv[,1],fv[,2])<br>[1] 0.9267311<br>&gt; bm=read.csv(&quot;Correlation_valid_test_20140131T194629.csv&quot;,header=T)<br>&gt; bm[2,]<br> NET_neuronI_neuronJ Strength<br>2 valid_1_2 0.353957</p>\n<p>Thanks!!</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "38557",
      "postDate": "02/10/2014 15:25:54",
      "content": "<p>Hi,</p>\n<p>1) The fluorescence intensity is calculated based on the calcium concentration inside the cell with a saturating function that limits the fluorescence signal to the (0,1) interval. However, there's some noise added to the signal, that can effectively lower it below 0.</p>\n<p>2) If i recall correctly the correlation calculation on the test suite is performed after discretization of the data, not on the raw data itself.</p>\n<p>&nbsp;</p>\n<p>Best,</p>\n<p>&nbsp;</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "38566",
      "postDate": "02/10/2014 16:41:52",
      "content": "<p>[quote=Javier G. Orlandi;38557]</p>\n<p>2) If i recall correctly the correlation calculation on the test suite is performed after discretization of the data, not on the raw data itself.&nbsp;</p>\n<p>[/quote]</p>\n<p>I confirm that. The correlation benchmark is produced by the script &quot;challengeFastBaseline&quot; in the Matlab sample code, see&nbsp;http://connectomics.chalearn.org/help/sample-code. It takes only a few minutes to run.</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "38731",
      "postDate": "02/13/2014 15:21:11",
      "content": "<p>If a given connection {x;y} &nbsp; is absent from 'network_normal-j' files , then should I assume that weight is null?</p>\n<p>&nbsp;</p>\n<p>&nbsp;</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "38734",
      "postDate": "02/13/2014 17:51:27",
      "content": "<p>[quote=Herimanitra;38731]</p>\n<p>If a given connection {x;y} &nbsp; is absent from 'network_normal-j' files , then should I assume that weight is null?</p>\n<p>[/quote]</p>\n<p>Yes, this is correct, no connection, absent connection, weight 0 is all the same thing. Weights -1 also correspond to no connection. They in fact mean &quot;blocked&quot; connection because we simulated the effect of a chemical inhibitor blocking inhibitory connections. In this challenge, all remaining connections are excitatory. So you can replace -1 by 0.</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "40130",
      "postDate": "03/11/2014 00:44:00",
      "content": "<p>Not being a Matlab user I'm having a little trouble understanding the sample code provided, so hope you don't mind if I ask a couple questions.</p>\n<p>In the case of the GTE benchmark now showing on the leaderboard, is this result for the (presumably 3-bin) GTE algorithm by itself, or is it for the combination of GTE and simple correlation?</p>\n<p>Also, can someone please describe briefly in words how the &quot;blanking&quot; signal G enters into the calculation of the conditional probabilities -- is it just a matter of skipping over time intervals during which this condition is detected?</p>\n<p>Thanks!!</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "40144",
      "postDate": "03/11/2014 05:09:32",
      "content": "<p>G adds an additional dimension to the computed histogram that results in two separate GTE score matrices being computed -- scores for the&nbsp;non-bursting and&nbsp;bursting states of the network.&nbsp;In the MATLAB code, this ends up with the GTE array having the shape (N, N, 2). Only the scores for the non-bursting state (GTE(:, :, 1)) are returned by default, however.</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "40221",
      "postDate": "03/12/2014 16:22:55",
      "content": "<p>Exactly :)</p>\n<p>Regarding the benchmark, it's the 2-bin case with the same discretization as the other benchmarks.</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "40307",
      "postDate": "03/13/2014 22:12:55",
      "content": "<p>A) Are the blocked connections (-1) as good as random? Or are there features possible that could detect them?</p>\n<p>B) The activity values above 1 and below 0 due to noise/measuring: What is a good way to treat them? 1.2 as 1? Or rescale all values between 0 and 1?</p>\n<p>C) The bin size of 20ms seems too long to capture causality. If so, is there any hope of detecting the direction of a&nbsp;neuron connection? What features from the provided data could help with this? Time-shifting gives different scores for connection_i_j and connection_j_i, is this a sane approach, or simply introducing artifacts?</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "40318",
      "postDate": "03/14/2014 02:54:09",
      "content": "<p>[quote]</p>\n<p>A) Are the blocked connections (-1) as good as random? Or are there features possible that could detect them?</p>\n<p>[/quote]</p>\n<p>The short answer is you should treat them as 0 or no connection. They have no bearing on the fluorescence data.</p>\n<p>The long answer is that they are there because the method used to create the networks apparently created both excitatory (induces activity) and inhibitory (suppresses activity) connections to be more biologically plausible. To make the challenge more tractable, the inhibitory connections are &quot;blocked&quot;, which is possible in real neuronal preparations using certain drugs.</p>\n<p>[quote]</p>\n<p>B) The activity values above 1 and below 0 due to noise/measuring: What is a good way to treat them? 1.2 as 1? Or rescale all values between 0 and 1?</p>\n<p>[/quote]</p>\n<p>The fluorescence values themselves are somewhat arbitrary, really -- you can scale them however you like. What they represent is the underlying spiking activity of the neurons, and it is this spiking time series i.e. a series of binary events (spike, no spike) that you want to infer. The challenge of obtaining the spike events is digging the rare ones out of the noise and trying to figure out if a spike really came from a neuron or is just the effect of being physically very close to another neuron that did spike. This also leads into your next question.</p>\n<p>[quote]</p>\n<p>C) The bin size of 20ms seems too long to capture causality. If so, is there any hope of detecting the direction of a&nbsp;neuron connection? What features from the provided data could help with this? Time-shifting gives different scores for connection_i_j and connection_j_i, is this a sane approach, or simply introducing artifacts?</p>\n<p>[/quote]</p>\n<p>The slow time scale of the camera image data (i.e. the 20ms) relative to possible spiking intervals (as little as ~2ms though likely a bit slower) is another challenge, and one that the GTE method offers a solution for.</p>\n<p>If you haven't already, I'd highly recommend at least skimming Stetter et al (2012)&nbsp;Model-Free Reconstruction of Excitatory Neuronal Connectivity from Calcium Imaging Signals (http://www.ploscompbiol.org/article/info%3Adoi%2F10.1371%2Fjournal.pcbi.1002653). This paper is essentially the basis for this competition and describes how they approached this exact problem by formulating the GTE approach.</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "40335",
      "postDate": "03/14/2014 14:57:39",
      "content": "<p>Byron explained it much better than I could :)</p>\n<p>Regarding c), that's one of the main reasons behind the challenge. Establishing causality from spike trains with sub-ms precision is already hard. Dealing with 20ms time precision fluorescence data makes it even&nbsp;harder. However, fluorescence imaging techniques allows us to monitor many more neurons simultaneously than we could with electrodes or more invasive methods.&nbsp;</p>\n<p>To put it in context, one of the goals behind the Brain Initiative is to simultaneously accquire in vivo data from&nbsp;as many neurons as possible, and algorithms that can make sense of all this data are still needed. Even if we never achieve 100% reconstructions, just identifying patterns and causal&nbsp;paths or structures is going to be amazing!</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "40389",
      "postDate": "03/15/2014 13:33:53",
      "content": "<p>Hi Admins</p>\n<p>Quick question for you.&nbsp; Can you please clarify the description of the network files data:</p>\n<ul>\n<li><strong>&quot;network&quot; files (type N)</strong>: Each row is a connection. The column structure is of the form I,J,W denoting a connection from neuron I to neuron J with weight W. Connections with weight -1 are &quot;blocked&quot; in the simulations (simulating a chemical blockage), so you should consider them as absent.</li>\n</ul>\n<p>I interpreted this to mean that line I,J,W with W=1 indicates that a spike on neuron I is likely to cause a spike on neuron J.&nbsp; Is that correct?&nbsp; Or do I have it backwards?</p>\n\n<p>Thanks</p>\n<p>John</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "40586",
      "postDate": "03/18/2014 21:34:59",
      "content": "<p>Hi John,</p>\n<p>You got it right.</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "40603",
      "postDate": "03/19/2014 00:36:56",
      "content": "<p>Thanks Javier - I must have a bug then&nbsp; :-(</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "40633",
      "postDate": "03/19/2014 16:23:01",
      "content": "<p>Another way to check it:</p>\n<p>Just look for a pair (I,J) with a -1 in the network file.&nbsp;Given that pair, if you look the entries (I,K) (for any K) they also have to be -1.&nbsp;Since the neuron I is inhibitory, all its output connections K's must have weight -1. Hence, (I,J) means I-&gt;J.&nbsp;</p>\n<p>Also, if you think of the network entries (I,J) as (row,column) in a matrix, all the -1s have to be concentrated on the same rows (and also the +1s). In this representation the rows represent the output connections of a neuron. And since the connection type (excitatory/inhibitory) is based on Dale's principle, all non-zero entries of a row must have the same sign.</p>",
      "rawMarkdown": "",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 38557,
      "author_name": "orlandi",
      "author_url": "",
      "post_date": "02/10/2014 15:25:54",
      "content": "<p>Hi,</p>\n<p>1) The fluorescence intensity is calculated based on the calcium concentration inside the cell with a saturating function that limits the fluorescence signal to the (0,1) interval. However, there's some noise added to the signal, that can effectively lower it below 0.</p>\n<p>2) If i recall correctly the correlation calculation on the test suite is performed after discretization of the data, not on the raw data itself.</p>\n<p>&nbsp;</p>\n<p>Best,</p>\n<p>&nbsp;</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 38566,
      "author_name": "iguyon",
      "author_url": "",
      "post_date": "02/10/2014 16:41:52",
      "content": "<p>[quote=Javier G. Orlandi;38557]</p>\n<p>2) If i recall correctly the correlation calculation on the test suite is performed after discretization of the data, not on the raw data itself.&nbsp;</p>\n<p>[/quote]</p>\n<p>I confirm that. The correlation benchmark is produced by the script &quot;challengeFastBaseline&quot; in the Matlab sample code, see&nbsp;http://connectomics.chalearn.org/help/sample-code. It takes only a few minutes to run.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 38731,
      "author_name": "herimanitra",
      "author_url": "",
      "post_date": "02/13/2014 15:21:11",
      "content": "<p>If a given connection {x;y} &nbsp; is absent from 'network_normal-j' files , then should I assume that weight is null?</p>\n<p>&nbsp;</p>\n<p>&nbsp;</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 38734,
      "author_name": "iguyon",
      "author_url": "",
      "post_date": "02/13/2014 17:51:27",
      "content": "<p>[quote=Herimanitra;38731]</p>\n<p>If a given connection {x;y} &nbsp; is absent from 'network_normal-j' files , then should I assume that weight is null?</p>\n<p>[/quote]</p>\n<p>Yes, this is correct, no connection, absent connection, weight 0 is all the same thing. Weights -1 also correspond to no connection. They in fact mean &quot;blocked&quot; connection because we simulated the effect of a chemical inhibitor blocking inhibitory connections. In this challenge, all remaining connections are excitatory. So you can replace -1 by 0.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 40130,
      "author_name": "bcragin",
      "author_url": "",
      "post_date": "03/11/2014 00:44:00",
      "content": "<p>Not being a Matlab user I'm having a little trouble understanding the sample code provided, so hope you don't mind if I ask a couple questions.</p>\n<p>In the case of the GTE benchmark now showing on the leaderboard, is this result for the (presumably 3-bin) GTE algorithm by itself, or is it for the combination of GTE and simple correlation?</p>\n<p>Also, can someone please describe briefly in words how the &quot;blanking&quot; signal G enters into the calculation of the conditional probabilities -- is it just a matter of skipping over time intervals during which this condition is detected?</p>\n<p>Thanks!!</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 40144,
      "author_name": "bgalbraith",
      "author_url": "",
      "post_date": "03/11/2014 05:09:32",
      "content": "<p>G adds an additional dimension to the computed histogram that results in two separate GTE score matrices being computed -- scores for the&nbsp;non-bursting and&nbsp;bursting states of the network.&nbsp;In the MATLAB code, this ends up with the GTE array having the shape (N, N, 2). Only the scores for the non-bursting state (GTE(:, :, 1)) are returned by default, however.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 40221,
      "author_name": "orlandi",
      "author_url": "",
      "post_date": "03/12/2014 16:22:55",
      "content": "<p>Exactly :)</p>\n<p>Regarding the benchmark, it's the 2-bin case with the same discretization as the other benchmarks.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 40307,
      "author_name": "triskelion",
      "author_url": "",
      "post_date": "03/13/2014 22:12:55",
      "content": "<p>A) Are the blocked connections (-1) as good as random? Or are there features possible that could detect them?</p>\n<p>B) The activity values above 1 and below 0 due to noise/measuring: What is a good way to treat them? 1.2 as 1? Or rescale all values between 0 and 1?</p>\n<p>C) The bin size of 20ms seems too long to capture causality. If so, is there any hope of detecting the direction of a&nbsp;neuron connection? What features from the provided data could help with this? Time-shifting gives different scores for connection_i_j and connection_j_i, is this a sane approach, or simply introducing artifacts?</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 40318,
      "author_name": "bgalbraith",
      "author_url": "",
      "post_date": "03/14/2014 02:54:09",
      "content": "<p>[quote]</p>\n<p>A) Are the blocked connections (-1) as good as random? Or are there features possible that could detect them?</p>\n<p>[/quote]</p>\n<p>The short answer is you should treat them as 0 or no connection. They have no bearing on the fluorescence data.</p>\n<p>The long answer is that they are there because the method used to create the networks apparently created both excitatory (induces activity) and inhibitory (suppresses activity) connections to be more biologically plausible. To make the challenge more tractable, the inhibitory connections are &quot;blocked&quot;, which is possible in real neuronal preparations using certain drugs.</p>\n<p>[quote]</p>\n<p>B) The activity values above 1 and below 0 due to noise/measuring: What is a good way to treat them? 1.2 as 1? Or rescale all values between 0 and 1?</p>\n<p>[/quote]</p>\n<p>The fluorescence values themselves are somewhat arbitrary, really -- you can scale them however you like. What they represent is the underlying spiking activity of the neurons, and it is this spiking time series i.e. a series of binary events (spike, no spike) that you want to infer. The challenge of obtaining the spike events is digging the rare ones out of the noise and trying to figure out if a spike really came from a neuron or is just the effect of being physically very close to another neuron that did spike. This also leads into your next question.</p>\n<p>[quote]</p>\n<p>C) The bin size of 20ms seems too long to capture causality. If so, is there any hope of detecting the direction of a&nbsp;neuron connection? What features from the provided data could help with this? Time-shifting gives different scores for connection_i_j and connection_j_i, is this a sane approach, or simply introducing artifacts?</p>\n<p>[/quote]</p>\n<p>The slow time scale of the camera image data (i.e. the 20ms) relative to possible spiking intervals (as little as ~2ms though likely a bit slower) is another challenge, and one that the GTE method offers a solution for.</p>\n<p>If you haven't already, I'd highly recommend at least skimming Stetter et al (2012)&nbsp;Model-Free Reconstruction of Excitatory Neuronal Connectivity from Calcium Imaging Signals (http://www.ploscompbiol.org/article/info%3Adoi%2F10.1371%2Fjournal.pcbi.1002653). This paper is essentially the basis for this competition and describes how they approached this exact problem by formulating the GTE approach.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 40335,
      "author_name": "orlandi",
      "author_url": "",
      "post_date": "03/14/2014 14:57:39",
      "content": "<p>Byron explained it much better than I could :)</p>\n<p>Regarding c), that's one of the main reasons behind the challenge. Establishing causality from spike trains with sub-ms precision is already hard. Dealing with 20ms time precision fluorescence data makes it even&nbsp;harder. However, fluorescence imaging techniques allows us to monitor many more neurons simultaneously than we could with electrodes or more invasive methods.&nbsp;</p>\n<p>To put it in context, one of the goals behind the Brain Initiative is to simultaneously accquire in vivo data from&nbsp;as many neurons as possible, and algorithms that can make sense of all this data are still needed. Even if we never achieve 100% reconstructions, just identifying patterns and causal&nbsp;paths or structures is going to be amazing!</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 40389,
      "author_name": "johnclarke",
      "author_url": "",
      "post_date": "03/15/2014 13:33:53",
      "content": "<p>Hi Admins</p>\n<p>Quick question for you.&nbsp; Can you please clarify the description of the network files data:</p>\n<ul>\n<li><strong>&quot;network&quot; files (type N)</strong>: Each row is a connection. The column structure is of the form I,J,W denoting a connection from neuron I to neuron J with weight W. Connections with weight -1 are &quot;blocked&quot; in the simulations (simulating a chemical blockage), so you should consider them as absent.</li>\n</ul>\n<p>I interpreted this to mean that line I,J,W with W=1 indicates that a spike on neuron I is likely to cause a spike on neuron J.&nbsp; Is that correct?&nbsp; Or do I have it backwards?</p>\n\n<p>Thanks</p>\n<p>John</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 40586,
      "author_name": "orlandi",
      "author_url": "",
      "post_date": "03/18/2014 21:34:59",
      "content": "<p>Hi John,</p>\n<p>You got it right.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 40603,
      "author_name": "johnclarke",
      "author_url": "",
      "post_date": "03/19/2014 00:36:56",
      "content": "<p>Thanks Javier - I must have a bug then&nbsp; :-(</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 40633,
      "author_name": "orlandi",
      "author_url": "",
      "post_date": "03/19/2014 16:23:01",
      "content": "<p>Another way to check it:</p>\n<p>Just look for a pair (I,J) with a -1 in the network file.&nbsp;Given that pair, if you look the entries (I,K) (for any K) they also have to be -1.&nbsp;Since the neuron I is inhibitory, all its output connections K's must have weight -1. Hence, (I,J) means I-&gt;J.&nbsp;</p>\n<p>Also, if you think of the network entries (I,J) as (row,column) in a matrix, all the -1s have to be concentrated on the same rows (and also the +1s). In this representation the rows represent the output connections of a neuron. And since the connection type (excitatory/inhibitory) is based on Dale's principle, all non-zero entries of a row must have the same sign.</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "38535": "",
    "38557": "",
    "38566": "",
    "38731": "",
    "38734": "",
    "40130": "",
    "40144": "",
    "40221": "",
    "40307": "",
    "40318": "",
    "40335": "",
    "40389": "",
    "40586": "",
    "40603": "",
    "40633": ""
  },
  "source": "meta"
}