{
  "id": 16617,
  "title": "Team daheimao solution",
  "url": "/competitions/grasp-and-lift-eeg-detection/writeups/daheimao-team-daheimao-solution",
  "author_name": "",
  "post_date": "2015-10-02T03:57:23.423Z",
  "votes": 15,
  "comment_count": 11,
  "views": 5008,
  "content": "<p>Code and documentation can be found <a href=\"https://github.com/stupiding/kaggle_EEG\">here</a>.</p>\n\n<p>I am a Ph.D student majored in neural networks and computer vision. This is my first Kaggle competition. My intention of participating in this competition is to evaluate the performance of recurrent convolutional neural network (RCNN) in processing time series data. RCNN is firstly proposed by I for image classification (<a href=\"http://www.xlhu.cn/papers/Liang15-cvpr.pdf\">CVPR 2015</a>), and then used for scene labeling (will appear in <a href=\"https://nips.cc/Conferences/2015/AcceptedPapers\">NIPS 2015</a> soon). In these two tasks input data is static image, and RCNN performs well. It turns out that RCNN also performs well for EEG data. Without any domain knowledge related modification, our best single model achieves 0.97652/0.97661 public/private LB scores. </p>\n\n<p>The pipeline of our solution is simple:</p>\n\n<p>1.A set of candidate models (mostly RCNN) are trained in a 4-fold cross validation (CV) manner over the training set.</p>\n\n<p>2.Greedy forward selection (GFS) is applied to the candidate model set, so that some models are selected for combination and submission.</p>\n\n<p>The selected models are re-train over all the training data, and their predictions over the test data are averaged and submitted.</p>\n\n<p>The pre-processing is very simple, just removing the mean of each input sample, which is the common practice in image classification. The model is neither subject-specific nor event-specific. A model receives the data of any subject as input, and outputs six predictions.</p>\n\n<p>Most of our models are RCNN, whose structure is described in the <a href=\"https://github.com/stupiding/kaggle_EEG/blob/master/README.md\">documentation</a> and the papers. I also tried CNN, and the best CNN model achieves 0.97136/0.97297 public/priviate LB scores.</p>\n\n<p>GFS seems to work well and its results are consistent with the LB scores. However, I made some mistakes when there were 3 days before the deadline. And our ensemble almost stops improving since then.</p>\n\n<p>To sum up,  my first kaggle competition is very exciting. I want to give thanks to the organizers and all the other teams!</p>",
  "messages": [
    {
      "id": "93167",
      "postDate": "09/22/2015 11:23:55",
      "content": "<p>Code and documentation can be found <a href=\"https://github.com/stupiding/kaggle_EEG\">here</a>.</p>\n\n<p>I am a Ph.D student majored in neural networks and computer vision. This is my first Kaggle competition. My intention of participating in this competition is to evaluate the performance of recurrent convolutional neural network (RCNN) in processing time series data. RCNN is firstly proposed by I for image classification (<a href=\"http://www.xlhu.cn/papers/Liang15-cvpr.pdf\">CVPR 2015</a>), and then used for scene labeling (will appear in <a href=\"https://nips.cc/Conferences/2015/AcceptedPapers\">NIPS 2015</a> soon). In these two tasks input data is static image, and RCNN performs well. It turns out that RCNN also performs well for EEG data. Without any domain knowledge related modification, our best single model achieves 0.97652/0.97661 public/private LB scores. </p>\n\n<p>The pipeline of our solution is simple:</p>\n\n<p>1.A set of candidate models (mostly RCNN) are trained in a 4-fold cross validation (CV) manner over the training set.</p>\n\n<p>2.Greedy forward selection (GFS) is applied to the candidate model set, so that some models are selected for combination and submission.</p>\n\n<p>The selected models are re-train over all the training data, and their predictions over the test data are averaged and submitted.</p>\n\n<p>The pre-processing is very simple, just removing the mean of each input sample, which is the common practice in image classification. The model is neither subject-specific nor event-specific. A model receives the data of any subject as input, and outputs six predictions.</p>\n\n<p>Most of our models are RCNN, whose structure is described in the <a href=\"https://github.com/stupiding/kaggle_EEG/blob/master/README.md\">documentation</a> and the papers. I also tried CNN, and the best CNN model achieves 0.97136/0.97297 public/priviate LB scores.</p>\n\n<p>GFS seems to work well and its results are consistent with the LB scores. However, I made some mistakes when there were 3 days before the deadline. And our ensemble almost stops improving since then.</p>\n\n<p>To sum up,  my first kaggle competition is very exciting. I want to give thanks to the organizers and all the other teams!</p>",
      "rawMarkdown": "Code and documentation can be found [here](https://github.com/stupiding/kaggle_EEG).\r\n\r\nI am a Ph.D student majored in neural networks and computer vision. This is my first Kaggle competition. My intention of participating in this competition is to evaluate the performance of recurrent convolutional neural network (RCNN) in processing time series data. RCNN is firstly proposed by I for image classification ([CVPR 2015](http://www.xlhu.cn/papers/Liang15-cvpr.pdf)), and then used for scene labeling (will appear in [NIPS 2015](https://nips.cc/Conferences/2015/AcceptedPapers) soon). In these two tasks input data is static image, and RCNN performs well. It turns out that RCNN also performs well for EEG data. Without any domain knowledge related modification, our best single model achieves 0.97652/0.97661 public/private LB scores. \r\n\r\nThe pipeline of our solution is simple:\r\n\r\n1.A set of candidate models (mostly RCNN) are trained in a 4-fold cross validation (CV) manner over the training set.\r\n\r\n2.Greedy forward selection (GFS) is applied to the candidate model set, so that some models are selected for combination and submission.\r\n\r\nThe selected models are re-train over all the training data, and their predictions over the test data are averaged and submitted.\r\n\r\nThe pre-processing is very simple, just removing the mean of each input sample, which is the common practice in image classification. The model is neither subject-specific nor event-specific. A model receives the data of any subject as input, and outputs six predictions.\r\n\r\nMost of our models are RCNN, whose structure is described in the [documentation](https://github.com/stupiding/kaggle_EEG/blob/master/README.md) and the papers. I also tried CNN, and the best CNN model achieves 0.97136/0.97297 public/priviate LB scores.\r\n\r\nGFS seems to work well and its results are consistent with the LB scores. However, I made some mistakes when there were 3 days before the deadline. And our ensemble almost stops improving since then.\r\n\r\nTo sum up,  my first kaggle competition is very exciting. I want to give thanks to the organizers and all the other teams!",
      "votes": null
    },
    {
      "id": "93208",
      "postDate": "09/22/2015 20:47:24",
      "content": "<p>@daheimao That is very impressive for a single RCNN to go as far as 97.6%. Thanks for sharing your solution. I am very surprised that your ensemble didn't go much further given you have this much strong single model. It could be they are not as diversified, I guess.</p>\n\n<p>I am so surprised how convolutional networks seemed a perfect fit for this time-series data. I didn't see any paper out there which did so well on time-series using CNN or RCNN let alone on electrophysiological signals. The origanizers of this competition were content to have a per subject and per event based training as they expressed in the forums and they didn't think it was possible to generalize across subjects so well. Guess what, your team and our team proved them wrong. We didn't do any per subject or per event training, although we were tempted at some point. This is so astounding given that these nets are beating up the traditional domain expert hand crafted features pipeline so well. </p>\n\n<p>I would be curious to see what could have happened if our CNN models and your RCNN models were ensembled together :)</p>\n\n<p>If you are interested in writing a paper on either this EEG dataset (which I think is less interesting for a paper as the way the data was collected is already published) or another electrophysiological data set (including several channel peripheral physiological signals and EEG. This dataset is not published yet) for emotion and psychological state prediction, let me know.</p>",
      "rawMarkdown": "daheimao That is very impressive for a single RCNN to go as far as 97.6%. Thanks for sharing your solution. I am very surprised that your ensemble didn't go much further given you have this much strong single model. It could be they are not as diversified, I guess.\r\n\r\nI am so surprised how convolutional networks seemed a perfect fit for this time-series data. I didn't see any paper out there which did so well on time-series using CNN or RCNN let alone on electrophysiological signals. The origanizers of this competition were content to have a per subject and per event based training as they expressed in the forums and they didn't think it was possible to generalize across subjects so well. Guess what, your team and our team proved them wrong. We didn't do any per subject or per event training, although we were tempted at some point. This is so astounding given that these nets are beating up the traditional domain expert hand crafted features pipeline so well. \r\n\r\nI would be curious to see what could have happened if our CNN models and your RCNN models were ensembled together :)\r\n\r\nIf you are interested in writing a paper on either this EEG dataset (which I think is less interesting for a paper as the way the data was collected is already published) or another electrophysiological data set (including several channel peripheral physiological signals and EEG. This dataset is not published yet) for emotion and psychological state prediction, let me know.",
      "votes": null
    },
    {
      "id": "93211",
      "postDate": "09/22/2015 21:09:01",
      "content": "<p>Hey,\nThis is a very impressive architecture / performance. Congrats !</p>\n\n<p>I Join @deep to share my surprise about how well CNN worked for this dataset. \nWhile there is a few paper trying to apply CNN on EEG, it's the first time i see such good results, which is very encouraging for the field of BCI.</p>\n\n<p>However, there still remains doubts about what kind of activity the network is triggering on. The high perf of low pass time-domain is very suspicious, and i can not wrap my head around it. \nThere is plenty of other motor imagery dataset, and it could be interesting to give it a try.</p>\n\n<p>I think that one very interesting thing to do will be to analyze the coefficients of each layer and see if somehow we can draw some conclusion from it (and that would make a great paper :) )</p>",
      "rawMarkdown": "Hey,\r\nThis is a very impressive architecture / performance. Congrats !\r\n\r\nI Join @deep to share my surprise about how well CNN worked for this dataset. \r\nWhile there is a few paper trying to apply CNN on EEG, it's the first time i see such good results, which is very encouraging for the field of BCI.\r\n\r\nHowever, there still remains doubts about what kind of activity the network is triggering on. The high perf of low pass time-domain is very suspicious, and i can not wrap my head around it. \r\nThere is plenty of other motor imagery dataset, and it could be interesting to give it a try.\r\n\r\nI think that one very interesting thing to do will be to analyze the coefficients of each layer and see if somehow we can draw some conclusion from it (and that would make a great paper :) )",
      "votes": null
    },
    {
      "id": "93315",
      "postDate": "09/23/2015 19:28:41",
      "content": "<p>@Alex, I've had the same doubts about the low frequency components seemingly high performance at the start of the competition. </p>\n\n<p>However, after I explored the data more and trained nets with the known EEG frequency bands, i.e. delta, theta, alpha, beta and gamma, I was convinced that the low frequency components aren't as much helpful as originally thought. Actually, the very low freq component trained net only bested the others by a very insignificant marigin both locally and the LB. </p>\n\n<p>So, I am confident that if we visualize the weight matrices in the different layers, it won't just be dominated with low freq component features. But, have to do that and confirm, aws :-)</p>",
      "rawMarkdown": "Alex, I've had the same doubts about the low frequency components seemingly high performance at the start of the competition. \r\n\r\nHowever, after I explored the data more and trained nets with the known EEG frequency bands, i.e. delta, theta, alpha, beta and gamma, I was convinced that the low frequency components aren't as much helpful as originally thought. Actually, the very low freq component trained net only bested the others by a very insignificant marigin both locally and the LB. \r\n\r\nSo, I am confident that if we visualize the weight matrices in the different layers, it won't just be dominated with low freq component features. But, have to do that and confirm, aws :-)",
      "votes": null
    },
    {
      "id": "93353",
      "postDate": "09/24/2015 05:24:49",
      "content": "<p>@Deep, thanks. Yes, I am interested in writing a paper on electrophysiological signal processing. I am not familiar with this field, so it will be great if you can find some other datasets :)</p>\n\n<p>@Alex, thanks. I agree with you that visualizing the learned filters is interesting. Hopefully some useful conclusions may be drawn.</p>",
      "rawMarkdown": "Deep, thanks. Yes, I am interested in writing a paper on electrophysiological signal processing. I am not familiar with this field, so it will be great if you can find some other datasets :)\r\n\r\n@Alex, thanks. I agree with you that visualizing the learned filters is interesting. Hopefully some useful conclusions may be drawn.",
      "votes": null
    },
    {
      "id": "93690",
      "postDate": "09/30/2015 11:48:29",
      "content": "<p>Quick update: Training all your 36 submission models on a single GPU takes ages. :-) </p>\n\n<p>So far I'm done with 20 of them. So it will take at least another week before I can reproduce your results.</p>",
      "rawMarkdown": "Quick update: Training all your 36 submission models on a single GPU takes ages. :-) \r\n\r\nSo far I'm done with 20 of them. So it will take at least another week before I can reproduce your results.",
      "votes": null
    },
    {
      "id": "93816",
      "postDate": "10/01/2015 18:50:21",
      "content": "<p>daheimao's blog post on their approach is now live on no free hunch <a href=\"http://blog.kaggle.com/2015/09/29/grasp-and-lift-eeg-detection-winners-interview-2nd-place-daheimao/\">here</a>.</p>",
      "rawMarkdown": "daheimao's blog post on their approach is now live on no free hunch [here][1].\r\n\r\n\r\n  [1]: http://blog.kaggle.com/2015/09/29/grasp-and-lift-eeg-detection-winners-interview-2nd-place-daheimao/",
      "votes": null
    },
    {
      "id": "95652",
      "postDate": "10/10/2015 03:37:36",
      "content": "<p>@daheimao @Klaus Greff I've already installed Lasagne and configured GPU, but it shows&quot;dnn not available&quot;, if I missed install other modules? or caffe?</p>",
      "rawMarkdown": "daheimao @Klaus Greff I've already installed Lasagne and configured GPU, but it shows\"dnn not available\", if I missed install other modules? or caffe?",
      "votes": null
    },
    {
      "id": "97158",
      "postDate": "10/23/2015 11:06:43",
      "content": "<p>Congratulations!  The RCNN's seem very interesting...</p>\n\n<p>Which single model architecture (from those in your github repository) gave that single model LB score that you mentioned (0.97652/0.97661)?</p>",
      "rawMarkdown": "Congratulations!  The RCNN's seem very interesting...\r\n\r\nWhich single model architecture (from those in your github repository) gave that single model LB score that you mentioned (0.97652/0.97661)?",
      "votes": null
    },
    {
      "id": "97286",
      "postDate": "10/25/2015 05:15:27",
      "content": "<p>Hi @rmldj, the model is len3584_resize3_bs_c1r4p5_f9n256r35p1</p>",
      "rawMarkdown": "Hi @rmldj, the model is len3584_resize3_bs_c1r4p5_f9n256r35p1",
      "votes": null
    },
    {
      "id": "106722",
      "postDate": "02/03/2016 14:26:50",
      "content": "<p>Hi @daheimao !\nI'm trying to understand how you've implemented Recurrent Convolution Layers (RCL from your paper) in code. <br>\nCould you shed a light where exactly RCLs are in model files (models/len***.py )? \nI can see there usage of standard Lasagne layers only .</p>\n\n<p>Thanks in advance.</p>",
      "rawMarkdown": "Hi @daheimao !\r\nI'm trying to understand how you've implemented Recurrent Convolution Layers (RCL from your paper) in code.  \r\nCould you shed a light where exactly RCLs are in model files (models/len***.py )? \r\nI can see there usage of standard Lasagne layers only .\r\n\r\nThanks in advance.",
      "votes": null
    },
    {
      "id": "151596",
      "postDate": "12/21/2016 13:45:35",
      "content": "<p>Hi ！I am trying to run your code, but my computer only has CPU, can  I only use CPU running this code? What should I do if it can?</p>",
      "rawMarkdown": "Hi ！I am trying to run your code, but my computer only has CPU, can  I only use CPU running this code? What should I do if it can?",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 93208,
      "author_name": "deepcnn",
      "author_url": "",
      "post_date": "09/22/2015 20:47:24",
      "content": "<p>@daheimao That is very impressive for a single RCNN to go as far as 97.6%. Thanks for sharing your solution. I am very surprised that your ensemble didn't go much further given you have this much strong single model. It could be they are not as diversified, I guess.</p>\n\n<p>I am so surprised how convolutional networks seemed a perfect fit for this time-series data. I didn't see any paper out there which did so well on time-series using CNN or RCNN let alone on electrophysiological signals. The origanizers of this competition were content to have a per subject and per event based training as they expressed in the forums and they didn't think it was possible to generalize across subjects so well. Guess what, your team and our team proved them wrong. We didn't do any per subject or per event training, although we were tempted at some point. This is so astounding given that these nets are beating up the traditional domain expert hand crafted features pipeline so well. </p>\n\n<p>I would be curious to see what could have happened if our CNN models and your RCNN models were ensembled together :)</p>\n\n<p>If you are interested in writing a paper on either this EEG dataset (which I think is less interesting for a paper as the way the data was collected is already published) or another electrophysiological data set (including several channel peripheral physiological signals and EEG. This dataset is not published yet) for emotion and psychological state prediction, let me know.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 93211,
      "author_name": "alexandrebarachant",
      "author_url": "",
      "post_date": "09/22/2015 21:09:01",
      "content": "<p>Hey,\nThis is a very impressive architecture / performance. Congrats !</p>\n\n<p>I Join @deep to share my surprise about how well CNN worked for this dataset. \nWhile there is a few paper trying to apply CNN on EEG, it's the first time i see such good results, which is very encouraging for the field of BCI.</p>\n\n<p>However, there still remains doubts about what kind of activity the network is triggering on. The high perf of low pass time-domain is very suspicious, and i can not wrap my head around it. \nThere is plenty of other motor imagery dataset, and it could be interesting to give it a try.</p>\n\n<p>I think that one very interesting thing to do will be to analyze the coefficients of each layer and see if somehow we can draw some conclusion from it (and that would make a great paper :) )</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 93315,
      "author_name": "deepcnn",
      "author_url": "",
      "post_date": "09/23/2015 19:28:41",
      "content": "<p>@Alex, I've had the same doubts about the low frequency components seemingly high performance at the start of the competition. </p>\n\n<p>However, after I explored the data more and trained nets with the known EEG frequency bands, i.e. delta, theta, alpha, beta and gamma, I was convinced that the low frequency components aren't as much helpful as originally thought. Actually, the very low freq component trained net only bested the others by a very insignificant marigin both locally and the LB. </p>\n\n<p>So, I am confident that if we visualize the weight matrices in the different layers, it won't just be dominated with low freq component features. But, have to do that and confirm, aws :-)</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 93353,
      "author_name": "daheimao",
      "author_url": "",
      "post_date": "09/24/2015 05:24:49",
      "content": "<p>@Deep, thanks. Yes, I am interested in writing a paper on electrophysiological signal processing. I am not familiar with this field, so it will be great if you can find some other datasets :)</p>\n\n<p>@Alex, thanks. I agree with you that visualizing the learned filters is interesting. Hopefully some useful conclusions may be drawn.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 93690,
      "author_name": "qwlouse",
      "author_url": "",
      "post_date": "09/30/2015 11:48:29",
      "content": "<p>Quick update: Training all your 36 submission models on a single GPU takes ages. :-) </p>\n\n<p>So far I'm done with 20 of them. So it will take at least another week before I can reproduce your results.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 93816,
      "author_name": "annavictoria",
      "author_url": "",
      "post_date": "10/01/2015 18:50:21",
      "content": "<p>daheimao's blog post on their approach is now live on no free hunch <a href=\"http://blog.kaggle.com/2015/09/29/grasp-and-lift-eeg-detection-winners-interview-2nd-place-daheimao/\">here</a>.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 95652,
      "author_name": "columnchou",
      "author_url": "",
      "post_date": "10/10/2015 03:37:36",
      "content": "<p>@daheimao @Klaus Greff I've already installed Lasagne and configured GPU, but it shows&quot;dnn not available&quot;, if I missed install other modules? or caffe?</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 97158,
      "author_name": "romualdj",
      "author_url": "",
      "post_date": "10/23/2015 11:06:43",
      "content": "<p>Congratulations!  The RCNN's seem very interesting...</p>\n\n<p>Which single model architecture (from those in your github repository) gave that single model LB score that you mentioned (0.97652/0.97661)?</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 97286,
      "author_name": "daheimao",
      "author_url": "",
      "post_date": "10/25/2015 05:15:27",
      "content": "<p>Hi @rmldj, the model is len3584_resize3_bs_c1r4p5_f9n256r35p1</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 106722,
      "author_name": "semenpyatakov",
      "author_url": "",
      "post_date": "02/03/2016 14:26:50",
      "content": "<p>Hi @daheimao !\nI'm trying to understand how you've implemented Recurrent Convolution Layers (RCL from your paper) in code. <br>\nCould you shed a light where exactly RCLs are in model files (models/len***.py )? \nI can see there usage of standard Lasagne layers only .</p>\n\n<p>Thanks in advance.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 151596,
      "author_name": "hxiancui",
      "author_url": "",
      "post_date": "12/21/2016 13:45:35",
      "content": "<p>Hi ！I am trying to run your code, but my computer only has CPU, can  I only use CPU running this code? What should I do if it can?</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "93167": "Code and documentation can be found [here](https://github.com/stupiding/kaggle_EEG).\r\n\r\nI am a Ph.D student majored in neural networks and computer vision. This is my first Kaggle competition. My intention of participating in this competition is to evaluate the performance of recurrent convolutional neural network (RCNN) in processing time series data. RCNN is firstly proposed by I for image classification ([CVPR 2015](http://www.xlhu.cn/papers/Liang15-cvpr.pdf)), and then used for scene labeling (will appear in [NIPS 2015](https://nips.cc/Conferences/2015/AcceptedPapers) soon). In these two tasks input data is static image, and RCNN performs well. It turns out that RCNN also performs well for EEG data. Without any domain knowledge related modification, our best single model achieves 0.97652/0.97661 public/private LB scores. \r\n\r\nThe pipeline of our solution is simple:\r\n\r\n1.A set of candidate models (mostly RCNN) are trained in a 4-fold cross validation (CV) manner over the training set.\r\n\r\n2.Greedy forward selection (GFS) is applied to the candidate model set, so that some models are selected for combination and submission.\r\n\r\nThe selected models are re-train over all the training data, and their predictions over the test data are averaged and submitted.\r\n\r\nThe pre-processing is very simple, just removing the mean of each input sample, which is the common practice in image classification. The model is neither subject-specific nor event-specific. A model receives the data of any subject as input, and outputs six predictions.\r\n\r\nMost of our models are RCNN, whose structure is described in the [documentation](https://github.com/stupiding/kaggle_EEG/blob/master/README.md) and the papers. I also tried CNN, and the best CNN model achieves 0.97136/0.97297 public/priviate LB scores.\r\n\r\nGFS seems to work well and its results are consistent with the LB scores. However, I made some mistakes when there were 3 days before the deadline. And our ensemble almost stops improving since then.\r\n\r\nTo sum up,  my first kaggle competition is very exciting. I want to give thanks to the organizers and all the other teams!",
    "93208": "daheimao That is very impressive for a single RCNN to go as far as 97.6%. Thanks for sharing your solution. I am very surprised that your ensemble didn't go much further given you have this much strong single model. It could be they are not as diversified, I guess.\r\n\r\nI am so surprised how convolutional networks seemed a perfect fit for this time-series data. I didn't see any paper out there which did so well on time-series using CNN or RCNN let alone on electrophysiological signals. The origanizers of this competition were content to have a per subject and per event based training as they expressed in the forums and they didn't think it was possible to generalize across subjects so well. Guess what, your team and our team proved them wrong. We didn't do any per subject or per event training, although we were tempted at some point. This is so astounding given that these nets are beating up the traditional domain expert hand crafted features pipeline so well. \r\n\r\nI would be curious to see what could have happened if our CNN models and your RCNN models were ensembled together :)\r\n\r\nIf you are interested in writing a paper on either this EEG dataset (which I think is less interesting for a paper as the way the data was collected is already published) or another electrophysiological data set (including several channel peripheral physiological signals and EEG. This dataset is not published yet) for emotion and psychological state prediction, let me know.",
    "93211": "Hey,\r\nThis is a very impressive architecture / performance. Congrats !\r\n\r\nI Join @deep to share my surprise about how well CNN worked for this dataset. \r\nWhile there is a few paper trying to apply CNN on EEG, it's the first time i see such good results, which is very encouraging for the field of BCI.\r\n\r\nHowever, there still remains doubts about what kind of activity the network is triggering on. The high perf of low pass time-domain is very suspicious, and i can not wrap my head around it. \r\nThere is plenty of other motor imagery dataset, and it could be interesting to give it a try.\r\n\r\nI think that one very interesting thing to do will be to analyze the coefficients of each layer and see if somehow we can draw some conclusion from it (and that would make a great paper :) )",
    "93315": "Alex, I've had the same doubts about the low frequency components seemingly high performance at the start of the competition. \r\n\r\nHowever, after I explored the data more and trained nets with the known EEG frequency bands, i.e. delta, theta, alpha, beta and gamma, I was convinced that the low frequency components aren't as much helpful as originally thought. Actually, the very low freq component trained net only bested the others by a very insignificant marigin both locally and the LB. \r\n\r\nSo, I am confident that if we visualize the weight matrices in the different layers, it won't just be dominated with low freq component features. But, have to do that and confirm, aws :-)",
    "93353": "Deep, thanks. Yes, I am interested in writing a paper on electrophysiological signal processing. I am not familiar with this field, so it will be great if you can find some other datasets :)\r\n\r\n@Alex, thanks. I agree with you that visualizing the learned filters is interesting. Hopefully some useful conclusions may be drawn.",
    "93690": "Quick update: Training all your 36 submission models on a single GPU takes ages. :-) \r\n\r\nSo far I'm done with 20 of them. So it will take at least another week before I can reproduce your results.",
    "93816": "daheimao's blog post on their approach is now live on no free hunch [here][1].\r\n\r\n\r\n  [1]: http://blog.kaggle.com/2015/09/29/grasp-and-lift-eeg-detection-winners-interview-2nd-place-daheimao/",
    "95652": "daheimao @Klaus Greff I've already installed Lasagne and configured GPU, but it shows\"dnn not available\", if I missed install other modules? or caffe?",
    "97158": "Congratulations!  The RCNN's seem very interesting...\r\n\r\nWhich single model architecture (from those in your github repository) gave that single model LB score that you mentioned (0.97652/0.97661)?",
    "97286": "Hi @rmldj, the model is len3584_resize3_bs_c1r4p5_f9n256r35p1",
    "106722": "Hi @daheimao !\r\nI'm trying to understand how you've implemented Recurrent Convolution Layers (RCL from your paper) in code.  \r\nCould you shed a light where exactly RCLs are in model files (models/len***.py )? \r\nI can see there usage of standard Lasagne layers only .\r\n\r\nThanks in advance.",
    "151596": "Hi ！I am trying to run your code, but my computer only has CPU, can  I only use CPU running this code? What should I do if it can?"
  },
  "source": "existing_discussion",
  "complete": true,
  "competition_id": 4477,
  "competition_slug": "grasp-and-lift-eeg-detection",
  "team_id": 207008,
  "solution_url": "/competitions/grasp-and-lift-eeg-detection/writeups/daheimao-team-daheimao-solution"
}