# Team HEDJ solution

We've posted our code to github
[here](https://github.com/bitsofbits/kaggle_grasp_and_lift_eeg_detection)
.
Our approach was simple: we developed a neural net that scored ~0.971 on the leaderboard (I discussed the structure of this net previously in this
[post](https://www.kaggle.com/c/grasp-and-lift-eeg-detection/forums/t/16288/structure-of-our-best-net-hedj)
), generated a bunch of variations on this net and then just averaged the results together.  We weighted nets that we knew or strongly suspected would score well (>0.97) on the public leaderboard higher and we discarded poorly performing (0.963)  nets, but other than that we didn't really have a second level model.
The highest scoring single net that made it into our ensemble score 0.97147 on the private leaderboard. However, we didn't submit this net individually till after the close of the competition so, we didn't know that until recently.
We learned several things when we went to clean up, write up and recreate our scores. Two important ones were,  first  that we had undertrained many of our nets. Training them for longer times generally resulted in better scores, this was particularly true when paired with the improved validation setup we put together near the end of the competition.  Second, we didn't keep track of the exact parameters we used when training some of the nets. As a result, when we went to recreate everything we couldn't recreate the results of those early nets exactly and had to content ourselves with a score slightly above the score we submitted for third place.
A total of 34 nets made it into the final ensemble, and training all of these nets and generating a solution takes several days of runtime on machine with two high powered GPUs.  Using a large ensemble of nets is probably impractical, but a single net or even a few nets averaged together good probably make predictions fairly efficiently. This is because the output top several levels of the net, where most of the computation takes place,  could update their outputs incrementally rather than recomputing them from scratch each time.  The bookkeeping involved in implementing this would likely be something of headache but it seems possible.
If you have any trouble recreating any of the nets, please let us know and as soon as @deep is done with his Thesis/Paper marathon and I'm done moving, we'll get right back to:-)  (That is probably Tuesday or Wednesday).
Thanks for the opportunity to compete in the competition with this great dataset. It was a lot of fun.