We're at around the halfway point of the competition; thanks to everyone who has submitted results so far!
\r\nI wanted to check in and see if anyone has applied deep learning methods or deep auto-encoders so far. Have they helped?
\r\nWhat other methods have folks tried that have worked well -- or not as well as expected?
", "messages": [ { "id": "5930", "postDate": "10/07/2011 12:53:29", "content": "We're at around the halfway point of the competition; thanks to everyone who has submitted results so far!
\r\nI wanted to check in and see if anyone has applied deep learning methods or deep auto-encoders so far. Have they helped?
\r\nWhat other methods have folks tried that have worked well -- or not as well as expected?
", "rawMarkdown": "", "votes": null }, { "id": "5952", "postDate": "10/08/2011 02:01:54", "content": "I've been spending much of my "free" time trying to make sense out of the sqishy low-fidelity Heritage Health data (got a late start), so my original intention was to avoid this contest. Unfortunately I've always been fascinated by dimensionality reduction\r\n problems, so I couldn't stay away; more unfortunately I didn't start until yesterday. Even though I have very little time (human or compute) available to throw at this, I'll try to get a few submissions done during the next 9 days. But no guarantees.
\r\nYou asked about deep learning and auto-encoding: My first submission (if I can get the coding and training done) will be a layered SOM-like gizmo of my own design that I've (hopefully!) modified to handle semi-supervised learning. Although not classical\r\n deep learning (no RBMs involved a la Hinton, for example), I believe it has similar intent.
\r\nI know it's a little premature to mention this, having submitted nothing at this point. And the whole thing might crash and burn. But since nobody else has responded I thought I'd get the ball rolling.
\r\nBy the way, given more time it would be fun to see how far the feature count could be reduced and still maintain an acceptable level of accuracy. Since the probability of winning this thing in the amount of time left is infinitesimal, my second submission\r\n may be along these lines...
\r\nThanks for putting this up... fun data!
", "rawMarkdown": "", "votes": null }, { "id": "5953", "postDate": "10/08/2011 02:40:43", "content": "@Clueless: I've also been pulled into working on this, even though I have too many other things to do :)
\r\nAnyway, I was thinking of using some old RBM code of mine for this, but the last time I used it on this much data it took\r\ndays of runtime & multiple runs to get it tuned properly. Given that experience & the short timeframe, I've been focusing on prototyping some other methods.
", "rawMarkdown": "", "votes": null }, { "id": "5955", "postDate": "10/08/2011 03:05:10", "content": "I know what you mean Chris. I have some RBM code that I thought about using, too, but I haven't touched it since the Netflix competition - and even then it did indeed take days of processing to get anything useful out of it. Plus I wrote the darn thing\r\n in about 20 hours, near the end of the competition, so the code is NOT optimal. I did sketch out some strategies for using CUDA with it (but moved on to other things). Ah well...
", "rawMarkdown": "", "votes": null } ], "comments": [ { "id": 5952, "author_name": "clueless0", "author_url": "", "post_date": "10/08/2011 02:01:54", "content": "I've been spending much of my "free" time trying to make sense out of the sqishy low-fidelity Heritage Health data (got a late start), so my original intention was to avoid this contest. Unfortunately I've always been fascinated by dimensionality reduction\r\n problems, so I couldn't stay away; more unfortunately I didn't start until yesterday. Even though I have very little time (human or compute) available to throw at this, I'll try to get a few submissions done during the next 9 days. But no guarantees.
\r\nYou asked about deep learning and auto-encoding: My first submission (if I can get the coding and training done) will be a layered SOM-like gizmo of my own design that I've (hopefully!) modified to handle semi-supervised learning. Although not classical\r\n deep learning (no RBMs involved a la Hinton, for example), I believe it has similar intent.
\r\nI know it's a little premature to mention this, having submitted nothing at this point. And the whole thing might crash and burn. But since nobody else has responded I thought I'd get the ball rolling.
\r\nBy the way, given more time it would be fun to see how far the feature count could be reduced and still maintain an acceptable level of accuracy. Since the probability of winning this thing in the amount of time left is infinitesimal, my second submission\r\n may be along these lines...
\r\nThanks for putting this up... fun data!
", "votes": null, "replies": [] }, { "id": 5953, "author_name": "chefele", "author_url": "", "post_date": "10/08/2011 02:40:43", "content": "@Clueless: I've also been pulled into working on this, even though I have too many other things to do :)
\r\nAnyway, I was thinking of using some old RBM code of mine for this, but the last time I used it on this much data it took\r\ndays of runtime & multiple runs to get it tuned properly. Given that experience & the short timeframe, I've been focusing on prototyping some other methods.
", "votes": null, "replies": [] }, { "id": 5955, "author_name": "clueless0", "author_url": "", "post_date": "10/08/2011 03:05:10", "content": "I know what you mean Chris. I have some RBM code that I thought about using, too, but I haven't touched it since the Netflix competition - and even then it did indeed take days of processing to get anything useful out of it. Plus I wrote the darn thing\r\n in about 20 hours, near the end of the competition, so the code is NOT optimal. I did sketch out some strategies for using CUDA with it (but moved on to other things). Ah well...
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "5930": "", "5952": "", "5953": "", "5955": "" }, "source": "meta" }