{ "id": 11514, "title": "Getting Started on Domino Data Lab", "url": "/competitions/datasciencebowl/discussion/11514", "author_name": "", "post_date": "2015-01-13T21:28:33.093Z", "votes": 5, "comment_count": 3, "views": 3194, "content": "

Our friends at Domino have published a nice post to get started on this competition. Not only do they walk you through setting up the getting started tutorial in the cloud, but they add some nice features around version control, productionalizing, and parallelizing. It's well worth a read - 

\n

http://blog.dominodatalab.com/how-to-get-started-with-the-data-science-bowl/

", "messages": [ { "id": "62704", "postDate": "01/13/2015 21:28:33", "content": "

Our friends at Domino have published a nice post to get started on this competition. Not only do they walk you through setting up the getting started tutorial in the cloud, but they add some nice features around version control, productionalizing, and parallelizing. It's well worth a read - 

\n

http://blog.dominodatalab.com/how-to-get-started-with-the-data-science-bowl/

", "rawMarkdown": "", "votes": null }, { "id": "62710", "postDate": "01/14/2015 04:48:01", "content": "

Thanks for posting this. Learned a bit. I tried parallelizing just the random forest training in the R code and received an error message about a line that had me curious when pasting it in.

\n

trees_per_core = floor(num_trees / numCores)
Error: object 'num_trees' not found.

", "rawMarkdown": "", "votes": null }, { "id": "62711", "postDate": "01/14/2015 04:55:11", "content": "

Hi Charles,

\n

The code snippet in the post makes use of a variable (num_trees) that isdefined earlier in the code. You can see the full code that ran, here (note that to find this, I viewed the specific run and clicked on the "train_parallel.R" filename under the "command" that ran).

\n

In the non-parallel version, ntree isn't specified in the call to randomForest -- 500 is the default. But when we train a forest in parallel, each job gets a fraction of the total trees, so we need to explicitly set that parameter value. 

\n

Let me know if that makes sense...

", "rawMarkdown": "", "votes": null }, { "id": "62712", "postDate": "01/14/2015 05:52:28", "content": "

That worked very nicely. The random forest training time was about 4:11 on one CPU. On 4 cores with hyperthreading turned on it took 1:20. A nice speed up for so little coding! 

\n

 Thanks for the info.

", "rawMarkdown": "", "votes": null } ], "comments": [ { "id": 62710, "author_name": "charlesroberson", "author_url": "", "post_date": "01/14/2015 04:48:01", "content": "

Thanks for posting this. Learned a bit. I tried parallelizing just the random forest training in the R code and received an error message about a line that had me curious when pasting it in.

\n

trees_per_core = floor(num_trees / numCores)
Error: object 'num_trees' not found.

", "votes": null, "replies": [] }, { "id": 62711, "author_name": "dominodatalab", "author_url": "", "post_date": "01/14/2015 04:55:11", "content": "

Hi Charles,

\n

The code snippet in the post makes use of a variable (num_trees) that isdefined earlier in the code. You can see the full code that ran, here (note that to find this, I viewed the specific run and clicked on the "train_parallel.R" filename under the "command" that ran).

\n

In the non-parallel version, ntree isn't specified in the call to randomForest -- 500 is the default. But when we train a forest in parallel, each job gets a fraction of the total trees, so we need to explicitly set that parameter value. 

\n

Let me know if that makes sense...

", "votes": null, "replies": [] }, { "id": 62712, "author_name": "charlesroberson", "author_url": "", "post_date": "01/14/2015 05:52:28", "content": "

That worked very nicely. The random forest training time was about 4:11 on one CPU. On 4 cores with hyperthreading turned on it took 1:20. A nice speed up for so little coding! 

\n

 Thanks for the info.

", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "62704": "", "62710": "", "62711": "", "62712": "" }, "source": "meta" }