{ "id": 10821, "title": "Beat the benchmark with H2O - LB 0.4033703", "url": "/competitions/avazu-ctr-prediction/discussion/10821", "author_name": "", "post_date": "2014-11-02T07:13:06.793Z", "votes": 11, "comment_count": 54, "views": 29277, "content": "

Here's starter R code that should get you to a decent LB score.

\n

It does some common-sense feature engineering (extract day/hour, turn integers into categoricals, and add interactions between categoricals, some of which probably doesn't matter much), then builds a simple model using H2O's distributed GBM, but you can use GLM, RF or Deep Learning as well.  There's a validation step with training on days 21-29 and validation on the 30th day, then another model is trained on all 10 days and used for making predictions on the test set.

\n

Of course, while everything is done from R, the data resides inside the H2O cluster node(s) and scales on a distributed compute cluster, and everything is open source:

\n

https://github.com/0xdata/h2o/blob/master/R/examples/Kaggle/CTR.R

\n

LogLoss on training data: 0.392463
LogLoss on validation data: 0.4027492
LB: 0.4033703

\n

Note: I'm not exactly sure how much memory you'll need but I'd think that you need at least 32GB, and even then there might be some swapping of unused data frames to disk from time to time.  Probably best to use -Xmx128GB (or reduce the interaction features).

\n

Note: You'll need version 1597 (http://s3.amazonaws.com/h2o-release/h2o/master/1597/index.html) or later.

\n

NEW: Extensive training material at http://learn.h2o.ai with datasets and scripts at http://data.h2o.ai

\n

More info at: http://h2o.ai @hexadata https://twitter.com/ArnoCandel

\n

For Deep Learning, there's a R Vignette at https://t.co/kWzyFMGJ2S

\n

Also see these links for other Kaggle code:

\n

http://www.kaggle.com/c/tradeshift-text-classification/forums/t/10665/beat-the-benchmark-with-h2o-distributed-random-forest

\n

https://www.kaggle.com/c/afsis-soil-properties/forums/t/10568/ensemble-deep-learning-from-r-with-h2o-starter-kit

\n

Thanks, and good luck improving on this - Please let me know if it works for you!
Arno

", "messages": [ { "id": "57179", "postDate": "11/02/2014 07:13:06", "content": "

Here's starter R code that should get you to a decent LB score.

\n

It does some common-sense feature engineering (extract day/hour, turn integers into categoricals, and add interactions between categoricals, some of which probably doesn't matter much), then builds a simple model using H2O's distributed GBM, but you can use GLM, RF or Deep Learning as well.  There's a validation step with training on days 21-29 and validation on the 30th day, then another model is trained on all 10 days and used for making predictions on the test set.

\n

Of course, while everything is done from R, the data resides inside the H2O cluster node(s) and scales on a distributed compute cluster, and everything is open source:

\n

https://github.com/0xdata/h2o/blob/master/R/examples/Kaggle/CTR.R

\n

LogLoss on training data: 0.392463
LogLoss on validation data: 0.4027492
LB: 0.4033703

\n

Note: I'm not exactly sure how much memory you'll need but I'd think that you need at least 32GB, and even then there might be some swapping of unused data frames to disk from time to time.  Probably best to use -Xmx128GB (or reduce the interaction features).

\n

Note: You'll need version 1597 (http://s3.amazonaws.com/h2o-release/h2o/master/1597/index.html) or later.

\n

NEW: Extensive training material at http://learn.h2o.ai with datasets and scripts at http://data.h2o.ai

\n

More info at: http://h2o.ai @hexadata https://twitter.com/ArnoCandel

\n

For Deep Learning, there's a R Vignette at https://t.co/kWzyFMGJ2S

\n

Also see these links for other Kaggle code:

\n

http://www.kaggle.com/c/tradeshift-text-classification/forums/t/10665/beat-the-benchmark-with-h2o-distributed-random-forest

\n

https://www.kaggle.com/c/afsis-soil-properties/forums/t/10568/ensemble-deep-learning-from-r-with-h2o-starter-kit

\n

Thanks, and good luck improving on this - Please let me know if it works for you!
Arno

", "rawMarkdown": "", "votes": null }, { "id": "57185", "postDate": "11/02/2014 10:50:24", "content": "

Thanks again Arno,

\n

Really appreciate the 'tutorial' approach on real data for h2o.

\n

Something rather nice for engineering features for binary data.

\n

https://www.rochester.edu/college/psc/signorino/research/Carter_Signorino_2010_PA.pdf

", "rawMarkdown": "", "votes": null }, { "id": "57189", "postDate": "11/02/2014 13:35:31", "content": "

[quote=Arno Candel;57179]

\n

Here's starter R code that should get you to a LB score of 0.393!  Not sure whether there's a benchmark, but the title sounded good to me :)

\n\n

[/quote]

\n\n

I receive an error here:

\n

h2oServer <- h2o.init(ip="mr-0xd2", port = 53322)

\n\n

Silly question I am sure...but do we need to run on our own ip address? If so, will this code run locally and require massive amount of memory?

", "rawMarkdown": "", "votes": null }, { "id": "57199", "postDate": "11/02/2014 17:28:54", "content": "

[quote=Inspector;57189]

\n

Silly question I am sure...but do we need to run on our own ip address? If so, will this code run locally and require massive amount of memory?

\n

[/quote]

\n

If you want to run the server in your computer you can simply type "localhost" and allocate a maximum amount of memory and number of cores when you start the h2o server like this:

\n

h2oServer <- h2o.init(ip = "localhost", port = 54321, max_mem_size = '5g', nthreads = 4)

\n

don't forget to use the commands: h2o.ls() and h2o.rm() to keep h2o's server data size within the maximum memory limits otherwise you will get a server / memory error

", "rawMarkdown": "", "votes": null }, { "id": "57202", "postDate": "11/02/2014 18:01:30", "content": "

[quote=wacax;57199]

\n

If you want to run the server in your computer you can simple type "localhost" and allocate a maximum amount of memory and number of cores when you start the h2o server like this:

\n

h2oServer <- h2o.init(ip = "localhost", port = 54321, max_mem_size = '5g', nthreads = 4)

\n

don't forget to use the commands: h2o.ls() and h2o.rm() to keep h2o's server data size within the maximum memory limits otherwise you will get a server / memory error

\n

[/quote]

\n\n

Thank you. Do you just mean to use those commands as a general 'clean up' or do you need to somehow adjust the calls to the model function? Perhaps this is self evident if I knew more about H20 - if the algorithms can be used when the data is large relative to memory.

", "rawMarkdown": "", "votes": null }, { "id": "57207", "postDate": "11/02/2014 19:03:30", "content": "

Arno What was the wall time for training?

", "rawMarkdown": "", "votes": null }, { "id": "57212", "postDate": "11/02/2014 19:18:13", "content": "

[quote=Inspector;57202]

\n

Do you just mean to use those commands as a general 'clean up' 

\n

[/quote]

\n

Yes, that precisely, I meant it as a general clean up. These commands are useful to list and remove files you will no longer need from the server. 

", "rawMarkdown": "", "votes": null }, { "id": "57215", "postDate": "11/02/2014 20:13:11", "content": "

wacax - thanks for helping out.  Yes, you can launch H2O from within R, but I prefer to launch H2O myself on the server, as you have access to more options that way (e.g., you currently can't change the factor level limit default of 65k from R's h2o.init() function, but we will probably fix that soon):

\n\n

Inspector - I left a comment in the code that explains what to run on the server, it also tells you that you get access to a GUI via a web browser:

\n

## Connect to H2O server (On server(s), run 'java -jar h2o.jar -Xmx32G -port 53322 -name CTR -data_max_factor_levels 100000000' first)
## Go to http://server:53322/ to check Jobs/Data/Models etc.
h2oServer <- h2o.init(ip="mr-0xd2", port = 53322)

\n\n

WeDontKnowWhatWeAreDoing - it took maybe 1 hour (or less) on a 16-core server with -Xmx64G, but most time was spent in the factor-to-factor interaction terms, as they all become (materialized) new features of the enriched data set.  If you enable feature importances (for RF, it's importances=T, you can later query them from the model or see a chart from the Web UI), then you can see how important those new features are, which is nice.

", "rawMarkdown": "", "votes": null }, { "id": "57221", "postDate": "11/03/2014 00:20:29", "content": "

Arno, awesome setup, thanks very much for passing it along! Thought I'd share this, obviously not as clutch as arno's full script but at the end you can solve the ID issue in the submission file by reading it in as character and then writing out with quote = F as follows:

\n


submission <- read.csv(path_submission, colClasses = c("character"))
submission[,2] <- as.data.frame(pred)
colnames(submission) <- c("id", "click")
write.csv(as.data.frame(submission), file = paste(path,"/data/submission.csv", sep = ''), quote = F, row.names = F)

", "rawMarkdown": "", "votes": null }, { "id": "57224", "postDate": "11/03/2014 01:23:33", "content": "

Thanks Joe - I updated the script on github!

", "rawMarkdown": "", "votes": null }, { "id": "57299", "postDate": "11/04/2014 15:14:04", "content": "

Thanks Arno, it's looking good.

\n

I wonder if you can shed any light on this - I'm starting up directly in java, and trying to give it 64G or 128G (the server has 250G or so).  When it starts, it reports Java heap maxMemory: 26.52 gb even though it recognises the full system RAM.  When it is processing it seems to max out at about 30G RAM for the java process, and goes into swapping from time to time.

\n

I've had a couple of time-outs as well when it loses connection with httpd, and I've had to start from scratch, but so far I am up to 'Splitting into train/validation' and keeping fingers crossed!

", "rawMarkdown": "", "votes": null }, { "id": "57314", "postDate": "11/04/2014 19:29:10", "content": "

Jay - Please try to launch H2O via "java -Xmx128g -ea -jar h2o.jar -name NAME -data_max_factor_levels 100000000 -baseport PORT", that should work, I use this all the time:

\n

R is connected to H2O cluster:
H2O cluster uptime: 6 seconds 387 milliseconds
H2O cluster version: 2.9.0.99999
H2O cluster name: ArnoDemo
H2O cluster total nodes: 10
H2O cluster total memory: 1137.78 GB
H2O cluster total cores: 320
H2O cluster allowed cores: 320
H2O cluster healthy: TRUE

\n

I usually launch R on (one of) the server node(s) itself (in a back-grounded 'screen' session), so that the REST call traffic takes less time, especially with so many factor levels (which all get sent back in JSON format from the server to R when the model is finished).

\n

Hope this helps,
Arno

", "rawMarkdown": "", "votes": null }, { "id": "57332", "postDate": "11/05/2014 03:22:47", "content": "

Hi Arno,

\n

I'm running a locally with <snip>

\n

h2oServer <- h2o.init(nthreads = 12, max_mem_size = '30G')

\n

I start running everything and get into the feature engineering phase, and then error.

\n

Feature engineering for factor columns.

\n

Error in  .h2o__remoteSend(client, .h2o.__PAGE_EXEC2, str = expr) 

\n

  http://127.0.0.1:54321/2/Exec2.json returned the following error: 

\n

    Frame is already locked by job null.

\n

Any ideas?

", "rawMarkdown": "", "votes": null }, { "id": "57375", "postDate": "11/06/2014 03:47:15", "content": "

TechnoJunkie -

\n

Maybe it has to do with not starting H2O manually with -data_max_factor_levels 100000000 and some columns get parsed as all missing values (default limit of factor levels is 65k).

\n

You can you look at the log files from the browser:

\n

http://localhost:54321/LogView.html

\n

If you prefer, you can also download them there and send them to me (support@0xdata.com).

\n

This error message happens when an R expression fails and leaves some temporary vector in an invalid state.  Our next version of H2O (h2o-dev) will have improved error reporting...

\n

Thanks,

\n

Arno

", "rawMarkdown": "", "votes": null }, { "id": "57396", "postDate": "11/06/2014 10:52:50", "content": "

from where is the function h2o.interaction?

\n

I got the following message Error: could not find function "h2o.interaction"

\n

I can't find the function in the package. 

", "rawMarkdown": "", "votes": null }, { "id": "57431", "postDate": "11/06/2014 19:50:25", "content": "

Van de Rakt - You probably have to upgrade your H2O installation:

\n\n

devtools::install_github("woobe/deepr")
deepr::install_h2o()

\n\n

Should do the trick.

", "rawMarkdown": "", "votes": null }, { "id": "57459", "postDate": "11/07/2014 09:34:51", "content": "

-- this question was answered above.

", "rawMarkdown": "", "votes": null }, { "id": "57482", "postDate": "11/07/2014 19:13:58", "content": "

@Arno, thanks for the code. I am getting an error at following line:

\n

train_hex <- h2o.addNewFeatures(train_hex, "hour", intcols, factorcols, "train_hex")

\n

## ERROR

\n

Error in .h2o.__remoteSend(client, .h2o.__PAGE_EXEC2, str = expr) :
http://localhost:54321/2/Exec2.json returned the following error:
Unknown var train_rev2.hex
Last.value.2 = train_rev2.hex[,c(3)]

\n\n

I think this is because of the class of train_hex namely H2OParsedData. It is not in usual data frame kind of thing to run a function.

\n

Can somebody help me with this.

", "rawMarkdown": "", "votes": null }, { "id": "57487", "postDate": "11/07/2014 20:23:25", "content": "

Ankit - I need to see more of your code.  Did you make modifications?  Does the original code that I posted work for you?

\n

The H2OParsedData object behaves very similarly to a regular data frame in R, but all the data is on the H2O server(s), and not in R.

\n

The error seems to suggest that train_hex wasn't found.  Can you run summary(train_hex) before you call the addnewfeatures() call?

\n

You can also inspect the H2O in-memory store at http://localhost:54321/StoreView.html, or look at the Log (see my post above).

\n

Arno

", "rawMarkdown": "", "votes": null }, { "id": "57489", "postDate": "11/07/2014 21:50:16", "content": "

Hey Arno,

\n

Thanks for the prompt reply. Appreciate it.

\n

I figured out the mistake. Initially I read the train, test files as they take a long time and saved the  workspace in R. But when I load that workspace again in a different R session it seems that I have to read the files again to run anything even if they show up in the environment window in R studio.

\n

Also, I have follow up question. I am using the following parameters.

\n

h2oserver = h2o.init(ip = "localhost", port = 54321, max_mem_size = '25g', nthreads = 12)

\n

However, my computer gets hanged even if I run the functions of feature engineering with the test data.

\n

Can you please suggest what is going on. 

\n

Thanks for help again.

", "rawMarkdown": "", "votes": null }, { "id": "57490", "postDate": "11/07/2014 21:57:28", "content": "

Ankit - 25g might not be enough to avoid swapping, so it is probably just very slow trying to dump gigabytes to a directory in /tmp (default location of the '--ice_root' option in the java -jar startup command line).  You can reduce the dataset size (see downsampling code below) or reduce the amount of feature engineering (fewer categorical interaction columns).

\n

random <- h2o.runif(train_hex, seed = 123456789)
train_hex_downsampled <- train_hex[random < .1,]

\n\n

Arno

", "rawMarkdown": "", "votes": null }, { "id": "57799", "postDate": "11/11/2014 14:40:06", "content": "

@Arno. Can you summarize the nature of the H2o algorithms in regards to memory use during model fit - does the data and processing have to fit within memory (like most functions in R) or will the models fit using a mini batch type of process? Basically asking if H2o allows training models on a laptop where the data set doesn't fit into memory? Thanks!

", "rawMarkdown": "", "votes": null }, { "id": "57900", "postDate": "11/12/2014 13:29:02", "content": "

I have a problem with reading the files correctly. More precisely, columns device_id and device_ip are read as NA. Anyone else experiencing the same problem?

", "rawMarkdown": "", "votes": null }, { "id": "57918", "postDate": "11/12/2014 15:36:38", "content": "

[quote=Matfyzak;57900]

\n

I have a problem with reading the files correctly. More precisely, columns device_id and device_ip are read as NA. Anyone else experiencing the same problem?

\n

[/quote]

\n\n

I am not sure, but it's possible no. of factor in device_id might be more than 65000. Default no. of factors to read in H2O is 65000. You can change it by starting H2O manually from command line. Refer to this

\n

http://www.kaggle.com/c/tradeshift-text-classification/forums/t/10665/beat-the-benchmark-with-h2o-distributed-random-forest   14th post 

", "rawMarkdown": "", "votes": null }, { "id": "57923", "postDate": "11/12/2014 16:36:23", "content": "

Well, I ran the code as suggested - not initiated H2O from R, but initiated the server first via

\n

java -Xmx32g -ea -jar h2o.jar -port 53322 -name CTR -data_max_factor_levels 100000000

\n

so the number of factors should not be an issue. H2O interprets columns device_id and device_ip as a numeric variable - there are hashes made just from numbers - and have problems switching to factor. However, I don't know why these two columns are a problem and others are not.

", "rawMarkdown": "", "votes": null }, { "id": "57943", "postDate": "11/12/2014 20:08:07", "content": "

Matfyzak

\n

Check these two things:

\n

1) You might have multiple instances of H2O running.  I'd double-check the H2O cluster name on http://server:53322/Cloud.html, or, even better check what R reported when you did the h2o.init(ip=..., port=...) call.  You can also kill all java processes to make sure nothing is running before you launch H2O.

\n

2) You need a new-enough version of H2O to pass the -data_max_factor_levels.

\n

Hope this helps,

\n

Arno

", "rawMarkdown": "", "votes": null }, { "id": "57988", "postDate": "11/13/2014 13:44:40", "content": "

Arno, thanks for suggestions. Unfortunately, the problem prevails.

\n

I have killed all Java processes, started H2O server and double checked that the H2O cluster name coincide with the one provided by h2o.init(ip=..., port=...). I have the newest stable version (Markov). Still no good for the two columns, others are fine.

\n

 I run H2O locally (I initiate it with ip="127.0.0.1") and thus access it at http://localhost:53322/.

", "rawMarkdown": "", "votes": null }, { "id": "58021", "postDate": "11/13/2014 18:34:31", "content": "

Markov just missed this feature.  Please try this version:

\n

http://h2o-release.s3.amazonaws.com/h2o/rel-maxwell/2/index.html

", "rawMarkdown": "", "votes": null }, { "id": "58077", "postDate": "11/14/2014 15:22:43", "content": "

Thanks a lot, Arno! Now it works just fine. :)

", "rawMarkdown": "", "votes": null }, { "id": "58265", "postDate": "11/18/2014 03:07:54", "content": "

When I start the h2o server, I get memory related restriction errors ( is there free vs commercial versions ??) :

\n

Invalid maximum heap size: -Xmx12g
The specified size exceeds the maximum representable size.
Error: Could not create the Java Virtual Machine.
Error: A fatal exception has occurred. Program will exit.

", "rawMarkdown": "", "votes": null }, { "id": "58266", "postDate": "11/18/2014 03:20:59", "content": "

saurk - that's likely a Java issue, has nothing to do with H2O.  What version of java are you using?  We recommend Oracle Java 1.7 or later.

", "rawMarkdown": "", "votes": null }, { "id": "58267", "postDate": "11/18/2014 03:48:28", "content": "

I think the java version seems OK. Interestingly restricting it to 1 Gig works fine , i.e.   -Xmx1g

\n

>java -version
java version "1.7.0_55"
Java(TM) SE Runtime Environment (build 1.7.0_55-b30)
Java HotSpot(TM) Client VM (build 24.55-b03, mixed mode, sharing)

", "rawMarkdown": "", "votes": null }, { "id": "58269", "postDate": "11/18/2014 04:03:58", "content": "

Hi Arno,

\n

You were right. I switched my Java to Server edition and now it works fine !

\n

Thanks, Will try it out !

", "rawMarkdown": "", "votes": null }, { "id": "58598", "postDate": "11/21/2014 06:23:03", "content": "

Hi Arno,

\n\n

New data released and im trying to import via h2o, with bad results.  The train dataset is forcing "site_category" to be NA for all values.

\n

str(train_hex)
Formal class 'H2OParsedData' [package "h2o"] with 7 slots
..@ h2o :Formal class 'H2OClient' [package "h2o"] with 2 slots
.. .. ..@ ip : chr "127.0.0.1"
.. .. ..@ port: num 54321
..@ key : chr "train.hex"
..@ logic : logi FALSE
..@ col_names: chr "id" "click" "hour" "C1" ...
..@ nrows : num 40428967
..@ ncols : num 24
..@ any_enum : logi TRUE

\n

H2O dataset 'train.hex': 40428967 obs. of 24 variables:
$ id : num ...
$ click : num ...
$ hour : num ...
$ C1 : num ...
$ banner_pos : num ...
$ site_id : Factor w/ 4737 levels "000aa1a4","00255fb4",..: ...
$ site_domain : Factor w/ 7745 levels "000129ff","0035f25a",..: ...
$ site_category : num ...
$ app_id : Factor w/ 8552 levels "000d6291","000f21f1",..: ...
$ app_domain : Factor w/ 559 levels "001b87ae","002e4064",..: ...
$ app_category : Factor w/ 36 levels "07d7df22","09481d60",..: ...
$ device_id : num ...
$ device_ip : num ...
$ device_model : Factor w/ 8251 levels "00097428","0009f4d7",..: ...
$ device_type : num ...
$ device_conn_type: num ...
$ C14 : num ...
$ C15 : num ...
$ C16 : num ...
$ C17 : num ...
$ C18 : num ...
$ C19 : num ...
$ C20 : num ...
$ C21 : num ...

\n\n

I have no issues with importing the test dataset however

\n

H2O dataset 'test.hex': 4577464 obs. of 23 variables:
$ id : num ...
$ hour : num ...
$ C1 : num ...
$ banner_pos : num ...
$ site_id : Factor w/ 2825 levels "00255fb4","003cf93d",..: ...
$ site_domain : Factor w/ 3366 levels "0045caf0","005b495a",..: ...
$ site_category : Factor w/ 22 levels "0569f928","28905ebd",..: ...
$ app_id : Factor w/ 3952 levels "000d6291","00222d0c",..: ...
$ app_domain : Factor w/ 201 levels "03da86e1","0654b444",..: ...
$ app_category : Factor w/ 28 levels "07d7df22","09481d60",..: ...
$ device_id : num ...
$ device_ip : num ...
$ device_model : Factor w/ 5438 levels "00097428","0009f4d7",..: ...
$ device_type : num ...
$ device_conn_type: num ...
$ C14 : num ...
$ C15 : num ...
$ C16 : num ...
$ C17 : num ...
$ C18 : num ...
$ C19 : num ...
$ C20 : num ...
$ C21 : num ...

\n

I've tried to use as.h2o as well to make sure the data is being imported correctly but end up with the same results. 

\n

Any suggestions?

\n

H2O version 1594.

\n

Java 1.7

", "rawMarkdown": "", "votes": null }, { "id": "58637", "postDate": "11/21/2014 21:20:49", "content": "

TechnoJunkie - Have you tried specifying -data_max_factor_levels during the start from the command line?

\n

I'm currently working on updating the starter script to the new dataset, so stay tuned.

", "rawMarkdown": "", "votes": null }, { "id": "58712", "postDate": "11/23/2014 02:59:24", "content": "

TechnoJunkie - More than 10% of values in the "site_category" column were actual numbers (such as 3e814130, arguably, with a huge exponent), and H2O converted the entire column into an int column. 

\n

I've relaxed this 10% threshold to 25%, so the latest build (http://s3.amazonaws.com/h2o-release/h2o/master/1597/index.html or later) should work for you.  Don't forget the -data_max_factor_levels for some of the other columns, of course.

\n

I'm still working on updating the script I provided above.

\n

Thanks,
Arno

", "rawMarkdown": "", "votes": null }, { "id": "58722", "postDate": "11/23/2014 08:59:09", "content": "

title of the post should be updated!

", "rawMarkdown": "", "votes": null }, { "id": "58749", "postDate": "11/23/2014 22:40:45", "content": "

Hi, Arno!

\n

For me it seems a bit strange that you have validation score better then training. 

\n

Is it true or just set names were mixed?

", "rawMarkdown": "", "votes": null }, { "id": "58757", "postDate": "11/24/2014 00:58:44", "content": "

Updated the script linked in the first post.  Our VPN is down, so I can't run anything until Monday or Tuesday.  Would love to see what the accuracy is if anyone volunteers to run this script :)

\n

Thanks,
Arno

\n\n

PS. Note the new training material at http://learn.h2o.ai with datasets at http://s3.h2o.ai

", "rawMarkdown": "", "votes": null }, { "id": "58760", "postDate": "11/24/2014 02:47:55", "content": "

Hi Arno,

\n

I'll try i tomorrow on a 48g server, 16 cores.

", "rawMarkdown": "", "votes": null }, { "id": "58773", "postDate": "11/24/2014 13:51:29", "content": "

So, neither 32g nor 48g are enough, it swaps to disk like crazy when building the interactions.

\n

I decided to drop the interactions piece, and am now running the code.

\n

Will report later with the results.

", "rawMarkdown": "", "votes": null }, { "id": "59117", "postDate": "11/27/2014 20:56:10", "content": "

Just ran the current version (0b3f98855) of the R script with GBM bernoulli distribution, got

\n

LogLoss on training data: 0.392463
LogLoss on validation data: 0.4027492
LB: 0.4033703

", "rawMarkdown": "", "votes": null }, { "id": "59133", "postDate": "11/28/2014 04:19:16", "content": "

[quote=Arno Candel;59117]

\n

Just ran the current version (0b3f98855) of the R script with GBM bernoulli distribution, got

\n

LogLoss on training data: 0.392463
LogLoss on validation data: 0.4027492
LB: 0.4033703

\n

[/quote]

\n

the discrepancy between LB and validation remains high ...

", "rawMarkdown": "", "votes": null }, { "id": "59135", "postDate": "11/28/2014 04:35:44", "content": "

Hi Arno,  Can we run the scripts across a cluster of 2 h2o servers with 32 GB each? Intending to try out on a cluster of 2 iMacs each with 32GB ram?  Please advise.  Thanks.  Anyway to reduce the memory requirements further? 

", "rawMarkdown": "", "votes": null }, { "id": "59139", "postDate": "11/28/2014 05:40:17", "content": "

Herimanitra - why do you think that?  0.4027492 vs 0.4033703 isn't that far off... Note that it's not N-fold cross-validation, just a simple 90/10 split based on time.

\n\n

Patrick Chan - Yes, you can definitely try that.  I also added some more h2o.rm() calls everywhere, so the latest version should be better already.  To further reduce the memory footprint, you can comment out the feature engineering for integer and factor columns, and just leave the feature engineering for the time column.  Interactions might not matter much to the tree-based methods anyway, since they come up with those on their own.

", "rawMarkdown": "", "votes": null }, { "id": "59141", "postDate": "11/28/2014 06:10:33", "content": "

Sorry :) I was seeing 0.39xx

\n

[quote=Arno Candel;59139]

\n

Herimanitra - why do you think that?  0.4027492 vs 0.4033703 isn't that far off... Note that it's not N-fold cross-validation, just a simple 90/10 split based on time.

\n

Patrick Chan - Yes, you can definitely try that.  I also added some more h2o.rm() calls everywhere, so the latest version should be better already.  To further reduce the memory footprint, you can comment out the feature engineering for integer and factor columns, and just leave the feature engineering for the time column.  Interactions might not matter much to the tree-based methods anyway, since they come up with those on their own.

\n

[/quote]

", "rawMarkdown": "", "votes": null }, { "id": "59347", "postDate": "12/01/2014 18:18:14", "content": "

Hi,

\n

Have you some explanation about the h2o.interaction(), or relevant documentation ?

\n

It seems very interesting to know what is done in this step. I didn't find something important about it.

\n

The script is stoping at this step with error.

\n

Thanks to Arno.

", "rawMarkdown": "", "votes": null }, { "id": "59349", "postDate": "12/01/2014 18:32:38", "content": "

khattab - you can find the description and example code with

\n

> help(h2o.interaction)

\n

It's probably using up too much memory creating those interactions, so either disable it or use java -Xmx64g or more.

", "rawMarkdown": "", "votes": null }, { "id": "59360", "postDate": "12/01/2014 22:04:56", "content": "

Hi, Arno

\n

Are there a way to do in h2o?:

\n

given two vectors obtain which are in first vector but not in the second vector

\n

(equivalent setdiff R function)

\n\n

And second question:

\n

Is possible to do a join of two h2oParsedData by a column or I need to send them to R?

", "rawMarkdown": "", "votes": null }, { "id": "59362", "postDate": "12/01/2014 22:23:36", "content": "

Jose - No, both things are not currently possible in H2O, but are on the road map (for h2o-dev).  You will need to do it in R for now.

", "rawMarkdown": "", "votes": null }, { "id": "62080", "postDate": "12/30/2014 04:14:40", "content": "

Hello everyone,
I have a question regarding the function h2o.kmeans. I am having trouble finding what centroids the training data were assigned to. In other words I cannot find the cluster assignments per observation

\n

According to the documentation found on: http://learn.h2o.ai/content/hands-on_training/kmeans_clustering.html and the examples in R this is the correct call:

\n

km.model <- h2o.kmeans(data = iris.h2o, centers = 5, cols = 1:4, init="furthest")

\n

km.model@model$centers # centroids
km.model@model$tot.withinss # total within cluster sum of squares
km.model@model$cluster # cluster assignments per observation

\n

However these are the only components in km.model:

\n

Available components:

\n

[1] "params" "centers" "withinss" "tot.withinss" "size" "iter"

\n

and the component cluster is missing.

\n

printing:

\n

print(km.model@model$cluster)

\n

returns:

\n

NULL

\n

If anyone can help me find said component will be of great help to me. Matching the data to each centroid is not an option as the data is too large.

\n

PD: it'd be useful too if someone told me where I could find information about other h2o users that can help me, such as a mailing list, a tag in cross-validated or stack-overflow. I looked around but I couldn't find it.

", "rawMarkdown": "", "votes": null }, { "id": "62113", "postDate": "12/30/2014 21:48:23", "content": "

Works for me. What version of H2O are you using?

\n

> library(h2o)
> h2oServer = h2o.init(nthreads=-1)
Successfully connected to http://127.0.0.1:54321R is connected to H2O cluster:
H2O cluster uptime: 6 seconds 568 milliseconds
H2O cluster version: 2.9.0.99999
H2O cluster name: arno
H2O cluster total nodes: 1
H2O cluster total memory: 7.11 GB
H2O cluster total cores: 8
H2O cluster allowed cores: 8
H2O cluster healthy: TRUE

\n

> datadir = "/data"
> homedir = file.path(datadir, "h2o-training", "clustering")
> iris.h2o = as.h2o(h2oServer, iris)
|============================================================================================================================| 100%
>
>
> ### Our first KMeans model
> km.model = h2o.kmeans(data = iris.h2o, centers = 5, cols = 1:4, init="furthest")
|============================================================================================================================| 100%
> ###### Let's look at the model summary:
> km.model
IP Address: 127.0.0.1
Port : 54321
Parsed Data Key: Last.value.1K-Means Model Key: KMeans2_a82272b2617c36ffce89426894944f90K-means clustering with 5 clusters of sizes 50, 12, 39, 24, 25Cluster means:
Sepal.Length Sepal.Width Petal.Length Petal.Width
1 5.006000 3.428000 1.462000 0.246000
2 7.475000 3.125000 6.300000 2.050000
3 6.207692 2.853846 4.746154 1.564103
4 6.529167 3.058333 5.508333 2.162500
5 5.508000 2.600000 3.908000 1.204000Clustering vector:
Cluster ID
0 :50
2 :39
4 :25
3 :24
1 :12Within cluster sum of squares by cluster:
[1] 15.15100 4.65500 12.81128 5.46250 8.36640Available components:[1] "params" "cluster" "centers" "withinss" "tot.withinss" "size" "iter"
> km.model@model$centers # The centers for each cluster
Sepal.Length Sepal.Width Petal.Length Petal.Width
1 5.006000 3.428000 1.462000 0.246000
2 7.475000 3.125000 6.300000 2.050000
3 6.207692 2.853846 4.746154 1.564103
4 6.529167 3.058333 5.508333 2.162500
5 5.508000 2.600000 3.908000 1.204000
> km.model@model$tot.withinss # total within cluster sum of squares
[1] 46.44618
> km.model@model$cluster # cluster assignments per observation
IP Address: 127.0.0.1
Port : 54321
Parsed Data Key: KMeans2_a82272b2617c36ffce89426894944f90_clustersCluster.ID
1 0
2 0
3 0
4 0
5 0
6 0> summary(km.model@model$cluster)
Cluster ID
0 :50
2 :39
4 :25
3 :24
1 :12

\n

Also, you can use

\n

clusters.hex = h2o.predict(km.model,iris.h2o)

\n

to assign a centroid to each row of a given dataset ("scoring").

\n

Note that clusters.hex will be stored in the H2O cluster, and is only 1 column, so that should be no problem in terms of memory footprint.

\n

H2O Forum: https://groups.google.com/forum/#!forum/h2ostream

\n

More info:

\n

http://learn.h2o.ai and http://data.h2o.ai

\n

https://www.gitbook.com/@h2o

\n

http://www.slideshare.net/0xdata/presentations?order=popular

\n

http://docs.h2o.ai

", "rawMarkdown": "", "votes": null }, { "id": "62119", "postDate": "12/31/2014 04:44:55", "content": "

Thanks for the quick reply

\n

[quote=Arno Candel;62113]

\n

What version of H2O are you using?

\n

[/quote]

\n

I also tried updating to the newest version but I got the same result, it must be my computer's issue then. I am using: Package h2o version 2.9.0.1641

\n

this is the output I get from running the example

\n

Still no trace of cluster. Neither in my example nor in the dataset I am trying to use kmeans on.

\n

[quote=Arno Candel;62113]

\n

Also, you can use

\n

clusters.hex = h2o.predict(km.model,iris.h2o)

\n

to assign a centroid to each row of a given dataset ("scoring"). 

\n

[/quote]

\n

Yes, that is actually a better alternative since I could use a smaller (but significant) subset and then assign centroids to the whole dataset, This fixes my problem at hand.

\n

Thank you Arno for the answer and the links, you rock!

", "rawMarkdown": "", "votes": null }, { "id": "62253", "postDate": "01/04/2015 07:21:04", "content": "

Hi Arno,

\n

I've just installed H2o and explored the web ui like here:https://www.youtube.com/watch?v=HzXoDbpm33E&list=PLNtMya54qvOFQhSZ4IKKXRbMkyLMn0caa&index=5

\n

Given the following information, does it mean that I'm bounded by 3,46gb of RAM when using h2o?

\n

10:08:53.705 main INFO WATER: Java heap totalMemory: 0,23 gb
10:08:53.706 main INFO WATER: Java heap maxMemory: 3,46 gb
10:08:53.713 main INFO WATER: Java version: Java 1.8.0_25 (from Oracle Corporation)
10:08:53.713 main INFO WATER: OS version: Linux 3.13.0-43-generic (amd64)
10:08:53.938 main INFO WATER: Machine physical memory: 15,56 gb

", "rawMarkdown": "", "votes": null }, { "id": "62254", "postDate": "01/04/2015 08:17:08", "content": "

Herimanitra - The default might only use 4GB, but you can launch H2O with, e.g., "java -Xmx14g -jar h2o.jar -ea -name MyH2O" to allocate up to 14GB of heap memory for H2O on your 16GB machine.

", "rawMarkdown": "", "votes": null } ], "comments": [ { "id": 57185, "author_name": "mpearmain", "author_url": "", "post_date": "11/02/2014 10:50:24", "content": "

Thanks again Arno,

\n

Really appreciate the 'tutorial' approach on real data for h2o.

\n

Something rather nice for engineering features for binary data.

\n

https://www.rochester.edu/college/psc/signorino/research/Carter_Signorino_2010_PA.pdf

", "votes": null, "replies": [] }, { "id": 57189, "author_name": "inspector", "author_url": "", "post_date": "11/02/2014 13:35:31", "content": "

[quote=Arno Candel;57179]

\n

Here's starter R code that should get you to a LB score of 0.393!  Not sure whether there's a benchmark, but the title sounded good to me :)

\n\n

[/quote]

\n\n

I receive an error here:

\n

h2oServer <- h2o.init(ip="mr-0xd2", port = 53322)

\n\n

Silly question I am sure...but do we need to run on our own ip address? If so, will this code run locally and require massive amount of memory?

", "votes": null, "replies": [] }, { "id": 57199, "author_name": "wacaxx", "author_url": "", "post_date": "11/02/2014 17:28:54", "content": "

[quote=Inspector;57189]

\n

Silly question I am sure...but do we need to run on our own ip address? If so, will this code run locally and require massive amount of memory?

\n

[/quote]

\n

If you want to run the server in your computer you can simply type "localhost" and allocate a maximum amount of memory and number of cores when you start the h2o server like this:

\n

h2oServer <- h2o.init(ip = "localhost", port = 54321, max_mem_size = '5g', nthreads = 4)

\n

don't forget to use the commands: h2o.ls() and h2o.rm() to keep h2o's server data size within the maximum memory limits otherwise you will get a server / memory error

", "votes": null, "replies": [] }, { "id": 57202, "author_name": "inspector", "author_url": "", "post_date": "11/02/2014 18:01:30", "content": "

[quote=wacax;57199]

\n

If you want to run the server in your computer you can simple type "localhost" and allocate a maximum amount of memory and number of cores when you start the h2o server like this:

\n

h2oServer <- h2o.init(ip = "localhost", port = 54321, max_mem_size = '5g', nthreads = 4)

\n

don't forget to use the commands: h2o.ls() and h2o.rm() to keep h2o's server data size within the maximum memory limits otherwise you will get a server / memory error

\n

[/quote]

\n\n

Thank you. Do you just mean to use those commands as a general 'clean up' or do you need to somehow adjust the calls to the model function? Perhaps this is self evident if I knew more about H20 - if the algorithms can be used when the data is large relative to memory.

", "votes": null, "replies": [] }, { "id": 57207, "author_name": "lapolonio", "author_url": "", "post_date": "11/02/2014 19:03:30", "content": "

Arno What was the wall time for training?

", "votes": null, "replies": [] }, { "id": 57212, "author_name": "wacaxx", "author_url": "", "post_date": "11/02/2014 19:18:13", "content": "

[quote=Inspector;57202]

\n

Do you just mean to use those commands as a general 'clean up' 

\n

[/quote]

\n

Yes, that precisely, I meant it as a general clean up. These commands are useful to list and remove files you will no longer need from the server. 

", "votes": null, "replies": [] }, { "id": 57215, "author_name": "arnocandel", "author_url": "", "post_date": "11/02/2014 20:13:11", "content": "

wacax - thanks for helping out.  Yes, you can launch H2O from within R, but I prefer to launch H2O myself on the server, as you have access to more options that way (e.g., you currently can't change the factor level limit default of 65k from R's h2o.init() function, but we will probably fix that soon):

\n\n

Inspector - I left a comment in the code that explains what to run on the server, it also tells you that you get access to a GUI via a web browser:

\n

## Connect to H2O server (On server(s), run 'java -jar h2o.jar -Xmx32G -port 53322 -name CTR -data_max_factor_levels 100000000' first)
## Go to http://server:53322/ to check Jobs/Data/Models etc.
h2oServer <- h2o.init(ip="mr-0xd2", port = 53322)

\n\n

WeDontKnowWhatWeAreDoing - it took maybe 1 hour (or less) on a 16-core server with -Xmx64G, but most time was spent in the factor-to-factor interaction terms, as they all become (materialized) new features of the enriched data set.  If you enable feature importances (for RF, it's importances=T, you can later query them from the model or see a chart from the Web UI), then you can see how important those new features are, which is nice.

", "votes": null, "replies": [] }, { "id": 57221, "author_name": "joec33r", "author_url": "", "post_date": "11/03/2014 00:20:29", "content": "

Arno, awesome setup, thanks very much for passing it along! Thought I'd share this, obviously not as clutch as arno's full script but at the end you can solve the ID issue in the submission file by reading it in as character and then writing out with quote = F as follows:

\n


submission <- read.csv(path_submission, colClasses = c("character"))
submission[,2] <- as.data.frame(pred)
colnames(submission) <- c("id", "click")
write.csv(as.data.frame(submission), file = paste(path,"/data/submission.csv", sep = ''), quote = F, row.names = F)

", "votes": null, "replies": [] }, { "id": 57224, "author_name": "arnocandel", "author_url": "", "post_date": "11/03/2014 01:23:33", "content": "

Thanks Joe - I updated the script on github!

", "votes": null, "replies": [] }, { "id": 57299, "author_name": "jaymoore", "author_url": "", "post_date": "11/04/2014 15:14:04", "content": "

Thanks Arno, it's looking good.

\n

I wonder if you can shed any light on this - I'm starting up directly in java, and trying to give it 64G or 128G (the server has 250G or so).  When it starts, it reports Java heap maxMemory: 26.52 gb even though it recognises the full system RAM.  When it is processing it seems to max out at about 30G RAM for the java process, and goes into swapping from time to time.

\n

I've had a couple of time-outs as well when it loses connection with httpd, and I've had to start from scratch, but so far I am up to 'Splitting into train/validation' and keeping fingers crossed!

", "votes": null, "replies": [] }, { "id": 57314, "author_name": "arnocandel", "author_url": "", "post_date": "11/04/2014 19:29:10", "content": "

Jay - Please try to launch H2O via "java -Xmx128g -ea -jar h2o.jar -name NAME -data_max_factor_levels 100000000 -baseport PORT", that should work, I use this all the time:

\n

R is connected to H2O cluster:
H2O cluster uptime: 6 seconds 387 milliseconds
H2O cluster version: 2.9.0.99999
H2O cluster name: ArnoDemo
H2O cluster total nodes: 10
H2O cluster total memory: 1137.78 GB
H2O cluster total cores: 320
H2O cluster allowed cores: 320
H2O cluster healthy: TRUE

\n

I usually launch R on (one of) the server node(s) itself (in a back-grounded 'screen' session), so that the REST call traffic takes less time, especially with so many factor levels (which all get sent back in JSON format from the server to R when the model is finished).

\n

Hope this helps,
Arno

", "votes": null, "replies": [] }, { "id": 57332, "author_name": "mpearmain", "author_url": "", "post_date": "11/05/2014 03:22:47", "content": "

Hi Arno,

\n

I'm running a locally with <snip>

\n

h2oServer <- h2o.init(nthreads = 12, max_mem_size = '30G')

\n

I start running everything and get into the feature engineering phase, and then error.

\n

Feature engineering for factor columns.

\n

Error in  .h2o__remoteSend(client, .h2o.__PAGE_EXEC2, str = expr) 

\n

  http://127.0.0.1:54321/2/Exec2.json returned the following error: 

\n

    Frame is already locked by job null.

\n

Any ideas?

", "votes": null, "replies": [] }, { "id": 57375, "author_name": "arnocandel", "author_url": "", "post_date": "11/06/2014 03:47:15", "content": "

TechnoJunkie -

\n

Maybe it has to do with not starting H2O manually with -data_max_factor_levels 100000000 and some columns get parsed as all missing values (default limit of factor levels is 65k).

\n

You can you look at the log files from the browser:

\n

http://localhost:54321/LogView.html

\n

If you prefer, you can also download them there and send them to me (support@0xdata.com).

\n

This error message happens when an R expression fails and leaves some temporary vector in an invalid state.  Our next version of H2O (h2o-dev) will have improved error reporting...

\n

Thanks,

\n

Arno

", "votes": null, "replies": [] }, { "id": 57396, "author_name": "vanderakt", "author_url": "", "post_date": "11/06/2014 10:52:50", "content": "

from where is the function h2o.interaction?

\n

I got the following message Error: could not find function "h2o.interaction"

\n

I can't find the function in the package. 

", "votes": null, "replies": [] }, { "id": 57431, "author_name": "arnocandel", "author_url": "", "post_date": "11/06/2014 19:50:25", "content": "

Van de Rakt - You probably have to upgrade your H2O installation:

\n\n

devtools::install_github("woobe/deepr")
deepr::install_h2o()

\n\n

Should do the trick.

", "votes": null, "replies": [] }, { "id": 57459, "author_name": "clintonboys", "author_url": "", "post_date": "11/07/2014 09:34:51", "content": "

-- this question was answered above.

", "votes": null, "replies": [] }, { "id": 57482, "author_name": "ankitj", "author_url": "", "post_date": "11/07/2014 19:13:58", "content": "

@Arno, thanks for the code. I am getting an error at following line:

\n

train_hex <- h2o.addNewFeatures(train_hex, "hour", intcols, factorcols, "train_hex")

\n

## ERROR

\n

Error in .h2o.__remoteSend(client, .h2o.__PAGE_EXEC2, str = expr) :
http://localhost:54321/2/Exec2.json returned the following error:
Unknown var train_rev2.hex
Last.value.2 = train_rev2.hex[,c(3)]

\n\n

I think this is because of the class of train_hex namely H2OParsedData. It is not in usual data frame kind of thing to run a function.

\n

Can somebody help me with this.

", "votes": null, "replies": [] }, { "id": 57487, "author_name": "arnocandel", "author_url": "", "post_date": "11/07/2014 20:23:25", "content": "

Ankit - I need to see more of your code.  Did you make modifications?  Does the original code that I posted work for you?

\n

The H2OParsedData object behaves very similarly to a regular data frame in R, but all the data is on the H2O server(s), and not in R.

\n

The error seems to suggest that train_hex wasn't found.  Can you run summary(train_hex) before you call the addnewfeatures() call?

\n

You can also inspect the H2O in-memory store at http://localhost:54321/StoreView.html, or look at the Log (see my post above).

\n

Arno

", "votes": null, "replies": [] }, { "id": 57489, "author_name": "ankitj", "author_url": "", "post_date": "11/07/2014 21:50:16", "content": "

Hey Arno,

\n

Thanks for the prompt reply. Appreciate it.

\n

I figured out the mistake. Initially I read the train, test files as they take a long time and saved the  workspace in R. But when I load that workspace again in a different R session it seems that I have to read the files again to run anything even if they show up in the environment window in R studio.

\n

Also, I have follow up question. I am using the following parameters.

\n

h2oserver = h2o.init(ip = "localhost", port = 54321, max_mem_size = '25g', nthreads = 12)

\n

However, my computer gets hanged even if I run the functions of feature engineering with the test data.

\n

Can you please suggest what is going on. 

\n

Thanks for help again.

", "votes": null, "replies": [] }, { "id": 57490, "author_name": "arnocandel", "author_url": "", "post_date": "11/07/2014 21:57:28", "content": "

Ankit - 25g might not be enough to avoid swapping, so it is probably just very slow trying to dump gigabytes to a directory in /tmp (default location of the '--ice_root' option in the java -jar startup command line).  You can reduce the dataset size (see downsampling code below) or reduce the amount of feature engineering (fewer categorical interaction columns).

\n

random <- h2o.runif(train_hex, seed = 123456789)
train_hex_downsampled <- train_hex[random < .1,]

\n\n

Arno

", "votes": null, "replies": [] }, { "id": 57799, "author_name": "inspector", "author_url": "", "post_date": "11/11/2014 14:40:06", "content": "

@Arno. Can you summarize the nature of the H2o algorithms in regards to memory use during model fit - does the data and processing have to fit within memory (like most functions in R) or will the models fit using a mini batch type of process? Basically asking if H2o allows training models on a laptop where the data set doesn't fit into memory? Thanks!

", "votes": null, "replies": [] }, { "id": 57900, "author_name": "lukasdrapal", "author_url": "", "post_date": "11/12/2014 13:29:02", "content": "

I have a problem with reading the files correctly. More precisely, columns device_id and device_ip are read as NA. Anyone else experiencing the same problem?

", "votes": null, "replies": [] }, { "id": 57918, "author_name": "thakurrajanand", "author_url": "", "post_date": "11/12/2014 15:36:38", "content": "

[quote=Matfyzak;57900]

\n

I have a problem with reading the files correctly. More precisely, columns device_id and device_ip are read as NA. Anyone else experiencing the same problem?

\n

[/quote]

\n\n

I am not sure, but it's possible no. of factor in device_id might be more than 65000. Default no. of factors to read in H2O is 65000. You can change it by starting H2O manually from command line. Refer to this

\n

http://www.kaggle.com/c/tradeshift-text-classification/forums/t/10665/beat-the-benchmark-with-h2o-distributed-random-forest   14th post 

", "votes": null, "replies": [] }, { "id": 57923, "author_name": "lukasdrapal", "author_url": "", "post_date": "11/12/2014 16:36:23", "content": "

Well, I ran the code as suggested - not initiated H2O from R, but initiated the server first via

\n

java -Xmx32g -ea -jar h2o.jar -port 53322 -name CTR -data_max_factor_levels 100000000

\n

so the number of factors should not be an issue. H2O interprets columns device_id and device_ip as a numeric variable - there are hashes made just from numbers - and have problems switching to factor. However, I don't know why these two columns are a problem and others are not.

", "votes": null, "replies": [] }, { "id": 57943, "author_name": "arnocandel", "author_url": "", "post_date": "11/12/2014 20:08:07", "content": "

Matfyzak

\n

Check these two things:

\n

1) You might have multiple instances of H2O running.  I'd double-check the H2O cluster name on http://server:53322/Cloud.html, or, even better check what R reported when you did the h2o.init(ip=..., port=...) call.  You can also kill all java processes to make sure nothing is running before you launch H2O.

\n

2) You need a new-enough version of H2O to pass the -data_max_factor_levels.

\n

Hope this helps,

\n

Arno

", "votes": null, "replies": [] }, { "id": 57988, "author_name": "lukasdrapal", "author_url": "", "post_date": "11/13/2014 13:44:40", "content": "

Arno, thanks for suggestions. Unfortunately, the problem prevails.

\n

I have killed all Java processes, started H2O server and double checked that the H2O cluster name coincide with the one provided by h2o.init(ip=..., port=...). I have the newest stable version (Markov). Still no good for the two columns, others are fine.

\n

 I run H2O locally (I initiate it with ip="127.0.0.1") and thus access it at http://localhost:53322/.

", "votes": null, "replies": [] }, { "id": 58021, "author_name": "arnocandel", "author_url": "", "post_date": "11/13/2014 18:34:31", "content": "

Markov just missed this feature.  Please try this version:

\n

http://h2o-release.s3.amazonaws.com/h2o/rel-maxwell/2/index.html

", "votes": null, "replies": [] }, { "id": 58077, "author_name": "lukasdrapal", "author_url": "", "post_date": "11/14/2014 15:22:43", "content": "

Thanks a lot, Arno! Now it works just fine. :)

", "votes": null, "replies": [] }, { "id": 58265, "author_name": "zsaurk", "author_url": "", "post_date": "11/18/2014 03:07:54", "content": "

When I start the h2o server, I get memory related restriction errors ( is there free vs commercial versions ??) :

\n

Invalid maximum heap size: -Xmx12g
The specified size exceeds the maximum representable size.
Error: Could not create the Java Virtual Machine.
Error: A fatal exception has occurred. Program will exit.

", "votes": null, "replies": [] }, { "id": 58266, "author_name": "arnocandel", "author_url": "", "post_date": "11/18/2014 03:20:59", "content": "

saurk - that's likely a Java issue, has nothing to do with H2O.  What version of java are you using?  We recommend Oracle Java 1.7 or later.

", "votes": null, "replies": [] }, { "id": 58267, "author_name": "zsaurk", "author_url": "", "post_date": "11/18/2014 03:48:28", "content": "

I think the java version seems OK. Interestingly restricting it to 1 Gig works fine , i.e.   -Xmx1g

\n

>java -version
java version "1.7.0_55"
Java(TM) SE Runtime Environment (build 1.7.0_55-b30)
Java HotSpot(TM) Client VM (build 24.55-b03, mixed mode, sharing)

", "votes": null, "replies": [] }, { "id": 58269, "author_name": "zsaurk", "author_url": "", "post_date": "11/18/2014 04:03:58", "content": "

Hi Arno,

\n

You were right. I switched my Java to Server edition and now it works fine !

\n

Thanks, Will try it out !

", "votes": null, "replies": [] }, { "id": 58598, "author_name": "mpearmain", "author_url": "", "post_date": "11/21/2014 06:23:03", "content": "

Hi Arno,

\n\n

New data released and im trying to import via h2o, with bad results.  The train dataset is forcing "site_category" to be NA for all values.

\n

str(train_hex)
Formal class 'H2OParsedData' [package "h2o"] with 7 slots
..@ h2o :Formal class 'H2OClient' [package "h2o"] with 2 slots
.. .. ..@ ip : chr "127.0.0.1"
.. .. ..@ port: num 54321
..@ key : chr "train.hex"
..@ logic : logi FALSE
..@ col_names: chr "id" "click" "hour" "C1" ...
..@ nrows : num 40428967
..@ ncols : num 24
..@ any_enum : logi TRUE

\n

H2O dataset 'train.hex': 40428967 obs. of 24 variables:
$ id : num ...
$ click : num ...
$ hour : num ...
$ C1 : num ...
$ banner_pos : num ...
$ site_id : Factor w/ 4737 levels "000aa1a4","00255fb4",..: ...
$ site_domain : Factor w/ 7745 levels "000129ff","0035f25a",..: ...
$ site_category : num ...
$ app_id : Factor w/ 8552 levels "000d6291","000f21f1",..: ...
$ app_domain : Factor w/ 559 levels "001b87ae","002e4064",..: ...
$ app_category : Factor w/ 36 levels "07d7df22","09481d60",..: ...
$ device_id : num ...
$ device_ip : num ...
$ device_model : Factor w/ 8251 levels "00097428","0009f4d7",..: ...
$ device_type : num ...
$ device_conn_type: num ...
$ C14 : num ...
$ C15 : num ...
$ C16 : num ...
$ C17 : num ...
$ C18 : num ...
$ C19 : num ...
$ C20 : num ...
$ C21 : num ...

\n\n

I have no issues with importing the test dataset however

\n

H2O dataset 'test.hex': 4577464 obs. of 23 variables:
$ id : num ...
$ hour : num ...
$ C1 : num ...
$ banner_pos : num ...
$ site_id : Factor w/ 2825 levels "00255fb4","003cf93d",..: ...
$ site_domain : Factor w/ 3366 levels "0045caf0","005b495a",..: ...
$ site_category : Factor w/ 22 levels "0569f928","28905ebd",..: ...
$ app_id : Factor w/ 3952 levels "000d6291","00222d0c",..: ...
$ app_domain : Factor w/ 201 levels "03da86e1","0654b444",..: ...
$ app_category : Factor w/ 28 levels "07d7df22","09481d60",..: ...
$ device_id : num ...
$ device_ip : num ...
$ device_model : Factor w/ 5438 levels "00097428","0009f4d7",..: ...
$ device_type : num ...
$ device_conn_type: num ...
$ C14 : num ...
$ C15 : num ...
$ C16 : num ...
$ C17 : num ...
$ C18 : num ...
$ C19 : num ...
$ C20 : num ...
$ C21 : num ...

\n

I've tried to use as.h2o as well to make sure the data is being imported correctly but end up with the same results. 

\n

Any suggestions?

\n

H2O version 1594.

\n

Java 1.7

", "votes": null, "replies": [] }, { "id": 58637, "author_name": "arnocandel", "author_url": "", "post_date": "11/21/2014 21:20:49", "content": "

TechnoJunkie - Have you tried specifying -data_max_factor_levels during the start from the command line?

\n

I'm currently working on updating the starter script to the new dataset, so stay tuned.

", "votes": null, "replies": [] }, { "id": 58712, "author_name": "arnocandel", "author_url": "", "post_date": "11/23/2014 02:59:24", "content": "

TechnoJunkie - More than 10% of values in the "site_category" column were actual numbers (such as 3e814130, arguably, with a huge exponent), and H2O converted the entire column into an int column. 

\n

I've relaxed this 10% threshold to 25%, so the latest build (http://s3.amazonaws.com/h2o-release/h2o/master/1597/index.html or later) should work for you.  Don't forget the -data_max_factor_levels for some of the other columns, of course.

\n

I'm still working on updating the script I provided above.

\n

Thanks,
Arno

", "votes": null, "replies": [] }, { "id": 58722, "author_name": "herimanitra", "author_url": "", "post_date": "11/23/2014 08:59:09", "content": "

title of the post should be updated!

", "votes": null, "replies": [] }, { "id": 58749, "author_name": "mikhailtrofimov", "author_url": "", "post_date": "11/23/2014 22:40:45", "content": "

Hi, Arno!

\n

For me it seems a bit strange that you have validation score better then training. 

\n

Is it true or just set names were mixed?

", "votes": null, "replies": [] }, { "id": 58757, "author_name": "arnocandel", "author_url": "", "post_date": "11/24/2014 00:58:44", "content": "

Updated the script linked in the first post.  Our VPN is down, so I can't run anything until Monday or Tuesday.  Would love to see what the accuracy is if anyone volunteers to run this script :)

\n

Thanks,
Arno

\n\n

PS. Note the new training material at http://learn.h2o.ai with datasets at http://s3.h2o.ai

", "votes": null, "replies": [] }, { "id": 58760, "author_name": "tanopereira", "author_url": "", "post_date": "11/24/2014 02:47:55", "content": "

Hi Arno,

\n

I'll try i tomorrow on a 48g server, 16 cores.

", "votes": null, "replies": [] }, { "id": 58773, "author_name": "tanopereira", "author_url": "", "post_date": "11/24/2014 13:51:29", "content": "

So, neither 32g nor 48g are enough, it swaps to disk like crazy when building the interactions.

\n

I decided to drop the interactions piece, and am now running the code.

\n

Will report later with the results.

", "votes": null, "replies": [] }, { "id": 59117, "author_name": "arnocandel", "author_url": "", "post_date": "11/27/2014 20:56:10", "content": "

Just ran the current version (0b3f98855) of the R script with GBM bernoulli distribution, got

\n

LogLoss on training data: 0.392463
LogLoss on validation data: 0.4027492
LB: 0.4033703

", "votes": null, "replies": [] }, { "id": 59133, "author_name": "herimanitra", "author_url": "", "post_date": "11/28/2014 04:19:16", "content": "

[quote=Arno Candel;59117]

\n

Just ran the current version (0b3f98855) of the R script with GBM bernoulli distribution, got

\n

LogLoss on training data: 0.392463
LogLoss on validation data: 0.4027492
LB: 0.4033703

\n

[/quote]

\n

the discrepancy between LB and validation remains high ...

", "votes": null, "replies": [] }, { "id": 59135, "author_name": "drpatrickchan", "author_url": "", "post_date": "11/28/2014 04:35:44", "content": "

Hi Arno,  Can we run the scripts across a cluster of 2 h2o servers with 32 GB each? Intending to try out on a cluster of 2 iMacs each with 32GB ram?  Please advise.  Thanks.  Anyway to reduce the memory requirements further? 

", "votes": null, "replies": [] }, { "id": 59139, "author_name": "arnocandel", "author_url": "", "post_date": "11/28/2014 05:40:17", "content": "

Herimanitra - why do you think that?  0.4027492 vs 0.4033703 isn't that far off... Note that it's not N-fold cross-validation, just a simple 90/10 split based on time.

\n\n

Patrick Chan - Yes, you can definitely try that.  I also added some more h2o.rm() calls everywhere, so the latest version should be better already.  To further reduce the memory footprint, you can comment out the feature engineering for integer and factor columns, and just leave the feature engineering for the time column.  Interactions might not matter much to the tree-based methods anyway, since they come up with those on their own.

", "votes": null, "replies": [] }, { "id": 59141, "author_name": "herimanitra", "author_url": "", "post_date": "11/28/2014 06:10:33", "content": "

Sorry :) I was seeing 0.39xx

\n

[quote=Arno Candel;59139]

\n

Herimanitra - why do you think that?  0.4027492 vs 0.4033703 isn't that far off... Note that it's not N-fold cross-validation, just a simple 90/10 split based on time.

\n

Patrick Chan - Yes, you can definitely try that.  I also added some more h2o.rm() calls everywhere, so the latest version should be better already.  To further reduce the memory footprint, you can comment out the feature engineering for integer and factor columns, and just leave the feature engineering for the time column.  Interactions might not matter much to the tree-based methods anyway, since they come up with those on their own.

\n

[/quote]

", "votes": null, "replies": [] }, { "id": 59347, "author_name": "khattab34", "author_url": "", "post_date": "12/01/2014 18:18:14", "content": "

Hi,

\n

Have you some explanation about the h2o.interaction(), or relevant documentation ?

\n

It seems very interesting to know what is done in this step. I didn't find something important about it.

\n

The script is stoping at this step with error.

\n

Thanks to Arno.

", "votes": null, "replies": [] }, { "id": 59349, "author_name": "arnocandel", "author_url": "", "post_date": "12/01/2014 18:32:38", "content": "

khattab - you can find the description and example code with

\n

> help(h2o.interaction)

\n

It's probably using up too much memory creating those interactions, so either disable it or use java -Xmx64g or more.

", "votes": null, "replies": [] }, { "id": 59360, "author_name": "blindape", "author_url": "", "post_date": "12/01/2014 22:04:56", "content": "

Hi, Arno

\n

Are there a way to do in h2o?:

\n

given two vectors obtain which are in first vector but not in the second vector

\n

(equivalent setdiff R function)

\n\n

And second question:

\n

Is possible to do a join of two h2oParsedData by a column or I need to send them to R?

", "votes": null, "replies": [] }, { "id": 59362, "author_name": "arnocandel", "author_url": "", "post_date": "12/01/2014 22:23:36", "content": "

Jose - No, both things are not currently possible in H2O, but are on the road map (for h2o-dev).  You will need to do it in R for now.

", "votes": null, "replies": [] }, { "id": 62080, "author_name": "wacaxx", "author_url": "", "post_date": "12/30/2014 04:14:40", "content": "

Hello everyone,
I have a question regarding the function h2o.kmeans. I am having trouble finding what centroids the training data were assigned to. In other words I cannot find the cluster assignments per observation

\n

According to the documentation found on: http://learn.h2o.ai/content/hands-on_training/kmeans_clustering.html and the examples in R this is the correct call:

\n

km.model <- h2o.kmeans(data = iris.h2o, centers = 5, cols = 1:4, init="furthest")

\n

km.model@model$centers # centroids
km.model@model$tot.withinss # total within cluster sum of squares
km.model@model$cluster # cluster assignments per observation

\n

However these are the only components in km.model:

\n

Available components:

\n

[1] "params" "centers" "withinss" "tot.withinss" "size" "iter"

\n

and the component cluster is missing.

\n

printing:

\n

print(km.model@model$cluster)

\n

returns:

\n

NULL

\n

If anyone can help me find said component will be of great help to me. Matching the data to each centroid is not an option as the data is too large.

\n

PD: it'd be useful too if someone told me where I could find information about other h2o users that can help me, such as a mailing list, a tag in cross-validated or stack-overflow. I looked around but I couldn't find it.

", "votes": null, "replies": [] }, { "id": 62113, "author_name": "arnocandel", "author_url": "", "post_date": "12/30/2014 21:48:23", "content": "

Works for me. What version of H2O are you using?

\n

> library(h2o)
> h2oServer = h2o.init(nthreads=-1)
Successfully connected to http://127.0.0.1:54321R is connected to H2O cluster:
H2O cluster uptime: 6 seconds 568 milliseconds
H2O cluster version: 2.9.0.99999
H2O cluster name: arno
H2O cluster total nodes: 1
H2O cluster total memory: 7.11 GB
H2O cluster total cores: 8
H2O cluster allowed cores: 8
H2O cluster healthy: TRUE

\n

> datadir = "/data"
> homedir = file.path(datadir, "h2o-training", "clustering")
> iris.h2o = as.h2o(h2oServer, iris)
|============================================================================================================================| 100%
>
>
> ### Our first KMeans model
> km.model = h2o.kmeans(data = iris.h2o, centers = 5, cols = 1:4, init="furthest")
|============================================================================================================================| 100%
> ###### Let's look at the model summary:
> km.model
IP Address: 127.0.0.1
Port : 54321
Parsed Data Key: Last.value.1K-Means Model Key: KMeans2_a82272b2617c36ffce89426894944f90K-means clustering with 5 clusters of sizes 50, 12, 39, 24, 25Cluster means:
Sepal.Length Sepal.Width Petal.Length Petal.Width
1 5.006000 3.428000 1.462000 0.246000
2 7.475000 3.125000 6.300000 2.050000
3 6.207692 2.853846 4.746154 1.564103
4 6.529167 3.058333 5.508333 2.162500
5 5.508000 2.600000 3.908000 1.204000Clustering vector:
Cluster ID
0 :50
2 :39
4 :25
3 :24
1 :12Within cluster sum of squares by cluster:
[1] 15.15100 4.65500 12.81128 5.46250 8.36640Available components:[1] "params" "cluster" "centers" "withinss" "tot.withinss" "size" "iter"
> km.model@model$centers # The centers for each cluster
Sepal.Length Sepal.Width Petal.Length Petal.Width
1 5.006000 3.428000 1.462000 0.246000
2 7.475000 3.125000 6.300000 2.050000
3 6.207692 2.853846 4.746154 1.564103
4 6.529167 3.058333 5.508333 2.162500
5 5.508000 2.600000 3.908000 1.204000
> km.model@model$tot.withinss # total within cluster sum of squares
[1] 46.44618
> km.model@model$cluster # cluster assignments per observation
IP Address: 127.0.0.1
Port : 54321
Parsed Data Key: KMeans2_a82272b2617c36ffce89426894944f90_clustersCluster.ID
1 0
2 0
3 0
4 0
5 0
6 0> summary(km.model@model$cluster)
Cluster ID
0 :50
2 :39
4 :25
3 :24
1 :12

\n

Also, you can use

\n

clusters.hex = h2o.predict(km.model,iris.h2o)

\n

to assign a centroid to each row of a given dataset ("scoring").

\n

Note that clusters.hex will be stored in the H2O cluster, and is only 1 column, so that should be no problem in terms of memory footprint.

\n

H2O Forum: https://groups.google.com/forum/#!forum/h2ostream

\n

More info:

\n

http://learn.h2o.ai and http://data.h2o.ai

\n

https://www.gitbook.com/@h2o

\n

http://www.slideshare.net/0xdata/presentations?order=popular

\n

http://docs.h2o.ai

", "votes": null, "replies": [] }, { "id": 62119, "author_name": "wacaxx", "author_url": "", "post_date": "12/31/2014 04:44:55", "content": "

Thanks for the quick reply

\n

[quote=Arno Candel;62113]

\n

What version of H2O are you using?

\n

[/quote]

\n

I also tried updating to the newest version but I got the same result, it must be my computer's issue then. I am using: Package h2o version 2.9.0.1641

\n

this is the output I get from running the example

\n

Still no trace of cluster. Neither in my example nor in the dataset I am trying to use kmeans on.

\n

[quote=Arno Candel;62113]

\n

Also, you can use

\n

clusters.hex = h2o.predict(km.model,iris.h2o)

\n

to assign a centroid to each row of a given dataset ("scoring"). 

\n

[/quote]

\n

Yes, that is actually a better alternative since I could use a smaller (but significant) subset and then assign centroids to the whole dataset, This fixes my problem at hand.

\n

Thank you Arno for the answer and the links, you rock!

", "votes": null, "replies": [] }, { "id": 62253, "author_name": "herimanitra", "author_url": "", "post_date": "01/04/2015 07:21:04", "content": "

Hi Arno,

\n

I've just installed H2o and explored the web ui like here:https://www.youtube.com/watch?v=HzXoDbpm33E&list=PLNtMya54qvOFQhSZ4IKKXRbMkyLMn0caa&index=5

\n

Given the following information, does it mean that I'm bounded by 3,46gb of RAM when using h2o?

\n

10:08:53.705 main INFO WATER: Java heap totalMemory: 0,23 gb
10:08:53.706 main INFO WATER: Java heap maxMemory: 3,46 gb
10:08:53.713 main INFO WATER: Java version: Java 1.8.0_25 (from Oracle Corporation)
10:08:53.713 main INFO WATER: OS version: Linux 3.13.0-43-generic (amd64)
10:08:53.938 main INFO WATER: Machine physical memory: 15,56 gb

", "votes": null, "replies": [] }, { "id": 62254, "author_name": "arnocandel", "author_url": "", "post_date": "01/04/2015 08:17:08", "content": "

Herimanitra - The default might only use 4GB, but you can launch H2O with, e.g., "java -Xmx14g -jar h2o.jar -ea -name MyH2O" to allocate up to 14GB of heap memory for H2O on your 16GB machine.

", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "57179": "", "57185": "", "57189": "", "57199": "", "57202": "", "57207": "", "57212": "", "57215": "", "57221": "", "57224": "", "57299": "", "57314": "", "57332": "", "57375": "", "57396": "", "57431": "", "57459": "", "57482": "", "57487": "", "57489": "", "57490": "", "57799": "", "57900": "", "57918": "", "57923": "", "57943": "", "57988": "", "58021": "", "58077": "", "58265": "", "58266": "", "58267": "", "58269": "", "58598": "", "58637": "", "58712": "", "58722": "", "58749": "", "58757": "", "58760": "", "58773": "", "59117": "", "59133": "", "59135": "", "59139": "", "59141": "", "59347": "", "59349": "", "59360": "", "59362": "", "62080": "", "62113": "", "62119": "", "62253": "", "62254": "" }, "source": "meta" }