Here are the Top 300+ importance features using H2O GBM trained on AWS m4.2xlarge.\nAttached R script uses data.table package to select top 200K instances and 300+ features and trains a xgb model.
\n\nA mac with 8G RAM can run this script before ---make prediction part without memory problem. \nIn order to make prediction, if your laptop memory is limited and don't want to get a AWS EC2, you can use script provided by Michael J. Sabal to split the test data into 5 chunks, process data, make predictions and rbind them together.\nhttps://www.kaggle.com/c/bosch-production-line-performance/forums/t/23128/using-split.
\n\nThis model achieve 3-fold cv AUC 0.712, and use my naive threshold can get you public leaderboard 0.13261. I am pretty sure you can find a better way to find the threshold and further boost the PL scores.
", "messages": [ { "id": "132734", "postDate": "08/28/2016 20:16:06", "content": "Here are the Top 300+ importance features using H2O GBM trained on AWS m4.2xlarge.\nAttached R script uses data.table package to select top 200K instances and 300+ features and trains a xgb model.
\n\nA mac with 8G RAM can run this script before ---make prediction part without memory problem. \nIn order to make prediction, if your laptop memory is limited and don't want to get a AWS EC2, you can use script provided by Michael J. Sabal to split the test data into 5 chunks, process data, make predictions and rbind them together.\nhttps://www.kaggle.com/c/bosch-production-line-performance/forums/t/23128/using-split.
\n\nThis model achieve 3-fold cv AUC 0.712, and use my naive threshold can get you public leaderboard 0.13261. I am pretty sure you can find a better way to find the threshold and further boost the PL scores.
", "rawMarkdown": "Here are the Top 300+ importance features using H2O GBM trained on AWS m4.2xlarge.\r\nAttached R script uses data.table package to select top 200K instances and 300+ features and trains a xgb model. \r\n\r\nA mac with 8G RAM can run this script before ---make prediction part without memory problem. \r\nIn order to make prediction, if your laptop memory is limited and don't want to get a AWS EC2, you can use script provided by Michael J. Sabal to split the test data into 5 chunks, process data, make predictions and rbind them together.\r\nhttps://www.kaggle.com/c/bosch-production-line-performance/forums/t/23128/using-split.\r\n\r\nThis model achieve 3-fold cv AUC 0.712, and use my naive threshold can get you public leaderboard 0.13261. I am pretty sure you can find a better way to find the threshold and further boost the PL scores.", "votes": null }, { "id": "132743", "postDate": "08/28/2016 23:25:54", "content": "It's great to see Wisconsin kagglers. :-)
\n\n(I'm in Neenah)
", "rawMarkdown": "It's great to see Wisconsin kagglers. :-)\r\n\r\n(I'm in Neenah)", "votes": null }, { "id": "132752", "postDate": "08/29/2016 02:07:46", "content": "[quote=inversion;132743]
\n\nIt's great to see Wisconsin kagglers. :-)
\n\n(I'm in Neenah)
\n\n[/quote]
\n\nYeah, Nice to see you here
", "rawMarkdown": "[quote=inversion;132743]\r\n\r\nIt's great to see Wisconsin kagglers. :-)\r\n\r\n(I'm in Neenah)\r\n\r\n[/quote]\r\n\r\nYeah, Nice to see you here", "votes": null }, { "id": "133037", "postDate": "08/31/2016 01:13:45", "content": "Hi. It's interesting. I,m going to use some part of your scripts and your idea about selecting important features by H2O. Would you please guide me how I can give you the credits for this part ? I am recently active here and have no clue if I can use your code or not.
", "rawMarkdown": "Hi. It's interesting. I,m going to use some part of your scripts and your idea about selecting important features by H2O. Would you please guide me how I can give you the credits for this part ? I am recently active here and have no clue if I can use your code or not.", "votes": null }, { "id": "133270", "postDate": "09/01/2016 06:51:07", "content": "Haozhen, can you please post the script how to get to the 300+ best features. I use h2o package for the first time and have various problems with the server while using gbm.
\n\nTHX
", "rawMarkdown": "Haozhen, can you please post the script how to get to the 300+ best features. I use h2o package for the first time and have various problems with the server while using gbm.\r\n\r\nTHX", "votes": null }, { "id": "133279", "postDate": "09/01/2016 07:33:46", "content": "Hi Haozhen, are u interested in teaming up. I have master degree in CS and years of experience in manufacturing information system. I currently use GBM via h2o python. I'll deal with features and you can handle the model parameter and ensemble.
", "rawMarkdown": "Hi Haozhen, are u interested in teaming up. I have master degree in CS and years of experience in manufacturing information system. I currently use GBM via h2o python. I'll deal with features and you can handle the model parameter and ensemble.", "votes": null }, { "id": "133364", "postDate": "09/01/2016 17:32:02", "content": "[quote=MehdiFarhangian;133037]
\n\nHi. It's interesting. I,m going to use some part of your scripts and your idea about selecting important features by H2O. Would you please guide me how I can give you the credits for this part ? I am recently active here and have no clue if I can use your code or not.
\n\n[/quote]\nHonestly, I don't know either. I think as long as I make it public, you can use it.
", "rawMarkdown": "[quote=MehdiFarhangian;133037]\r\n\r\nHi. It's interesting. I,m going to use some part of your scripts and your idea about selecting important features by H2O. Would you please guide me how I can give you the credits for this part ? I am recently active here and have no clue if I can use your code or not. \r\n\r\n[/quote]\r\nHonestly, I don't know either. I think as long as I make it public, you can use it.", "votes": null }, { "id": "133367", "postDate": "09/01/2016 17:38:15", "content": "[quote=MCAG;133270]
\n\nHaozhen, can you please post the script how to get to the 300+ best features. I use h2o package for the first time and have various problems with the server while using gbm.
\n\nTHX
\n\n[/quote]
\n\nlibrary(h2o)
\nlibrary(data.table)
\nh2o.init(ip='localhost',port=54321,nthreads=-1,max_mem_size='16g')
train_numeric <- h2o.importFile(path = "/home/ubuntu/kaggle/Bosch/data/train_numeric.csv",destination_frame = "train_numeric.hex")
\ntrain_categorical <- h2o.importFile(path = "/home/ubuntu/kaggle/Bosch/data/train_categorical.csv",destination_frame = "train_categorical.hex")
\ntrain_date <- h2o.importFile(path = "/home/ubuntu/kaggle/Bosch/data/train_date.csv",destination_frame = "train_date.hex")
train_numeric[,"Response"] <- as.factor(train_numeric[,"Response"])
\ntrain <- h2o.cbind(train_numeric,train_categorical,train_date)
\ntrain_numeric_name <- colnames(train_numeric)[-c(1,dim(train_numeric)[2])]
\ntrain_categorical_name <- colnames(train_categorical)[-1]
\ntrain_date_name <- colnames(train_date)[-1]
\ntrain_name <- c(train_numeric_name,train_categorical_name,train_date_name)
gbm1 <- h2o.gbm(train_name,"Response",train,
\n ignore_const_cols = TRUE,
\n ntrees = 100,
\n max_depth = 5,
\n min_rows = 20,
\n learn_rate = 0.1,
\n sample_rate = 0.8,
\n col_sample_rate = 0.8,
\n balance_classes = FALSE,
\n seed = 2016,
\n nfolds = 3,
\n fold_assignment = "Stratified",
\n keep_cross_validation_predictions = TRUE,
\n keep_cross_validation_fold_assignment = TRUE,
\n score_tree_interval = 20,
\n stopping_rounds = 10,
\n stopping_metric = "AUC",
\n stopping_tolerance = 0.01
\n )
importance <- h2o.varimp(gbm1)
\nimportance <- as.data.table(importance)
\nimportance <- importance[scaled_importance>0]
[quote=addiction;133279]
\n\nHi Haozhen, are u interested in teaming up. I have master degree in CS and years of experience in manufacturing information system. I currently use GBM via h2o python. I'll deal with features and you can handle the model parameter and ensemble.
\n\n[/quote]\nHi,
\nI am working on the feature engineering part now and have many ideas to try. I prefer considering team up after the middle stage of the competition. I don't want to spend too much time on model parameter and ensemble in this stage.
\nThanks
Hi Haozhen,
\n\nThank you for sharing the code.
\n\ntrain <- sparse.model.matrix(~.-1,data=train_combine)\ndtrain <- xgb.DMatrix(data = train,label = label[,Response])\nI applied the code to the full dataset, the code runs fine overall but I noticed the variable "dtrain" is blank. \nIt is supposed to be like this or I am missing something here interms of its class etc.\nHere is how the sample output looks when i check whats inside dtrain.
\n\n>dtrain\n<pointer: 0x102294980\nattr(,"class")\n[1] "xgb.DMatrix"\nHi Haozhen,
\n\nThank you for sharing the code.
\n\ntrain <- sparse.model.matrix(~.-1,data=train_combine)\ndtrain <- xgb.DMatrix(data = train,label = label[,Response])\nI applied the code to the full dataset, the code runs fine overall but I noticed the variable "dtrain" is blank. \nIt is supposed to be like this or I am missing something here interms of its class etc.\nHere is how the sample output looks when i check whats inside dtrain.
\n\n>dtrain\n<pointer: 0x102294980\nattr(,"class")\n[1] "xgb.DMatrix"\n[/quote]
\n\nIt is a special class of xgboost, which is just a reference.
", "rawMarkdown": "[quote=TarunParmar;133855]\r\n\r\nHi Haozhen,\r\n\r\nThank you for sharing the code. \r\n\r\n train <- sparse.model.matrix(~.-1,data=train_combine)\r\n dtrain <- xgb.DMatrix(data = train,label = label[,Response])\r\n\r\nI applied the code to the full dataset, the code runs fine overall but I noticed the variable \"dtrain\" is blank. \r\nIt is supposed to be like this or I am missing something here interms of its class etc.\r\nHere is how the sample output looks when i check whats inside dtrain.\r\n\r\n >dtrain\r\n(I'm in Neenah)
", "votes": null, "replies": [] }, { "id": 132752, "author_name": "alanwuxm", "author_url": "", "post_date": "08/29/2016 02:07:46", "content": "[quote=inversion;132743]
\n\nIt's great to see Wisconsin kagglers. :-)
\n\n(I'm in Neenah)
\n\n[/quote]
\n\nYeah, Nice to see you here
", "votes": null, "replies": [] }, { "id": 133037, "author_name": "mehdifarhangian", "author_url": "", "post_date": "08/31/2016 01:13:45", "content": "Hi. It's interesting. I,m going to use some part of your scripts and your idea about selecting important features by H2O. Would you please guide me how I can give you the credits for this part ? I am recently active here and have no clue if I can use your code or not.
", "votes": null, "replies": [] }, { "id": 133270, "author_name": "andreasgruber", "author_url": "", "post_date": "09/01/2016 06:51:07", "content": "Haozhen, can you please post the script how to get to the 300+ best features. I use h2o package for the first time and have various problems with the server while using gbm.
\n\nTHX
", "votes": null, "replies": [] }, { "id": 133279, "author_name": "addiction", "author_url": "", "post_date": "09/01/2016 07:33:46", "content": "Hi Haozhen, are u interested in teaming up. I have master degree in CS and years of experience in manufacturing information system. I currently use GBM via h2o python. I'll deal with features and you can handle the model parameter and ensemble.
", "votes": null, "replies": [] }, { "id": 133364, "author_name": "alanwuxm", "author_url": "", "post_date": "09/01/2016 17:32:02", "content": "[quote=MehdiFarhangian;133037]
\n\nHi. It's interesting. I,m going to use some part of your scripts and your idea about selecting important features by H2O. Would you please guide me how I can give you the credits for this part ? I am recently active here and have no clue if I can use your code or not.
\n\n[/quote]\nHonestly, I don't know either. I think as long as I make it public, you can use it.
", "votes": null, "replies": [] }, { "id": 133367, "author_name": "alanwuxm", "author_url": "", "post_date": "09/01/2016 17:38:15", "content": "[quote=MCAG;133270]
\n\nHaozhen, can you please post the script how to get to the 300+ best features. I use h2o package for the first time and have various problems with the server while using gbm.
\n\nTHX
\n\n[/quote]
\n\nlibrary(h2o)
\nlibrary(data.table)
\nh2o.init(ip='localhost',port=54321,nthreads=-1,max_mem_size='16g')
train_numeric <- h2o.importFile(path = "/home/ubuntu/kaggle/Bosch/data/train_numeric.csv",destination_frame = "train_numeric.hex")
\ntrain_categorical <- h2o.importFile(path = "/home/ubuntu/kaggle/Bosch/data/train_categorical.csv",destination_frame = "train_categorical.hex")
\ntrain_date <- h2o.importFile(path = "/home/ubuntu/kaggle/Bosch/data/train_date.csv",destination_frame = "train_date.hex")
train_numeric[,"Response"] <- as.factor(train_numeric[,"Response"])
\ntrain <- h2o.cbind(train_numeric,train_categorical,train_date)
\ntrain_numeric_name <- colnames(train_numeric)[-c(1,dim(train_numeric)[2])]
\ntrain_categorical_name <- colnames(train_categorical)[-1]
\ntrain_date_name <- colnames(train_date)[-1]
\ntrain_name <- c(train_numeric_name,train_categorical_name,train_date_name)
gbm1 <- h2o.gbm(train_name,"Response",train,
\n ignore_const_cols = TRUE,
\n ntrees = 100,
\n max_depth = 5,
\n min_rows = 20,
\n learn_rate = 0.1,
\n sample_rate = 0.8,
\n col_sample_rate = 0.8,
\n balance_classes = FALSE,
\n seed = 2016,
\n nfolds = 3,
\n fold_assignment = "Stratified",
\n keep_cross_validation_predictions = TRUE,
\n keep_cross_validation_fold_assignment = TRUE,
\n score_tree_interval = 20,
\n stopping_rounds = 10,
\n stopping_metric = "AUC",
\n stopping_tolerance = 0.01
\n )
importance <- h2o.varimp(gbm1)
\nimportance <- as.data.table(importance)
\nimportance <- importance[scaled_importance>0]
[quote=addiction;133279]
\n\nHi Haozhen, are u interested in teaming up. I have master degree in CS and years of experience in manufacturing information system. I currently use GBM via h2o python. I'll deal with features and you can handle the model parameter and ensemble.
\n\n[/quote]\nHi,
\nI am working on the feature engineering part now and have many ideas to try. I prefer considering team up after the middle stage of the competition. I don't want to spend too much time on model parameter and ensemble in this stage.
\nThanks
Hi Haozhen,
\n\nThank you for sharing the code.
\n\ntrain <- sparse.model.matrix(~.-1,data=train_combine)\ndtrain <- xgb.DMatrix(data = train,label = label[,Response])\nI applied the code to the full dataset, the code runs fine overall but I noticed the variable "dtrain" is blank. \nIt is supposed to be like this or I am missing something here interms of its class etc.\nHere is how the sample output looks when i check whats inside dtrain.
\n\n>dtrain\n<pointer: 0x102294980\nattr(,"class")\n[1] "xgb.DMatrix"\n[quote=TarunParmar;133855]
\n\nHi Haozhen,
\n\nThank you for sharing the code.
\n\ntrain <- sparse.model.matrix(~.-1,data=train_combine)\ndtrain <- xgb.DMatrix(data = train,label = label[,Response])\nI applied the code to the full dataset, the code runs fine overall but I noticed the variable "dtrain" is blank. \nIt is supposed to be like this or I am missing something here interms of its class etc.\nHere is how the sample output looks when i check whats inside dtrain.
\n\n>dtrain\n<pointer: 0x102294980\nattr(,"class")\n[1] "xgb.DMatrix"\n[/quote]
\n\nIt is a special class of xgboost, which is just a reference.
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "132734": "Here are the Top 300+ importance features using H2O GBM trained on AWS m4.2xlarge.\r\nAttached R script uses data.table package to select top 200K instances and 300+ features and trains a xgb model. \r\n\r\nA mac with 8G RAM can run this script before ---make prediction part without memory problem. \r\nIn order to make prediction, if your laptop memory is limited and don't want to get a AWS EC2, you can use script provided by Michael J. Sabal to split the test data into 5 chunks, process data, make predictions and rbind them together.\r\nhttps://www.kaggle.com/c/bosch-production-line-performance/forums/t/23128/using-split.\r\n\r\nThis model achieve 3-fold cv AUC 0.712, and use my naive threshold can get you public leaderboard 0.13261. I am pretty sure you can find a better way to find the threshold and further boost the PL scores.", "132743": "It's great to see Wisconsin kagglers. :-)\r\n\r\n(I'm in Neenah)", "132752": "[quote=inversion;132743]\r\n\r\nIt's great to see Wisconsin kagglers. :-)\r\n\r\n(I'm in Neenah)\r\n\r\n[/quote]\r\n\r\nYeah, Nice to see you here", "133037": "Hi. It's interesting. I,m going to use some part of your scripts and your idea about selecting important features by H2O. Would you please guide me how I can give you the credits for this part ? I am recently active here and have no clue if I can use your code or not.", "133270": "Haozhen, can you please post the script how to get to the 300+ best features. I use h2o package for the first time and have various problems with the server while using gbm.\r\n\r\nTHX", "133279": "Hi Haozhen, are u interested in teaming up. I have master degree in CS and years of experience in manufacturing information system. I currently use GBM via h2o python. I'll deal with features and you can handle the model parameter and ensemble.", "133364": "[quote=MehdiFarhangian;133037]\r\n\r\nHi. It's interesting. I,m going to use some part of your scripts and your idea about selecting important features by H2O. Would you please guide me how I can give you the credits for this part ? I am recently active here and have no clue if I can use your code or not. \r\n\r\n[/quote]\r\nHonestly, I don't know either. I think as long as I make it public, you can use it.", "133367": "[quote=MCAG;133270]\r\n\r\nHaozhen, can you please post the script how to get to the 300+ best features. I use h2o package for the first time and have various problems with the server while using gbm.\r\n\r\nTHX\r\n\r\n[/quote]\r\n\r\nlibrary(h2o) \r\nlibrary(data.table) \r\nh2o.init(ip='localhost',port=54321,nthreads=-1,max_mem_size='16g') \r\n\r\n#### load to H2O \r\ntrain_numeric <- h2o.importFile(path = \"/home/ubuntu/kaggle/Bosch/data/train_numeric.csv\",destination_frame = \"train_numeric.hex\") \r\ntrain_categorical <- h2o.importFile(path = \"/home/ubuntu/kaggle/Bosch/data/train_categorical.csv\",destination_frame = \"train_categorical.hex\") \r\ntrain_date <- h2o.importFile(path = \"/home/ubuntu/kaggle/Bosch/data/train_date.csv\",destination_frame = \"train_date.hex\")\r\n\r\ntrain_numeric[,\"Response\"] <- as.factor(train_numeric[,\"Response\"]) \r\ntrain <- h2o.cbind(train_numeric,train_categorical,train_date) \r\ntrain_numeric_name <- colnames(train_numeric)[-c(1,dim(train_numeric)[2])] \r\ntrain_categorical_name <- colnames(train_categorical)[-1] \r\ntrain_date_name <- colnames(train_date)[-1] \r\ntrain_name <- c(train_numeric_name,train_categorical_name,train_date_name) \r\n\r\n#### Train GBM model\r\ngbm1 <- h2o.gbm(train_name,\"Response\",train, \r\n ignore_const_cols = TRUE, \r\n ntrees = 100, \r\n max_depth = 5, \r\n min_rows = 20, \r\n learn_rate = 0.1, \r\n sample_rate = 0.8, \r\n col_sample_rate = 0.8, \r\n balance_classes = FALSE, \r\n seed = 2016, \r\n nfolds = 3, \r\n fold_assignment = \"Stratified\", \r\n keep_cross_validation_predictions = TRUE, \r\n keep_cross_validation_fold_assignment = TRUE, \r\n score_tree_interval = 20, \r\n stopping_rounds = 10, \r\n stopping_metric = \"AUC\", \r\n stopping_tolerance = 0.01 \r\n ) \r\n\r\nimportance <- h2o.varimp(gbm1) \r\nimportance <- as.data.table(importance) \r\nimportance <- importance[scaled_importance>0]", "133370": "[quote=addiction;133279]\r\n\r\nHi Haozhen, are u interested in teaming up. I have master degree in CS and years of experience in manufacturing information system. I currently use GBM via h2o python. I'll deal with features and you can handle the model parameter and ensemble. \r\n\r\n[/quote]\r\nHi, \r\nI am working on the feature engineering part now and have many ideas to try. I prefer considering team up after the middle stage of the competition. I don't want to spend too much time on model parameter and ensemble in this stage. \r\nThanks", "133855": "Hi Haozhen,\r\n\r\nThank you for sharing the code. \r\n\r\n train <- sparse.model.matrix(~.-1,data=train_combine)\r\n dtrain <- xgb.DMatrix(data = train,label = label[,Response])\r\n\r\nI applied the code to the full dataset, the code runs fine overall but I noticed the variable \"dtrain\" is blank. \r\nIt is supposed to be like this or I am missing something here interms of its class etc.\r\nHere is how the sample output looks when i check whats inside dtrain.\r\n\r\n >dtrain\r\n