{"cells":[{"metadata":{"_uuid":"89387fbdc0510b66c7186c1e7ab4d24a4c77244c","_cell_guid":"d219ef1c-2150-47e4-8a9a-3f8b2247a815"},"cell_type":"markdown","source":"# Digit Recognizer \n\"Each image is **28 pixels in height** and **28 pixels in width**, for a total of 784 pixels in total. Each pixel has a single pixel-value associated with it, indicating the lightness or darkness of that pixel, with higher numbers meaning darker. This pixel-value is an integer between 0 and 255, inclusive.\"\n\nSo each image have 28x28 pixels and a color on each pixel."},{"metadata":{"_uuid":"bde6bbe97933726719935de9ca911d05760fc0cb","_cell_guid":"7b79d1fd-35a5-4461-bf0a-b12d5337a438"},"cell_type":"markdown","source":"## Assign Necessary Libraries\n"},{"metadata":{"_uuid":"238fac25c47e292a0d93b1e8aa7837869374ddcc","trusted":false,"_cell_guid":"c4c3daa5-4e81-42af-9a0a-fcb8de196e4f"},"cell_type":"code","source":"# Necessary Libraries \n\nsuppressMessages(library(lattice))\nsuppressMessages(library(ggplot2))\nsuppressMessages(library(caret))\n\n# for describe the data\nsuppressMessages(library(psych))\n\n# install.packages(\"h2o\")\nsuppressMessages(library(h2o))","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"2d9e4ad1955c2bce00037610044129395a506272","_cell_guid":"5f1b33db-4089-4539-91e4-3c08090e3b7a"},"cell_type":"markdown","source":"## Loading data"},{"metadata":{"_uuid":"56abef5c2ca0222dff1caaf581f66930f00cc55c","trusted":false,"_cell_guid":"2731e6ed-5847-4ff0-b9a5-441c1363e09d"},"cell_type":"code","source":"# load train.csv\nDstTrain <- read.csv('../input/train.csv', stringsAsFactors = FALSE, na.strings = c(\"NA\", \"\"))\n# load test.csv\nDstTest  <- read.csv('../input/test.csv', stringsAsFactors = FALSE, na.strings = c(\"NA\", \"\"))","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"f3894f0ba6e056fc9eb0c27c9401b1e629a4d59d","_cell_guid":"d5f4e810-d95c-46bd-9ed1-eee629ac65ba"},"cell_type":"markdown","source":"## Checking the data\n\n**Check some info about structure**\n\nLet us explore the structure of the loaded data"},{"metadata":{"_uuid":"6fb06add639faa3183c5cf23d488501d395eaeb2","trusted":false,"_cell_guid":"934c974f-0d01-463e-8191-45ecf658de7b"},"cell_type":"code","source":"# check data sample \nhead(DstTest, n=5L)\nhead(DstTrain, n=5L)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"31b0d59fa10cc29845fe821bb9c98159a481bc63","_cell_guid":"293cd30e-96f4-448d-9b95-41f447cf0058"},"cell_type":"markdown","source":"## understanding the data\nLet's check the training data for number distribution.\n\n**Check the number distribution**"},{"metadata":{"_uuid":"95284f325ee544e9a489bcc26f833083e0923771","trusted":false,"_cell_guid":"f4c849f2-faa4-4f37-a0ef-6b62384abe07"},"cell_type":"code","source":"# check the data by each number\nt(as.data.frame(table(as.factor(DstTrain$label))))\n\n# Graphicaly is more easy to see\nggplot(DstTrain, aes(x=label,fill=as.factor(label))) +\n    geom_histogram(stat = \"count\") ","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"04f227389657f20514916955ff00a7c2fa26f621","_cell_guid":"642a2abc-5d08-4de5-aa82-e6ccda745286"},"cell_type":"markdown","source":"**Let's see some of the images**\n\nWhat's that we keep as image, let's check it out"},{"metadata":{"_uuid":"58a6f1759e677aa712fb803a4196313c52108ce6","trusted":false,"_cell_guid":"6799a1e4-3d12-4e31-88ee-f8fe084e29d1"},"cell_type":"code","source":"# 28*28 matrix with pixel color values\ndrawImage <- function(selrow) {\n    image(\n        t(apply(\n                matrix(unlist(DstTrain[selrow,-1]),\n                 nrow = 28,byrow = TRUE)\n            , 2, rev))\n    ,col=gray.colors(255))\n}\n\n# create a grid\npar(mfrow=c(3,4))\n\n# draw the images\nall <- sapply(sample(1:42000, 12, replace=T),drawImage)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"55aca7ed5ed535eddbedfb7e3d538a5ad80f5b5f","_cell_guid":"cb8edf80-3c82-4a29-b8cc-2fa491c777dc"},"cell_type":"markdown","source":"## Modeling \n\n**Start H2O**\n\nStart up a 1-node H2O server on your local machine, and allow it to use all CPU cores and up to 2GB of memory:"},{"metadata":{"_uuid":"12d66d15382ca70b7523290805f3a7ffc64f1b85","trusted":false,"_cell_guid":"1062ad61-1452-4346-a6e2-5855695a2974"},"cell_type":"code","source":"h2o.init(nthreads=-1, max_mem_size=\"2G\")\nh2o.removeAll()    # clean slate - just in case the cluster was already running\nh2o.no_progress()  # Don't show progress bars in RMarkdown output","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"92c2fd19c434c8f183cea372e0f31c506b630c47","_cell_guid":"2c0bebda-d9b9-4100-935c-07db97bed077"},"cell_type":"markdown","source":"**Set label as factor**"},{"metadata":{"_uuid":"ec092d76d01aeec3c3d60b2a87033a120516534b","trusted":false,"_cell_guid":"0ec634d0-8dae-43f0-9758-6d19ae51c7b6"},"cell_type":"code","source":"# label to factor\nDstTrain$label <- as.factor(DstTrain$label)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"7d3e942ac650929dae639f0c527d4f80d8c30cb4","_cell_guid":"ef3fa85d-5ab3-4618-a95a-5807e9c2b2d2"},"cell_type":"markdown","source":"**Covert Type Dataset**\n\nSplit the data 2 ways: 75% for training and 25% for test."},{"metadata":{"_uuid":"6f9e203ee0a541f20d13608c626cedc3c15e41a7","trusted":false,"_cell_guid":"b5af8489-f75f-41a6-a4ed-1448bf3d0641"},"cell_type":"code","source":"# convert to h2o frame \nh2o_TrainAux = as.h2o(DstTrain)\n\n# define the splits\nh2o_splits <- h2o.splitFrame(h2o_TrainAux, 0.75, seed=1234)\nh2o_DstTrain  <- h2o.assign(h2o_splits[[1]], \"train.hex\") # 75%\nh2o_DstTest  <- h2o.assign(h2o_splits[[2]], \"test.hex\") # 25%","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"aadd5bd386098c90f5713cc57419cc475589806c","trusted":false,"_cell_guid":"2674d939-5d21-41ac-bb4f-a30b7a62e94b"},"cell_type":"code","source":"# Identify predictors and response\nresponse <- \"label\"\npredictors <- setdiff(names(h2o_DstTrain), response)\n\n# Number of CV folds (to generate level-one data for stacking)\ncvfolds <- 5","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"550b4efc30fadfb7974f6be697a257fcb274452f","_cell_guid":"a880090a-9086-4526-a3a6-e841075e0b80"},"cell_type":"markdown","source":"**H2O base models**"},{"metadata":{"_uuid":"03ebbd5835473c4e5940d1e598302296473a1023","trusted":false,"_cell_guid":"7fe99c28-e677-49eb-b913-e690501fce9a"},"cell_type":"code","source":"# Train & Cross-validate a GBM\nmy_gbm <- h2o.gbm(x = predictors,\n                  y = response,\n                  training_frame = h2o_DstTrain,\n                  distribution = \"multinomial\",\n                  max_depth = 3,\n                  min_rows = 2,\n                  learn_rate = 0.2,\n                  nfolds = cvfolds,\n                  fold_assignment = \"Modulo\",\n                  keep_cross_validation_predictions = TRUE,\n                  seed = 1)\n\n# Train & Cross-validate a RF\nmy_rf <- h2o.randomForest(x = predictors,\n                          y = response,\n                          training_frame = h2o_DstTrain,\n                          nfolds = cvfolds,\n                          fold_assignment = \"Modulo\",\n                          keep_cross_validation_predictions = TRUE,\n                          seed = 1)\n\n# Train & Cross-validate a DNN\nmy_dl <- h2o.deeplearning(x = predictors,\n                          y = response,\n                          training_frame = h2o_DstTrain,\n                          l1 = 0.001,\n                          l2 = 0.001,\n                          hidden = c(200, 200, 200),\n                          nfolds = cvfolds,\n                          fold_assignment = \"Modulo\",\n                          keep_cross_validation_predictions = TRUE,\n                          seed = 1)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"1d12742dac054f05d1fb7c52a9b3be0104a007f3","_cell_guid":"ca564001-60d0-423b-a0a5-7fc605e06073"},"cell_type":"markdown","source":"**XGBoost base models**"},{"metadata":{"_uuid":"9b6a8645086c139a32c506edb3f974510385fcb1","trusted":false,"_cell_guid":"cf618dd8-880e-41e1-ab25-919095e21434"},"cell_type":"code","source":"# Train & Cross-validate a (shallow) XGB-GBM\nmy_xgb1 <- h2o.xgboost(x = predictors,\n                       y = response,\n                       training_frame = h2o_DstTrain,\n                       distribution = \"multinomial\",\n                       ntrees = 50,\n                       max_depth = 3,\n                       min_rows = 2,\n                       learn_rate = 0.2,\n                       nfolds = cvfolds,\n                       fold_assignment = \"Modulo\",\n                       keep_cross_validation_predictions = TRUE,\n                       seed = 1)\n\n\n# Train & Cross-validate another (deeper) XGB-GBM\nmy_xgb2 <- h2o.xgboost(x = predictors,\n                       y = response,\n                       training_frame = h2o_DstTrain,\n                       distribution = \"multinomial\",\n                       ntrees = 50,\n                       max_depth = 8,\n                       min_rows = 1,\n                       learn_rate = 0.1,\n                       sample_rate = 0.7,\n                       col_sample_rate = 0.9,\n                       nfolds = cvfolds,\n                       fold_assignment = \"Modulo\",\n                       keep_cross_validation_predictions = TRUE,\n                       seed = 1)\n","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"981584ffbabe9bd8636d0813ece19a1083c74871","_cell_guid":"8205639b-7ac9-4a58-a40c-f2956f6de425"},"cell_type":"markdown","source":"**Create a Stacked Ensemble**\n\nTo maximize predictive power, will create an H2O Stacked Ensemble from the models we created above and print the performance gain the ensemble has over the best base model."},{"metadata":{"_uuid":"08292cd931000c14a0ffad6b129cbf2bdd2dd601","trusted":false,"_cell_guid":"49a3a23a-3ec7-44df-8191-b05470412875"},"cell_type":"code","source":"# Train a stacked ensemble using the H2O and XGBoost models from above\nbase_models <- list(my_gbm@model_id, my_rf@model_id, my_dl@model_id,  \n                    my_xgb1@model_id, my_xgb2@model_id)\n\nensemble <- h2o.stackedEnsemble(x = predictors,\n                                y = response,\n                                training_frame = h2o_DstTrain,\n                                base_models = base_models)\n\n# Eval ensemble performance on a test set\nperf <- h2o.performance(ensemble, newdata = h2o_DstTest)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"e0c74b5fd0901c053049fcf6c3dbd4505b5f84ef","trusted":false,"_cell_guid":"8f9d9cba-3f08-4010-8ffa-32cc9b0ab0de"},"cell_type":"code","source":"# Compare to base learner performance on the test set\nget_rmse <- function(mm) h2o.rmse(h2o.performance(h2o.getModel(mm), newdata = h2o_DstTest))\nbaselearner_perf <- sapply(base_models, get_rmse)\nbaselearner_best_auc_test <- min(baselearner_perf)\nensemble_auc_test <- h2o.rmse(perf)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"300139e17e7ff4c624f7060b557535a6053c48a1","_cell_guid":"3461ba0c-5218-415e-91e3-46297013b9fb"},"cell_type":"markdown","source":"Compare the test set performance of the best base model to the ensemble."},{"metadata":{"_uuid":"ce8887c550803cb2d2c9f1c0c25c27e38511c694","trusted":false,"_cell_guid":"ccadd5e6-8825-4f30-b7c7-796367934487"},"cell_type":"code","source":"print(sprintf(\"Best Base-learner Test RMSE:  %s\", baselearner_best_auc_test))\nprint(sprintf(\"Ensemble Test RMSE:  %s\", ensemble_auc_test))","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"002f28b7ec98b690271dc3b04908bb44acf071de","_cell_guid":"1f761797-8dac-4afd-a1a5-1a12d994a2a5"},"cell_type":"markdown","source":"## Final predict and submition file\n\nLet's make the final prediction and create the file for submit"},{"metadata":{"_uuid":"bebdbad8a3a9749575bd1ad77f95b7f33af2dd52","trusted":false,"_cell_guid":"e04d1b2f-c8e2-4873-9123-5ca163df3163"},"cell_type":"code","source":"# convert to h2o frame \nh2o_FinalTest = as.h2o(DstTest)\n\n# predict with the model\npredictFinal <- h2o.predict(ensemble, h2o_FinalTest)\n\n# convert H2O format into data frame and save as csv\npredictFinal.df <- as.data.frame(predictFinal)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"38e7c4c59a074588bba1b74cf4f497dd3ecdc6b8","trusted":false,"_cell_guid":"277cd321-a41c-439b-aef0-7289d42d45ad"},"cell_type":"code","source":"# create the final dataset\ndstSubmit <- data.frame(ImageId = seq(1,length(predictFinal.df$predict)), Label = predictFinal.df$predict)\nhead(dstSubmit, n=5L)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"998e16d05398826a7df8e2dff691b9d068ab2f10","trusted":false,"_cell_guid":"3ae044c7-2ae1-4276-b3c3-ddcd7dcccb6b"},"cell_type":"code","source":"# write the submit csv file\n write.csv(dstSubmit,file = \"ImgSubmit.csv\",row.names = FALSE)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"ff2a5878f7d270d783f37adc9d55ea8da708bb14","trusted":false,"_cell_guid":"f7e1e2a4-3e21-4543-811a-4c655485652b"},"cell_type":"code","source":"# shut down virtual H2O cluster\nh2o.shutdown(prompt = FALSE)","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"display_name":"R","language":"R","name":"ir"},"language_info":{"mimetype":"text/x-r-source","version":"3.4.2","name":"R","pygments_lexer":"r","file_extension":".r","codemirror_mode":"r"}},"nbformat":4,"nbformat_minor":1}