{"cells":[{"metadata":{"_uuid":"89387fbdc0510b66c7186c1e7ab4d24a4c77244c","_cell_guid":"d219ef1c-2150-47e4-8a9a-3f8b2247a815"},"cell_type":"markdown","source":"# Digit Recognizer \n\"Each image is **28 pixels in height** and **28 pixels in width**, for a total of 784 pixels in total. Each pixel has a single pixel-value associated with it, indicating the lightness or darkness of that pixel, with higher numbers meaning darker. This pixel-value is an integer between 0 and 255, inclusive.\"\n\nSo each image have 28x28 pixels and a color on each pixel."},{"metadata":{"_uuid":"bde6bbe97933726719935de9ca911d05760fc0cb","_cell_guid":"7b79d1fd-35a5-4461-bf0a-b12d5337a438"},"cell_type":"markdown","source":"## Assign Necessary Libraries\n"},{"metadata":{"_uuid":"238fac25c47e292a0d93b1e8aa7837869374ddcc","trusted":false,"_cell_guid":"c4c3daa5-4e81-42af-9a0a-fcb8de196e4f"},"cell_type":"code","source":"# Necessary Libraries \n\nsuppressMessages(library(lattice))\nsuppressMessages(library(ggplot2))\nsuppressMessages(library(caret))\n\n# for describe the data\nsuppressMessages(library(psych))\n\n# install.packages(\"h2o\")\nsuppressMessages(library(h2o))","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"2d9e4ad1955c2bce00037610044129395a506272","_cell_guid":"5f1b33db-4089-4539-91e4-3c08090e3b7a"},"cell_type":"markdown","source":"## Loading data"},{"metadata":{"_uuid":"56abef5c2ca0222dff1caaf581f66930f00cc55c","trusted":false,"_cell_guid":"2731e6ed-5847-4ff0-b9a5-441c1363e09d"},"cell_type":"code","source":"# load train.csv\nDstTrain <- read.csv('../input/train.csv', stringsAsFactors = FALSE, na.strings = c(\"NA\", \"\"))\n# load test.csv\nDstTest  <- read.csv('../input/test.csv', stringsAsFactors = FALSE, na.strings = c(\"NA\", \"\"))","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"f3894f0ba6e056fc9eb0c27c9401b1e629a4d59d","_cell_guid":"d5f4e810-d95c-46bd-9ed1-eee629ac65ba"},"cell_type":"markdown","source":"## Checking the data\n\n**Check some info about structure**\n\nLet us explore the structure of the loaded data"},{"metadata":{"_uuid":"6fb06add639faa3183c5cf23d488501d395eaeb2","trusted":false,"_cell_guid":"934c974f-0d01-463e-8191-45ecf658de7b"},"cell_type":"code","source":"# check data sample \nhead(DstTest, n=5L)\nhead(DstTrain, n=5L)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"31b0d59fa10cc29845fe821bb9c98159a481bc63","_cell_guid":"293cd30e-96f4-448d-9b95-41f447cf0058"},"cell_type":"markdown","source":"## understanding the data\nLet's check the training data for number distribution.\n\n**Check the number distribution**"},{"metadata":{"_uuid":"95284f325ee544e9a489bcc26f833083e0923771","trusted":false,"_cell_guid":"f4c849f2-faa4-4f37-a0ef-6b62384abe07"},"cell_type":"code","source":"# check the data by each number\nt(as.data.frame(table(as.factor(DstTrain$label))))\n\n# supress the boring warning\noptions(warn=-1)\n# Graphicaly is more easy to see\nggplot(DstTrain, aes(x=label,fill=as.factor(label))) +\n    geom_histogram(stat = \"count\") +\n    scale_x_discrete(name=\"Numbers\", limits=c(0,1,2,3,4,5,6,7,8,9)) +\n    scale_y_discrete(name=\"Frequency\", limits=c(0,500,2000,4000, 6000))\n\n# to the defaul value \noptions(warn=0)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"04f227389657f20514916955ff00a7c2fa26f621","_cell_guid":"642a2abc-5d08-4de5-aa82-e6ccda745286"},"cell_type":"markdown","source":"**Let's see some of the images**\n\nWhat's that we keep as image, let's check it out"},{"metadata":{"_uuid":"58a6f1759e677aa712fb803a4196313c52108ce6","trusted":false,"_cell_guid":"6799a1e4-3d12-4e31-88ee-f8fe084e29d1"},"cell_type":"code","source":"# 28*28 matrix with pixel color values\ndrawImage <- function(selrow) {\n    image(\n        t(apply(\n                matrix(unlist(DstTrain[selrow,-1]),\n                 nrow = 28,byrow = TRUE)\n            , 2, rev))\n    ,col=gray.colors(255))\n}\n\n# create a grid\npar(mfrow=c(3,4))\n\n# draw the images\nall <- sapply(sample(1:42000, 12, replace=T),drawImage)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"55aca7ed5ed535eddbedfb7e3d538a5ad80f5b5f","_cell_guid":"cb8edf80-3c82-4a29-b8cc-2fa491c777dc"},"cell_type":"markdown","source":"## Modeling \n\n**Start H2O**\n\nStart up a 1-node H2O server on your local machine, and allow it to use all CPU cores and up to 2GB of memory:"},{"metadata":{"_uuid":"12d66d15382ca70b7523290805f3a7ffc64f1b85","trusted":false,"_cell_guid":"1062ad61-1452-4346-a6e2-5855695a2974"},"cell_type":"code","source":"h2o.init(nthreads=-1, max_mem_size=\"12G\")\nh2o.removeAll()    # clean slate - just in case the cluster was already running\nh2o.no_progress()  # Don't show progress bars in RMarkdown output","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"92c2fd19c434c8f183cea372e0f31c506b630c47","_cell_guid":"2c0bebda-d9b9-4100-935c-07db97bed077"},"cell_type":"markdown","source":"**Set label as factor**"},{"metadata":{"_uuid":"ec092d76d01aeec3c3d60b2a87033a120516534b","trusted":false,"_cell_guid":"0ec634d0-8dae-43f0-9758-6d19ae51c7b6"},"cell_type":"code","source":"# label to factor\nDstTrain$label <- as.factor(DstTrain$label)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"7d3e942ac650929dae639f0c527d4f80d8c30cb4","_cell_guid":"ef3fa85d-5ab3-4618-a95a-5807e9c2b2d2"},"cell_type":"markdown","source":"**Covert Type Dataset**\n\nSplit the data 2 ways: 80% for training and 20% for test."},{"metadata":{"_uuid":"6f9e203ee0a541f20d13608c626cedc3c15e41a7","trusted":false,"_cell_guid":"b5af8489-f75f-41a6-a4ed-1448bf3d0641"},"cell_type":"code","source":"# convert to h2o frame \nh2o_TrainAux = as.h2o(DstTrain)\n\n# define the splits\nh2o_splits <- h2o.splitFrame(h2o_TrainAux, 0.80, seed=1234)\nh2o_DstTrain  <- h2o.assign(h2o_splits[[1]], \"train.hex\") # 80%\nh2o_DstTest  <- h2o.assign(h2o_splits[[2]], \"test.hex\") # 20%","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"aadd5bd386098c90f5713cc57419cc475589806c","trusted":false,"_cell_guid":"2674d939-5d21-41ac-bb4f-a30b7a62e94b"},"cell_type":"code","source":"# Identify predictors and response\nresponse <- \"label\"\npredictors <- setdiff(names(h2o_DstTrain), response)\n\n# Number of CV folds (to generate level-one data for stacking)\ncvfolds <- 5","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"550b4efc30fadfb7974f6be697a257fcb274452f","_cell_guid":"a880090a-9086-4526-a3a6-e841075e0b80"},"cell_type":"markdown","source":"**H2O models**\n\nhttps://blog.h2o.ai/2015/02/deep-learning-performance/"},{"metadata":{"_uuid":"c125bbac03126c351d10b4144f881bb18161a843","trusted":false,"_cell_guid":"bf2b0cc0-88b3-4004-9732-ae0ff5c4bda1"},"cell_type":"code","source":"# # fit 1 - \n# # epochs 1 - Public Score 0.96385\n# fit.dl1 <- h2o.deeplearning(\n#                     x = predictors,  # predictors\n#                     y = response,   # label\n# #                     training_frame = h2o_DstTrain, \n#                     training_frame = h2o_TrainAux,\n#                     activation = \"MaxoutWithDropout\", # used algorithm\n#                     input_dropout_ratio = 0.2, # dropout\n#                     balance_classes = FALSE,\n#                     hidden = c(400,400,400), \n#                     momentum_stable = 0.99,\n#                     nesterov_accelerated_gradient = TRUE, # speed boost\n#                     epochs = 1, # epochs\n#                     nfolds = cvfolds,\n#                     fold_assignment = \"Modulo\",\n#                     keep_cross_validation_predictions = TRUE,\n#                     seed = 1\n#                 ) ","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"03ebbd5835473c4e5940d1e598302296473a1023","trusted":false,"_cell_guid":"7fe99c28-e677-49eb-b913-e690501fce9a"},"cell_type":"code","source":"# # fit 2\n# # 5 epochs - Public Score 0.95571\n# # 25 epochs - Public Score 0.96800\n# fit.dl2 <- h2o.deeplearning(\n#                     x = predictors,  # predictors\n#                     y = response,   # label\n# #                     training_frame = h2o_DstTrain, \n#                     training_frame = h2o_TrainAux, \n#                     activation = \"RectifierWithDropout\", # used algorithm\n#                     input_dropout_ratio = 0.2, # dropout\n#                     balance_classes = FALSE,\n#                     hidden = c(400,400,400), \n#                     momentum_stable = 0.99,\n#                     nesterov_accelerated_gradient = TRUE, # speed boost\n#                     epochs = 25, # epochs\n#                     nfolds = cvfolds,\n#                     fold_assignment = \"Modulo\",\n#                     keep_cross_validation_predictions = TRUE,\n#                     seed = 1\n#                 ) ","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"b9ba13978f0b224892b0964bf43cb5c26647d37f"},"cell_type":"code","source":"fit.dl3 <- h2o.deeplearning(\n            x = predictors,  # predictors\n            y = response,   # label\n            training_frame = h2o_TrainAux, \n            activation = \"Rectifier\", \n            input_dropout_ratio = 0.2, \n            balance_classes = FALSE,\n            hidden = c(500,500,500), \n            momentum_stable = 0.99,\n            nesterov_accelerated_gradient = TRUE, \n            epochs = 20,\n            seed = 1\n) ","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"002f28b7ec98b690271dc3b04908bb44acf071de","_cell_guid":"1f761797-8dac-4afd-a1a5-1a12d994a2a5"},"cell_type":"markdown","source":"## Final predict and submition file\n\nLet's make the final prediction and create the file for submit"},{"metadata":{"_uuid":"bebdbad8a3a9749575bd1ad77f95b7f33af2dd52","trusted":false,"_cell_guid":"e04d1b2f-c8e2-4873-9123-5ca163df3163"},"cell_type":"code","source":"# convert to h2o frame \nh2o_FinalTest = as.h2o(DstTest)\n\n# predict with the model\npredictFinal <- h2o.predict(fit.dl3, h2o_FinalTest)\n\n# convert H2O format into data frame and save as csv\npredictFinal.df <- as.data.frame(predictFinal)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"38e7c4c59a074588bba1b74cf4f497dd3ecdc6b8","trusted":false,"_cell_guid":"277cd321-a41c-439b-aef0-7289d42d45ad"},"cell_type":"code","source":"# create the final dataset\ndstSubmit <- data.frame(ImageId = seq(1,length(predictFinal.df$predict)), Label = predictFinal.df$predict)\nhead(dstSubmit, n=5L)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"998e16d05398826a7df8e2dff691b9d068ab2f10","trusted":false,"_cell_guid":"3ae044c7-2ae1-4276-b3c3-ddcd7dcccb6b"},"cell_type":"code","source":"# write the submit csv file\n write.csv(dstSubmit,file = \"ImgSubmit.csv\",row.names = FALSE)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"ff2a5878f7d270d783f37adc9d55ea8da708bb14","trusted":false,"_cell_guid":"f7e1e2a4-3e21-4543-811a-4c655485652b"},"cell_type":"code","source":"# shut down virtual H2O cluster\nh2o.shutdown(prompt = FALSE)","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"display_name":"R","language":"R","name":"ir"},"language_info":{"mimetype":"text/x-r-source","name":"R","pygments_lexer":"r","version":"3.4.2","file_extension":".r","codemirror_mode":"r"}},"nbformat":4,"nbformat_minor":1}