{"cells":[{"metadata":{"_uuid":"89387fbdc0510b66c7186c1e7ab4d24a4c77244c","_cell_guid":"d219ef1c-2150-47e4-8a9a-3f8b2247a815"},"cell_type":"markdown","source":"# Digit Recognizer \n\"Each image is **28 pixels in height** and **28 pixels in width**, for a total of 784 pixels in total. Each pixel has a single pixel-value associated with it, indicating the lightness or darkness of that pixel, with higher numbers meaning darker. This pixel-value is an integer between 0 and 255, inclusive.\"\n\nSo each image have 28x28 pixels and a color on each pixel.\n\n\n## Keras Power\n\nKeras is a high-level neural networks API developed with a focus on enabling fast experimentation. Being able to go from idea to result with the least possible delay is key to doing good research. Keras has the following key features:\n\n* Allows the same code to run on CPU or on GPU, seamlessly.\n* User-friendly API which makes it easy to quickly prototype deep learning models.\n* Built-in support for convolutional networks (for computer vision), recurrent networks (for sequence processing), and any combination of both.\n* Supports arbitrary network architectures: multi-input or multi-output models, layer sharing, model sharing, etc. This means that Keras is appropriate for building essentially any deep learning model, from a memory network to a neural Turing machine.\n* Is capable of running on top of multiple back-ends including TensorFlow, CNTK, or Theano.\n\n**source:** [R interface to Keras](https://keras.rstudio.com/)"},{"metadata":{"_uuid":"bde6bbe97933726719935de9ca911d05760fc0cb","_cell_guid":"7b79d1fd-35a5-4461-bf0a-b12d5337a438"},"cell_type":"markdown","source":"## Assign Necessary Libraries\n"},{"metadata":{"_uuid":"238fac25c47e292a0d93b1e8aa7837869374ddcc","trusted":true,"_cell_guid":"c4c3daa5-4e81-42af-9a0a-fcb8de196e4f"},"cell_type":"code","source":"# Necessary Libraries \nsuppressMessages(library(lattice))\nsuppressMessages(library(ggplot2))\n\n# for describe the data\nsuppressMessages(library(psych))\n\n# caret for the split\nsuppressMessages(library(caret))\n# Keras\nsuppressMessages(library(keras))","execution_count":15,"outputs":[]},{"metadata":{"_uuid":"2d9e4ad1955c2bce00037610044129395a506272","_cell_guid":"5f1b33db-4089-4539-91e4-3c08090e3b7a"},"cell_type":"markdown","source":"## Loading data"},{"metadata":{"_uuid":"56abef5c2ca0222dff1caaf581f66930f00cc55c","trusted":true,"_cell_guid":"2731e6ed-5847-4ff0-b9a5-441c1363e09d"},"cell_type":"code","source":"# load train.csv\nDstTrain <- read.csv('../input/train.csv', stringsAsFactors = FALSE, na.strings = c(\"NA\", \"\"))\n# load test.csv\nDstTest  <- read.csv('../input/test.csv', stringsAsFactors = FALSE, na.strings = c(\"NA\", \"\"))","execution_count":2,"outputs":[]},{"metadata":{"_uuid":"f3894f0ba6e056fc9eb0c27c9401b1e629a4d59d","_cell_guid":"d5f4e810-d95c-46bd-9ed1-eee629ac65ba"},"cell_type":"markdown","source":"## Checking the data\n\n**Check some info about structure**\n\nLet us explore the structure of the loaded data"},{"metadata":{"_uuid":"6fb06add639faa3183c5cf23d488501d395eaeb2","trusted":true,"_cell_guid":"934c974f-0d01-463e-8191-45ecf658de7b"},"cell_type":"code","source":"# check data sample \nhead(DstTest, n=5L)\nhead(DstTrain, n=5L)","execution_count":3,"outputs":[]},{"metadata":{"_uuid":"31b0d59fa10cc29845fe821bb9c98159a481bc63","_cell_guid":"293cd30e-96f4-448d-9b95-41f447cf0058"},"cell_type":"markdown","source":"## understanding the data\nLet's check the training data for number distribution.\n\n**Check the number distribution**"},{"metadata":{"_uuid":"95284f325ee544e9a489bcc26f833083e0923771","trusted":true,"_cell_guid":"f4c849f2-faa4-4f37-a0ef-6b62384abe07"},"cell_type":"code","source":"# check the data by each number\nt(as.data.frame(table(as.factor(DstTrain$label))))\n\n# supress the boring warning\noptions(warn=-1)\n# Graphicaly is more easy to see\nggplot(DstTrain, aes(x=label,fill=as.factor(label))) +\n    geom_histogram(stat = \"count\") +\n    scale_x_discrete(name=\"Numbers\", limits=c(0,1,2,3,4,5,6,7,8,9)) +\n    scale_y_discrete(name=\"Frequency\", limits=c(0,500,2000,4000, 6000))\n\n# to the defaul value \noptions(warn=0)","execution_count":12,"outputs":[]},{"metadata":{"_uuid":"04f227389657f20514916955ff00a7c2fa26f621","_cell_guid":"642a2abc-5d08-4de5-aa82-e6ccda745286"},"cell_type":"markdown","source":"**Let's see some of the images**\n\nWhat's that we keep as image, let's check it out"},{"metadata":{"_uuid":"58a6f1759e677aa712fb803a4196313c52108ce6","trusted":true,"_cell_guid":"6799a1e4-3d12-4e31-88ee-f8fe084e29d1"},"cell_type":"code","source":"# 28*28 matrix with pixel color values\ndrawImage <- function(selrow) {\n    image(\n        t(apply(\n                matrix(unlist(DstTrain[selrow,-1]),\n                 nrow = 28,byrow = TRUE)\n            , 2, rev))\n    ,col=gray.colors(255))\n}\n\n# create a grid\npar(mfrow=c(3,4))\n\n# draw the images\nall <- sapply(sample(1:42000, 12, replace=T),drawImage)","execution_count":13,"outputs":[]},{"metadata":{"_uuid":"2ec11d238447f9e25969815e0dce83a284157f78"},"cell_type":"markdown","source":"## Preparing the Data\n\nSpliting our data:\n* Split in Train and Test data\n* Split into X and Y"},{"metadata":{"trusted":true,"_uuid":"f0f59b3d681c9373ff7c9859f0cef1438b6837cf"},"cell_type":"code","source":"# Split in Train and Test data\nsplitRows <- createDataPartition(DstTrain$label, p=0.8, list=FALSE)\nKerasTrain <- DstTrain[c(splitRows), ]\nKerasTest <- DstTrain[c(-splitRows), ]","execution_count":19,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"05927b17d48930fcb929727435a2305ad675fc06"},"cell_type":"code","source":"# Train Data\nx_KerasTrain <- KerasTrain[,-c(1)]\ny_KerasTrain <- KerasTrain[,c(1)]\n\n# Test Data\nx_KerasTest <- KerasTest[,-c(1)]\ny_KerasTest <- KerasTest[,c(1)]","execution_count":20,"outputs":[]},{"metadata":{"_uuid":"54dcbeeb81772773180c5366bdd2de37da156cd2"},"cell_type":"markdown","source":"The x data is a 3-d array `(images,width,height)` of grayscale values . To prepare the data for training we convert the 3-d arrays into matrices by reshaping width and height into a single dimension (28x28 images are flattened into length 784 vectors). Then, we convert the grayscale values from integers ranging between 0 to 255 into floating point values ranging between 0 and 1:"},{"metadata":{"trusted":true,"_uuid":"e312f6d48265cc5b6e045f24085c4c4c4428b1ee"},"cell_type":"code","source":"# reshape\nx_KerasTrain <- matrix(as.numeric(unlist(x_KerasTrain)),nrow=nrow(x_KerasTrain))\nx_KerasTest <- matrix(as.numeric(unlist(x_KerasTest)),nrow=nrow(x_KerasTest))\n\n# rescale\nx_KerasTrain <- x_KerasTrain / 255\nx_KerasTest <- x_KerasTest / 255","execution_count":25,"outputs":[]},{"metadata":{"_uuid":"5c1876e41e09047cb29309810230f0eb51f96b35"},"cell_type":"markdown","source":"The `y` data is an integer vector with values ranging from 0 to 9. \n\nTo prepare this data for training we [one-hot encode](https://www.quora.com/What-is-one-hot-encoding-and-when-is-it-used-in-data-science) the vectors into binary class matrices using the Keras `to_categorical()` function:"},{"metadata":{"trusted":true,"_uuid":"67d8b338773733f7d9c9ee7154e6d4cca9018f0e"},"cell_type":"code","source":"y_KerasTrain <- to_categorical(y_KerasTrain, 10)\ny_KerasTest <- to_categorical(y_KerasTest, 10)","execution_count":26,"outputs":[]},{"metadata":{"_uuid":"55aca7ed5ed535eddbedfb7e3d538a5ad80f5b5f","_cell_guid":"cb8edf80-3c82-4a29-b8cc-2fa491c777dc"},"cell_type":"markdown","source":"## Defining the Model\n\nThe core data structure of Keras is a model, a way to organize layers. The simplest type of model is the [Sequential model](https://keras.rstudio.com/articles/sequential_model.html), a linear stack of layers.\n\nWe begin by creating a sequential model and then adding layers using the pipe (`%>%`) operator:"},{"metadata":{"trusted":true,"_uuid":"33a35dc5c072cb252227e784524f14aacf51bac2"},"cell_type":"code","source":"model <- keras_model_sequential() ","execution_count":27,"outputs":[]},{"metadata":{"_uuid":"12d66d15382ca70b7523290805f3a7ffc64f1b85","trusted":true,"_cell_guid":"1062ad61-1452-4346-a6e2-5855695a2974"},"cell_type":"code","source":"\nmodel %>% \n  layer_dense(units = 256, activation = 'relu', input_shape = c(784)) %>% \n  layer_dropout(rate = 0.4) %>% \n  layer_dense(units = 128, activation = 'relu') %>%\n  layer_dropout(rate = 0.3) %>%\n  layer_dense(units = 10, activation = 'softmax')","execution_count":28,"outputs":[]},{"metadata":{"_uuid":"c64feb74b5558a58dc20e450e2c1d56fe7f0c8b5"},"cell_type":"markdown","source":"The `input_shape` argument to the first layer specifies the shape of the input data (a length 784 numeric vector representing a grayscale image). The final layer outputs a length 10 numeric vector (probabilities for each digit) using a [softmax](https://en.wikipedia.org/wiki/Softmax_function) activation function.\n\nUse the `summary()` function to print the details of the model:"},{"metadata":{"_uuid":"ec092d76d01aeec3c3d60b2a87033a120516534b","trusted":true,"_cell_guid":"0ec634d0-8dae-43f0-9758-6d19ae51c7b6"},"cell_type":"code","source":"summary(model)","execution_count":29,"outputs":[]},{"metadata":{"_uuid":"7d3e942ac650929dae639f0c527d4f80d8c30cb4","_cell_guid":"ef3fa85d-5ab3-4618-a95a-5807e9c2b2d2"},"cell_type":"markdown","source":"Next, compile the model with appropriate loss function, optimizer, and metrics:"},{"metadata":{"_uuid":"6f9e203ee0a541f20d13608c626cedc3c15e41a7","trusted":true,"_cell_guid":"b5af8489-f75f-41a6-a4ed-1448bf3d0641"},"cell_type":"code","source":"model %>% compile(\n  loss = 'categorical_crossentropy',\n  optimizer = optimizer_rmsprop(),\n  metrics = c('accuracy')\n)","execution_count":30,"outputs":[]},{"metadata":{"_uuid":"550b4efc30fadfb7974f6be697a257fcb274452f","_cell_guid":"a880090a-9086-4526-a3a6-e841075e0b80"},"cell_type":"markdown","source":"**Training and Evaluation**\n\nUse the `fit()` function to train the model for 30 epochs using batches of 128 images:"},{"metadata":{"_uuid":"03ebbd5835473c4e5940d1e598302296473a1023","trusted":true,"_cell_guid":"7fe99c28-e677-49eb-b913-e690501fce9a"},"cell_type":"code","source":"history <- model %>% fit(\n  x_KerasTrain, y_KerasTrain, \n  epochs = 30, batch_size = 128, \n  validation_split = 0.2\n)","execution_count":32,"outputs":[]},{"metadata":{"_uuid":"1d12742dac054f05d1fb7c52a9b3be0104a007f3","_cell_guid":"ca564001-60d0-423b-a0a5-7fc605e06073"},"cell_type":"markdown","source":"The history object returned by `fit()` includes loss and accuracy metrics which we can `plot`:"},{"metadata":{"_uuid":"9b6a8645086c139a32c506edb3f974510385fcb1","trusted":true,"_cell_guid":"cf618dd8-880e-41e1-ab25-919095e21434"},"cell_type":"code","source":"plot(history)","execution_count":33,"outputs":[]},{"metadata":{"_uuid":"981584ffbabe9bd8636d0813ece19a1083c74871","_cell_guid":"8205639b-7ac9-4a58-a40c-f2956f6de425"},"cell_type":"markdown","source":"Evaluate the model’s performance on the test data:"},{"metadata":{"_uuid":"08292cd931000c14a0ffad6b129cbf2bdd2dd601","trusted":true,"_cell_guid":"49a3a23a-3ec7-44df-8191-b05470412875"},"cell_type":"code","source":"model %>% evaluate(x_KerasTest, y_KerasTest)","execution_count":34,"outputs":[]},{"metadata":{"_uuid":"002f28b7ec98b690271dc3b04908bb44acf071de","_cell_guid":"1f761797-8dac-4afd-a1a5-1a12d994a2a5"},"cell_type":"markdown","source":"## Final predict and submition file\n\nLet's make the final prediction and create the file for submit"},{"metadata":{"_uuid":"bebdbad8a3a9749575bd1ad77f95b7f33af2dd52","trusted":true,"_cell_guid":"e04d1b2f-c8e2-4873-9123-5ca163df3163"},"cell_type":"code","source":"# reshape\nDstTest <- matrix(as.numeric(unlist(DstTest)),nrow=nrow(DstTest))\n\n# rescale\nDstTest <- DstTest / 255","execution_count":35,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"a27f9e8cac54f1837c37b2ff7731a7f6dfb708ab"},"cell_type":"code","source":"# predict \npredictFinal <- model %>% predict_classes(DstTest,batch_size=100)\n# create the final dst\ndstSubmit <- data.frame(ImageId=1:nrow(DstTest),Label=predictFinal)","execution_count":36,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"221db282ee623eaf729e37435fe5d7ca49af3328"},"cell_type":"code","source":"# check some data\nhead(dstSubmit)","execution_count":37,"outputs":[]},{"metadata":{"_uuid":"998e16d05398826a7df8e2dff691b9d068ab2f10","trusted":false,"_cell_guid":"3ae044c7-2ae1-4276-b3c3-ddcd7dcccb6b"},"cell_type":"code","source":"# write the submit csv file\n write.csv(dstSubmit,file = \"ImgSubmit.csv\",row.names = FALSE)","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"display_name":"R","language":"R","name":"ir"},"language_info":{"mimetype":"text/x-r-source","name":"R","pygments_lexer":"r","version":"3.4.2","file_extension":".r","codemirror_mode":"r"}},"nbformat":4,"nbformat_minor":1}