---
title: "Keras DNN/CNN Part1"
author: "Chengran (Owen) Ouyang"
date: "`r format(Sys.Date())`"
output:
  html_document:
    theme: lumen
    number_sections: true
    font-family: Open Sans, sans-serif
    font-import: https://fonts.googleapis.com/css?family=Open+Sans
    code_folding: hide
    highlight: tango
editor_options: 
  chunk_output_type: console
---
# This is an easy approach to use Keras in R. 
The result is not very satisfying due to the limitation. If you are looking for a relatively robus model, please check out my kernel: [Easiest CNN with 99.6% Accurarcy using Keras (LB:Top 5%)](https://www.kaggle.com/couyang/easiest-cnn-with-99-accuracy-using-keras).

## Why this kernel is easy to understand?
* **Keras** as a interface of **tensorflow** is very easy to use with a small amount of coding.
* All codes here are done through pipeline via **tidyverse** Package, which ensure you to stop at any point to check and understand the progress.
* Most of the codes are self-explaining
* The last part, I hope my explaination below is easy enough to show the steps

This kernal is inspired by *Easy Neural Network in R for 0.994+* by TheTS, for which I believe is one of the best kernal for this competition in R. 

As one of the most popular tool for deep learning, **Keras** has gained lots of attention from data scientists. Now it is avaiable on R and this kernal is aiming to provide a brief guide in using this tool.

We will follow TheTS's footprint. Rather than using **MXnet**, we leverage the power of **Keras** to identify the numbers.

## Load packages and data
To set it up on your end, you need to install tensorflow and keras (*install_tensorflow()* & *install_keras()*)
```{r setup, include=FALSE}
library(tidyverse)
library(keras)
library(tensorflow)
# install_tensorflow()
# install_keras()
train <- data.matrix(read.csv("../input/train.csv", header=T))
test <- data.matrix(read.csv("../input/test.csv", header=T))
epochs <- 5
```

## Exploring the data  

The second step is always to do EDA and have a closer look to the data.

I checked the dimension of both train and test data and as expected train has one more column, which is the label of the numbers. Then, we dive into the labels to find out whether the dataset has roughly equal numbers of either number.

Then, I turned the pixels into a 28x28 matrix. It is very interesting to see how the computer 'see' these numbers and convert that into human readable grey color picture. I found most of the kernels are more complicated for beginners to learn how to graph the numbers. Here I only graph one number to make it as easy to understand as possible. 

```{r}
dim(train)
dim(test)

train[,1] %>% table() %>% barplot(col=1:10)

train[4,-1] %>% matrix(nrow=sqrt(dim(train)[2]-1),byrow=T) %>% apply(2,rev)%>% t() %>% image(col=grey.colors(255))
```


## Preprocess the data

Fortunately, **Keras** has the function to_categorical() & normalize(). I used the first one to turn label into categorical and applied the second one to normalized the pixels.

```{r}
train.label<-train[,1] %>% to_categorical()

train.feature<-train[,-1] %>% normalize()
test.feature<-test %>% normalize()
```

## Building A Simple Deep Neural Network

The model is using very basic dnn model with Fully Connected layers. The 4 level layers each has 128, 64, 32, and 10 neurons. The activation layers that I used is 'relu'.

I also used 'adam' as the optimizer and evaluated based on 'accuracy'. The epochs is only set to be `r epochs` to save time.

```{r}
model<-keras_model_sequential()

model %>% 
  layer_dense(units=128,activation='relu',input_shape=c(784))%>%
  layer_dense(units=64,activation='relu')%>%
  layer_dense(units=32,activation='relu')%>%
  layer_dense(units=10,activation='softmax')
  
 model%>%compile(
  loss='categorical_crossentropy',
  optimizer='adam',
  metrics='accuracy'
)

history<-model %>%  fit(
  train.feature,
  train.label,
  epochs=epochs, # make it higher to improve accuracy
  batch_size=5,
  validation_split=.1
)


```

Then, let's plot the training process of the `r epochs` epochs. 

```{r, eval=TRUE, include=TRUE}
plot(history)

```

As the graph shows, the accuracy has been improving as the epochs increase. I used a very lower epochs for this kernel to save time but feel free to bump it up to achieve better result. However, there is a better way to more significantly improve the score, which is to add convolutional layers and build a CNN model instead.

The next step is to predict based on the test dataset and submit

```{r, eval=TRUE, include=TRUE}
pred<-model %>% predict_classes(test.feature,batch_size=128)

dnnsubmission<-data.frame(ImageId=1:nrow(test),Label=pred)

write.csv(dnnsubmission, file="dnnsubmission.csv", row.names=F)
```
If you like my kernal so far, please give a upvote~ Thanks!

The next steps are to use a CNN model with data augmentation using **Keras** for more accurate prediction and a **handcraft** Deep Learning Model without using any DL packages.

To be continued!