{"cells":[{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"cell_type":"code","source":"import pandas as pd\nimport numpy as np\nimport matplotlib.pyplot as plt\nimport tensorflow as tf\nfrom tensorflow import keras \nimport seaborn as sns\n\nfrom keras.utils.np_utils import to_categorical\nfrom sklearn.model_selection import train_test_split\nfrom keras.preprocessing.image import ImageDataGenerator","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# data preparation"},{"metadata":{"trusted":true},"cell_type":"code","source":"#load data\ndf_train=pd.read_csv('../input/digit-recognizer/train.csv')\ndf_test=pd.read_csv('../input/digit-recognizer/test.csv')\nprint(df_train.shape)\nprint(df_test.shape)\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"df_test.head()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# split dataset"},{"metadata":{"trusted":true},"cell_type":"code","source":"X_train=df_train.iloc[:,1:]\ny_train=df_train.iloc[:,0]\nprint(X_train.shape)\nprint(y_train.shape)\ng=sns.countplot(y_train)\ny_train.value_counts()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"df_train.describe()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Check for null and missing values"},{"metadata":{"trusted":true},"cell_type":"code","source":" df_train.isnull().any().describe()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# normalization"},{"metadata":{},"cell_type":"markdown","source":"We perform a grayscale normalization to reduce the effect of illumination's differences.\n\nMoreover the CNN converg faster on [0..1] data than on [0..255]."},{"metadata":{"trusted":true},"cell_type":"code","source":"X_train=X_train/255.0\ndf_test=df_test/255.0","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":""},{"metadata":{"trusted":true},"cell_type":"code","source":"X_train=X_train.values.reshape(-1,28,28)\nfor i in range(6, 9):\n    plt.subplot(3,3,i+1)\n    plt.xticks([])\n    plt.yticks([])\n    plt.imshow(X_train[i],cmap='gray')\n    plt.title(y_train[i]);","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Reshape"},{"metadata":{},"cell_type":"markdown","source":"Train and test images (28px x 28px) has been stock into pandas.Dataframe as 1D vectors of 784 values. We reshape all data to 28x28x1 3D matrices.\n\nKeras requires an extra dimension in the end which correspond to channels. MNIST images are gray scaled so it use only one channel. For RGB images, there is 3 channels, we would have reshaped 784px vectors to 28x28x3 3D matrices."},{"metadata":{"trusted":true},"cell_type":"code","source":"X_train=X_train.reshape(-1,28,28,1)\ndf_test=df_test.values.reshape(-1,28,28,1)\n","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# CNN"},{"metadata":{},"cell_type":"markdown","source":"# Define the model\nI used the Keras Sequential API, where you have just to add one layer at a time, starting from the input.\n\nThe first is the convolutional (Conv2D) layer. It is like a set of learnable filters. I choosed to set 32 filters for the two firsts conv2D layers and 64 filters for the two last ones. Each filter transforms a part of the image (defined by the kernel size) using the kernel filter. The kernel filter matrix is applied on the whole image. Filters can be seen as a transformation of the image.\n\nThe CNN can isolate features that are useful everywhere from these transformed images (feature maps).\n\nThe second important layer in CNN is the pooling (MaxPool2D) layer. This layer simply acts as a downsampling filter. It looks at the 2 neighboring pixels and picks the maximal value. These are used to reduce computational cost, and to some extent also reduce overfitting. We have to choose the pooling size (i.e the area size pooled each time) more the pooling dimension is high, more the downsampling is important.\n\nCombining convolutional and pooling layers, CNN are able to combine local features and learn more global features of the image.\n\nDropout is a regularization method, where a proportion of nodes in the layer are randomly ignored (setting their wieghts to zero) for each training sample. This drops randomly a propotion of the network and forces the network to learn features in a distributed way. This technique also improves generalization and reduces the overfitting.\n\n'relu' is the rectifier (activation function max(0,x). The rectifier activation function is used to add non linearity to the network.\n\nThe Flatten layer is use to convert the final feature maps into a one single 1D vector. This flattening step is needed so that you can make use of fully connected layers after some convolutional/maxpool layers. It combines all the found local features of the previous convolutional layers.\n\nIn the end i used the features in two fully-connected (Dense) layers which is just artificial an neural networks (ANN) classifier. In the last layer(Dense(10,activation=\"softmax\")) the net outputs distribution of probability of each class."},{"metadata":{"trusted":true},"cell_type":"code","source":"model=keras.models.Sequential([\n    keras.layers.Conv2D(32,kernel_size=(3,3),input_shape=(28,28,1)),\n    keras.layers.Conv2D(32,kernel_size=(3,3),activation='relu'),\n    keras.layers.MaxPooling2D((2,2)),\n    keras.layers.Dropout(0.2),\n    \n    keras.layers.Conv2D(64,kernel_size=(3,3),activation='relu'),\n    keras.layers.Conv2D(64,kernel_size=(3,3),activation='relu'),\n    keras.layers.MaxPooling2D((2,2)),\n    keras.layers.Dropout(0.2),\n    \n    keras.layers.Flatten(),\n    keras.layers.Dense(480,activation='relu'),\n    keras.layers.Dropout(0.5),\n    keras.layers.Dense(10,activation='softmax')\n    ])\n\nmodel.compile(optimizer='adam',loss='sparse_categorical_crossentropy',metrics=['accuracy'])\n\nmodel.summary()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# fit model"},{"metadata":{"trusted":true},"cell_type":"code","source":"model.fit(X_train,y_train,epochs=3,validation_split=0.2)","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}