{"cells":[{"metadata":{},"cell_type":"markdown","source":"### Before the trip begin\n\n#####  If you like my work, please hit upvote since it will keep me motivated","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"* **1. Introduction**   \n* **2. Data Preparation**\n   * 2.1 Import Libaries\n   * 2.2 Load Data\n   * 2.3 Normalization\n   * 2.4 Reshape\n   * 2.5 Check the Data\n   * 2.6 Split to train and test\n* **3. Data Augmentation**\n* **4. CNN**\n   * 4.1 Label encoding\n   * 4.2 Build the Network\n   * 4.3 Evaluate the Model\n* **5. Let's Predict**","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"# 1. Introduction\n![](https://miro.medium.com/max/3744/1*SGPGG7oeSvVlV5sOSQ2iZw.png)\n\nA Convolutional Neural Network (ConvNet/CNN) is a Deep Learning algorithm which can take in an input image, assign importance (learnable weights and biases) to various aspects/objects in the image and be able to differentiate one from the other. The pre-processing required in a ConvNet is much lower as compared to other classification algorithms. While in primitive methods filters are hand-engineered, with enough training, ConvNets have the ability to learn these filters/characteristics.\n","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"# 2. Data Preparation\n## 2.1 Import libraries","execution_count":null},{"metadata":{"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","trusted":true},"cell_type":"code","source":"import numpy as np\nimport pandas as pd \nimport matplotlib.pyplot as plt\nimport seaborn as sns\nfrom keras.models import Sequential\nfrom keras.layers import Dense\nfrom keras.optimizers import Adam\nfrom keras.utils.np_utils import to_categorical\nfrom keras.layers import Dropout, Flatten ,BatchNormalization , MaxPool2D\nfrom keras.layers.convolutional import Conv2D\nfrom keras.preprocessing.image import ImageDataGenerator\nfrom sklearn.model_selection import train_test_split\nimport os","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## 2.2 Load Data","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"ds_train=pd.read_csv(\"../input/digit-recognizer/train.csv\")\nds_test=pd.read_csv(\"../input/digit-recognizer/test.csv\")","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"print(\"Train dataset :\",ds_train.shape)\n#print(\"Test dataset :\",ds_test.shape)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"X=ds_train.drop(['label'],axis=1)\ny=ds_train['label']\nprint(\"done!\")","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"sns.countplot(y)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"As you can see we have the count of all label are almost in the save interval [3500,5000], that's good for the training, for example if we had just 1000 samples of label 1 that will be a problem , the model will find difficulties to detect label 1\"less accuracy \", so that's not going to happend everything look fine .","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"## 2.3 Normalization","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"We will use Grayscale normalization the model will work faster if the interval of data between [0,1] instead of [0,255]","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"X=X/255.0\nds_test=ds_test/255.0","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## 2.4 Reshape","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"From 1D vectors to 28×28×1 (3D)","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"X = X.values.reshape(-1,28,28,1)\nds_test = ds_test.values.reshape(-1,28,28,1)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## 2.5 check the Dataset","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"Before we move forward we need to check our dataset first if there anything wrong and the label are correct.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"fig,axs=plt.subplots(1,5,figsize=(20,5))\nfig.tight_layout()\n\nfor i in range(5):\n    axs[i].imshow(X[i].reshape(28,28))\n    axs[i].axis('off')\n    axs[i].set_title(y[i])\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Everything look fine !","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"## 2.6 Split to train and test","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = 0.12, random_state=4)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"len(X_train) #we will set batch size to 56 and step per epouch to 660","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# 3. Data Augmentation","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"![](http://www.pyimagesearch.com/wp-content/uploads/2019/07/keras_data_augmentation_in_place.png)","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"We will use data augmentation is called in-place data augmentation, the process is input batch of images to the IDG(ImageDataGenerator), and transforms each image in the batch using random translation (given parameter),at the end the transformed batch returned to the calling function.\nThe idea here is to avoid Over-fitting , we won't let the model train with the same pictures for example the model should train using different pictures of label 1.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"dataGen= ImageDataGenerator(width_shift_range=0.1,   \n                            height_shift_range=0.1,\n                            zoom_range=0.2,  \n                            shear_range=0.1, \n                            rotation_range=10)  \ndataGen.fit(X_train)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### After Data Augmentation","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"batches= dataGen.flow(X_train,y_train,batch_size=20)\nX_batch,y_batch = next(batches)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"fig,axs=plt.subplots(1,5,figsize=(20,5))\nfig.tight_layout()\n\nfor i in range(5):\n    axs[i].imshow(X_batch[i].reshape(28,28))\n    axs[i].axis('off')\n    axs[i].set_title(y_batch[i])\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Cool!!","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"# 4. CNN","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"## 4.1 Label encoding","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"y_train = to_categorical(y_train,10)\ny_test = to_categorical(y_test,10)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## 4.2 Build the Network ","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"model = Sequential()\n\n#First\nmodel.add(Conv2D(filters = 64, kernel_size = (3,3) ,activation ='relu', input_shape = (28,28,1)))\nmodel.add(Conv2D(filters = 56, kernel_size = (3,3),activation ='relu'))\nmodel.add(BatchNormalization())\nmodel.add(MaxPool2D(pool_size=(2,2)))\nmodel.add(Dropout(0.2))\n\n#Second\nmodel.add(Conv2D(filters = 64, kernel_size = (3,3),activation ='relu'))\nmodel.add(Conv2D(filters = 48, kernel_size = (3,3),activation ='relu'))\nmodel.add(Conv2D(filters = 32, kernel_size = (3,3),activation ='relu'))\nmodel.add(BatchNormalization())\nmodel.add(MaxPool2D(pool_size=(2,2)))\nmodel.add(Dropout(0.2))\n\n#Third\nmodel.add(Flatten())\nmodel.add(Dense(256, activation = \"relu\"))\nmodel.add(Dense(128, activation = \"relu\"))\nmodel.add(Dense(64, activation = \"relu\"))\nmodel.add(Dropout(0.4))\n\n#Output\nmodel.add(Dense(10, activation = \"softmax\"))\n\n\nmodel.compile(Adam(lr=0.001),loss='categorical_crossentropy',metrics=['accuracy'])","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"print(model.summary())","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"#you can set more than 10 epochs to get more accuracy \nhistory = model.fit_generator(dataGen.flow(X_train,y_train, batch_size=56),\n                              epochs = 10, validation_data = (X_test,y_test),\n                              verbose = 2, steps_per_epoch=660)\n\n\n# For 10 epochs we get \n#  loss: 0.0614 \n#  accuracy: 0.9838 \n#  val_loss: 0.0364 \n#  val_accuracy: 0.9921","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## 4.3 Evaluate The Model","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"plt.figure()\nfig,(ax1, ax2)=plt.subplots(1,2,figsize=(19,7))\nax1.plot(history.history['loss'])\nax1.plot(history.history['val_loss'])\nax1.legend(['training','validation'])\nax1.set_title('loss')\nax1.set_xlabel('epoch')\n\nax2.plot(history.history['accuracy'])\nax2.plot(history.history['val_accuracy'])\nax2.legend(['training','validation'])\nax2.set_title('Acurracy')\nax2.set_xlabel('epoch')\n\n\n\nscore =model.evaluate(X_test,y_test,verbose=0)\nprint('Test Score:',score[0])\nprint('Test Accuracy:',score[1])","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"As you can see when we only used 10 epoch the model look great and we tried to avoid Overfitting and Underfitting","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"# 5. Let's Predict","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"results = model.predict(ds_test)\n\nresults = np.argmax(results,axis = 1)\n\nresults = pd.Series(results,name=\"Label\")\n\nsubmission = pd.concat([pd.Series(range(1,28001),name = \"ImageId\"),results],axis = 1)\n\nsubmission.to_csv(\"CNN_Digit_Recognizer.csv\",index=False)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"###  If you like my work, please hit upvote since it will keep me motivated","execution_count":null}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}