{"cells":[{"metadata":{"_uuid":"963422dcaf65b2d4022e0e92231ff05e241d52f8"},"cell_type":"markdown","source":"**This is my first kernel and I want all of your support.  Please upvote or comment your views if you find it useful. Your views will surely motivate me to write and work on more kernels. Thanks so much.**\n\n**What is Deep Learning?** \n\nDeep Learing referes to neural networks with multiple hidden layers that can learn increasingly abstract representation of input data. Neural network with many hidden layers is known as deep neural network. Deep neural networks with many hidden layers can sequentially learn more complex features from the raw image.\n* first hidden layers might learn local features from the input image.\n*  then each subsequent hidden layers learns more complex features.\n* and the last layer can classify image into categories. \n\nThese types of deep neural networks are called as **Convolutional Neural Networks (CNN)**\n\n**Where Deep Learning is used? **\n\nDeep learning is mostly used in image recognition. Apart from that there are many applications of deep learning such as:\n* Speech recognition.\n* Natural language processing. \n* Self-driving vehicle. \n* Computer vision and so on.\n\nThis kernel will help you dive deep into convolutional neural network (CNN) with the python coding using the Digit Recognizer MNIST datasets.\nCNN can be build using the following steps:\n* Data Preparation\n* Data Modeling using CNN\n* Prediction and Submission\n\nLets begin the journey!\n"},{"metadata":{"_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","trusted":true},"cell_type":"code","source":"# importing required libraries. \nfrom keras.models import Sequential\nfrom keras.layers import Conv2D\nfrom keras.layers import MaxPooling2D\nfrom keras.layers import Flatten\nfrom keras.layers import Dense\nfrom keras.utils.np_utils import to_categorical\n\nimport numpy as np\nimport pandas as pd\nimport matplotlib.pyplot as plt\n%matplotlib inline","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"fda6c13c2a586249603475e0d127ed1043cf4ebc"},"cell_type":"markdown","source":"**1. Data Preparation**\n* **Load Data**"},{"metadata":{"trusted":true,"_uuid":"c7992aaefd6f67e386d3b5eeaee9c927c09d61a0"},"cell_type":"code","source":"# Load Train and Test data\ntrain = pd.read_csv(\"../input/digit-recognizer/train.csv\")\ntest = pd.read_csv(\"../input/digit-recognizer/test.csv\")","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"3598b103a9449bb9f89d6940bc382ab69c0494a9"},"cell_type":"code","source":"#checking the shape of train and test dataset\ntrain.shape","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"fc07b670f31ea405225a51b01afd0e8bec7cf760"},"cell_type":"code","source":"test.shape","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"4e5331976a65208bc473d2c295313226efe82f01"},"cell_type":"code","source":"train_label = train.label.values\n\n# Dropping 'abel' column from train dataset\ntrain = train.drop(\"label\", axis = 1).values\n\ntrain.shape","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"dfd5025c45f555d5ae3f91ead603ca3168167a9b"},"cell_type":"markdown","source":"* **Check for missing and null values **"},{"metadata":{"trusted":true,"_uuid":"b3d411e0e819e081f628e6c76f7abd35d865a634"},"cell_type":"code","source":"# train dataset null value check\nnp.isnan(train).sum()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"6bdea0fa000be374ee760d9bbbc52b10b03af268"},"cell_type":"code","source":"# test dataset null value check\ntest.isnull().any().describe()","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"196e084df2d2eea8bddaecfa7c7201eb2208a434"},"cell_type":"markdown","source":"* **Normalization**\n\nNormalization accelerates training speed. As the range of the values of the features gets narrowed down to a particular range because of normalization, its easy to perform computations over a smaller range of values. "},{"metadata":{"trusted":true,"_uuid":"b2e1d39843724f4dd45375212320851115a2bd86"},"cell_type":"code","source":"train = train/255.0\ntest= test/255.0","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"24f14504a5606d738aef30863ab78d629619b79a"},"cell_type":"markdown","source":"* **Visualization of train dataset images**"},{"metadata":{"trusted":true,"_uuid":"1897bb52a2208b748cea1151d0095906cb43dc24"},"cell_type":"code","source":"fig1, ax1 = plt.subplots(1,15, figsize=(15,10))\nfor i in range(15):\n    # reshaping the images into 28*28 shape\n    ax1[i].imshow(train[i].reshape((28,28)))\n    ax1[i].axis('off')\n    ax1[i].set_title(train_label[i]) ","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"f0990b95cb4849a898d6a68390f7c2f3bd572547"},"cell_type":"markdown","source":"* **Reshaping**\n\nReshaping images into 3 dimensions with 28*28*1. Here **'1'** is the number of channel used. As MNIST images are gray scaled, so it only use one channel. "},{"metadata":{"trusted":true,"_uuid":"09bcb244913f439df7f72d989464b47a1619f43b"},"cell_type":"code","source":"train_image =np.array(train).reshape(-1,28,28,1)\ntest_image =np.array(test).reshape(-1,28,28,1)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"e3cd435d31ad320d125c7a4bbd170d482763845f"},"cell_type":"code","source":"train_image.shape","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"eff664e72f24788a970af822621d4a9e0936360a"},"cell_type":"code","source":"test_image.shape","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"783daa80760f6cb95615c3b9b742b9498f664752"},"cell_type":"markdown","source":"* **Label Encoding**"},{"metadata":{"trusted":true,"_uuid":"96b45ee405ebb62a6d822cdb83d63de6b1f98cb8"},"cell_type":"code","source":"# first checkin the shape of train_label\ntrain_label.shape","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"85cd36a445d1ed30800a706663ff52363573174a"},"cell_type":"code","source":"# Encoding labels to one hot encoder\ntrain_label = to_categorical(train_label)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"09adfa9d18d52b5fb41d2731dcf215e09537d81a"},"cell_type":"code","source":"# again checking the shape of train_label\ntrain_label.shape","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"10d1520647385d6007df6acfda50786a1986b180"},"cell_type":"markdown","source":"As there are 10 categories from 0 to 9, after one hot encoding, train_label is categorized into 10 columns representing 10 categories. "},{"metadata":{"_uuid":"077aeca709c25f7cb449467ca9c2c357a03f7a3f"},"cell_type":"markdown","source":"**2. Data Modeling using CNN**\n\nThe following are the steps in building Convolutional Neural Network (CNN):\n\n**1. Convolution**\n\n**2. Pooling**\n\n**3. Flattening**\n\n**4. Full Connection**"},{"metadata":{"_uuid":"19b9a487334fd6ab6aba073901f07775219ad63f"},"cell_type":"markdown","source":"<img src=\"https://imgur.com/nME3ghU.jpg\">"},{"metadata":{"_uuid":"b2be07bb6dcf65f4f021d335bfdfe8c36ea1292e"},"cell_type":"markdown","source":"* **Convolution**\n\nThere are **3** steps in this process of Convolution:\n\n1. **Input Image**: The input image is the image being detected. \n\n2. **Feature detector** : It is also called as Kernel or Filter. It is a matrix generally of 3*3 Size.\n\n3. **Feature map** : The matrix representation of the input image is **multiplied** element-wise with the feature detector to produce a feature map. Feature map aims to reduce the size of the image and make processing faster and easier. \n\nSome of the features of the image are lost in this step. However, the main features of the image that are important in image detection are retained. These features are the ones that are unique to identifying that specific object. The way we prevent loss of image information is by having **many feature maps**. Each feature map detects the location of certain features in the image.\n\n* **Apply ReLU (Rectifier Linear Unit)** : \nIn this step we apply the rectifier function to **increase non-linearity** in the CNN. Images are made of different objects that are not linear to each other. Without applying this function the image classification will be treated as a linear problem while it is actually a non-linear one."},{"metadata":{"trusted":true,"_uuid":"196f854c2c6330d4833b74f944d00ddb02f9815d"},"cell_type":"code","source":"# Initialising the CNN\nclassifier = Sequential()\n\n# Step 1 - Convolution\nclassifier.add(Conv2D(32, (3, 3), padding = 'Same', activation=\"relu\", input_shape=(28, 28, 1)))","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"3062e97d7a8748b8e1dade402c062825dfc0efb7"},"cell_type":"markdown","source":"* **Pooling**\n\nPooling enables the CNN to detect features in various images irrespective of the difference in lighting in the pictures and different angles of the images. \n\nThere are different types of pooling, for example, **max pooling** and **min pooling**. \n\nMax pooling works by placing a matrix of 2x2 on the feature map and picking the largest value in that box. The 2x2 matrix is moved from left to right through the entire feature map picking the largest value in each pass. These values then form a new matrix called a **Pooled Feature Map**. Max pooling works to preserve the main features while also reducing the size of the image. This helps reduce overfitting, which would occur if the CNN is given too much information, especially if that information is not relevant in classifying the image."},{"metadata":{"trusted":true,"_uuid":"43a234f42a84368a84ce98240cc3d3fda54d9008"},"cell_type":"code","source":"# Step 2 - Pooling\nclassifier.add(MaxPooling2D(pool_size = (2, 2)))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"b61ff8922d46ede5029c4a3a6804b141f64a7327"},"cell_type":"code","source":"#adding another convulationary layer. Since we are adding another convolution layer, we are not required \n#- to pass input shape parameter\nclassifier.add(Conv2D(32, (3, 3), activation=\"relu\"))\nclassifier.add(MaxPooling2D(pool_size = (2, 2)))","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"dc2af41a665c96606f3ab618a808592b1950727e"},"cell_type":"markdown","source":"* **Flattening**\n\nFlattening involves transforming the entire pooled feature map matrix into a single column which is then fed to the neural network for processing."},{"metadata":{"trusted":true,"_uuid":"7c11c433f09f21c8f11647abc11062d1495fd065"},"cell_type":"code","source":"# Step 3 - Flattening\nclassifier.add(Flatten())","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"4731c1b9f2b1e66deebda54c026df6c536f6a3eb"},"cell_type":"markdown","source":"* **Full Connection**\n\nThis step is made up of the **input layer**, **the fully connected layer,** and **the output layer**. The fully connected layer is similar to the hidden layer in ANNs but in this case it’s fully connected. The output layer is where we get the predicted classes or category"},{"metadata":{"trusted":true,"_uuid":"3c02fb31c9951d84307564d4c8fa64f8fb503be2"},"cell_type":"code","source":"#Step 4 - Full connection. \n#Here output_dim is the number of neurons per layer\nclassifier.add(Dense(output_dim = 256, activation = 'relu'))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"79b98bd7bfb0facade61242fe6077b9bb236c0a8"},"cell_type":"code","source":"#output layer\n# here we are using 10 output_dim (neurons) because there are 10 classes\nclassifier.add(Dense(output_dim = 10, activation = 'softmax'))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"0851ccf59d7d4e1bee904fb306e7afd19827da62"},"cell_type":"code","source":"# Compilint the CNN\nclassifier.compile(optimizer = 'adam', loss = 'categorical_crossentropy', metrics = ['accuracy'])","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"e1a44433c462ebbe35269a8eb0d4301eedeb99ce"},"cell_type":"code","source":"epochs=30\nbatch_size=90\n\n# fitting the CNN model \nclassifier.fit(train_image, train_label, batch_size=batch_size, epochs=epochs)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"40d485a684a48708719edef2bbadeef01e0fda26"},"cell_type":"markdown","source":"**3. Prediction and Submission**"},{"metadata":{"trusted":true,"_uuid":"6d16566da37ce9732b98d97d29f2427af6425f5e"},"cell_type":"code","source":"#Prediction\nresults = classifier.predict(test_image)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"22963ddd9ee0cbb7000472caa564230871840e08"},"cell_type":"code","source":"# Submission\npred = []\nnumTest = results.shape[0]\nfor i in range(numTest):\n    pred.append(np.argmax(results[i])) \npredictions = np.array(pred) \n\nsample_submission = pd.read_csv('../input/digit-recognizer/sample_submission.csv')\n\nresult=pd.DataFrame({'ImageId':sample_submission.ImageId, 'Label':predictions})\nresult.to_csv(\"submission.csv\",index=False)\nprint(result)","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"name":"python","version":"3.6.6","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"}},"nbformat":4,"nbformat_minor":1}