{"cells":[{"metadata":{},"cell_type":"markdown","source":"# MNIST Character Recognition using Deep Learning\n\n* First notebook on Kaggle.\n\n* The following implementation uses a Convolutional Neural Network for Handwritten Character Recognition on the MNIST Dataset.\n\n* More detailed information about the codes are in respective markdowns."},{"metadata":{"trusted":true},"cell_type":"code","source":"#Importing libraries\n\nimport numpy as np\nimport pandas as pd\nimport matplotlib.pyplot as plt\n\nfrom keras.models import Sequential\n\nfrom keras.utils.np_utils import to_categorical\nfrom sklearn.model_selection import train_test_split\nfrom keras.layers import Dense, Dropout, Flatten, Conv2D, MaxPool2D","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Data Preparation\n\n### The `train.csv` and `test.csv` files are first loaded into `training` and `testing` variables using Pandas."},{"metadata":{"trusted":true},"cell_type":"code","source":"#Loading data\n\ntraining = pd.read_csv('/kaggle/input/digit-recognizer/train.csv')\ntesting = pd.read_csv('/kaggle/input/digit-recognizer/test.csv')\n","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"\n### The dataset is divided into the labels and data - pixel values of each image and converted to **numpy arrays** for easier reshaping further in the code."},{"metadata":{"trusted":true},"cell_type":"code","source":"#Dividing data and corresponding label into x_train and y_train\ntrainingLabels = training['label']\ntrainingData = training.drop(labels=['label'], axis=1)\n\n#Converting data to numpy array\ntrainingLabels, trainingData = trainingLabels.values, trainingData.values","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Normalization is performed on the pixel values to bring them in a range of [0, 1]. Normalization is done as a CNN converges much faster and easier with smaller values."},{"metadata":{"trusted":true},"cell_type":"code","source":"#Normalizing Data\n\ntrainingData = trainingData / 255.0\ntesting = testing / 255.0","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Image data is reshaped to image in 3 dimensions - height 28px width 28px channel = 1"},{"metadata":{"trusted":true},"cell_type":"code","source":"#Reshape data to image in 3 dimensions\ntrainingData = trainingData.reshape(-1, 28, 28, 1)\ntesting = testing.values.reshape(-1, 28, 28, 1)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Each label is converted to a One Hot Vector of 10 classes where 1 represents the class a digit belongs to. \n\nExample - For the number 2, the one hot vector will be [0, 0, 1, 0, 0, 0, 0, 0, 0, 0]"},{"metadata":{"trusted":true},"cell_type":"code","source":"#One Hot Encoding\ntrainingLabels = to_categorical(trainingLabels, num_classes = 10)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Training Data is split into training and validation data in the ratio of 9:1 "},{"metadata":{"trusted":true},"cell_type":"code","source":"#Splitting data into training and validation datasets (9:1)\nx_train, x_val, y_train, y_val = train_test_split(trainingData, trainingLabels, test_size = 0.1, random_state=2)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Model Building\n\n* The CNN Architecture consists of 2 Convolutional 2D Layers with 64 and 32 filters respectively and a kernel size of 3.\n* Each Conv2D layer is followed by a MaxPooling Layer of size 2\n* Dropout of 0.25 after each set of Conv2D and MaxPool Layer prevents overfitting by disconnecting neurons in the network.\n* The final two layers are a Flatten Layer - to flatten data into a single dimension vector and an output dense layer with a 'softmax' activation function and 10 classes."},{"metadata":{"trusted":true},"cell_type":"code","source":"#Building CNN Model\nmodel = Sequential()\n\n#First Layer\nmodel.add(Conv2D(64, 3, activation = 'relu', input_shape = (28, 28, 1)))\nmodel.add(MaxPool2D(pool_size=(2,2)))\nmodel.add(Dropout(0.25))\n\n#Second Layer\nmodel.add(Conv2D(32, 3, activation = 'relu'))\nmodel.add(MaxPool2D(pool_size=(2,2), strides=(2,2)))\nmodel.add(Dropout(0.25))\n\n#Final Output Layer\nmodel.add(Flatten())\nmodel.add(Dense(10, activation = 'softmax'))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Compiling Model \n\nThe 'Adam' Optimizer was used to compile the model and loss function was a 'categorical_crossentropy'"},{"metadata":{"trusted":true},"cell_type":"code","source":"#Compiling Model\nmodel.compile(optimizer='adam' , loss=\"categorical_crossentropy\", metrics=[\"accuracy\"])","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"#Training Model \nhistory = model.fit(x_train, y_train, batch_size = 64, epochs = 10, \n                    validation_data = (x_val, y_val), verbose = 2)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"#Visualizing Model Accuracy\n\nplt.title('Model Accuracy')\nplt.plot(history.history['accuracy'])\nplt.plot(history.history['val_accuracy'])\nplt.xlabel('Epoch')\nplt.ylabel('Accuracy')\nplt.legend(['Train', 'Val'])\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"predictions = model.predict(testing[:3])\n#Printing prediction of model with labels of first 3 test images\nprint(np.argmax(predictions, axis = 1))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"def prep_test_data(raw):\n    x = raw[:,0:]\n    num_images = raw.shape[0]\n    out_x = x.reshape(num_images, 28, 28, 1)\n    out_x = out_x / 255\n    return out_x\n\nval_file = \"/kaggle/input/digit-recognizer/test.csv\"\nval_data = np.loadtxt(val_file, skiprows=1, delimiter=',')\nx_test = prep_test_data(val_data)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"predictions = model.predict_classes(x_test)\n\nindexes = [i for i in range(1,len(val_data)+1)]\noutput = pd.DataFrame({'ImageId': indexes,'Label': predictions})\noutput.to_csv('submission.csv', index=False)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"pd.read_csv('submission.csv')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}