{"cells":[{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"cell_type":"code","source":"import numpy as np \nimport pandas as pd \nimport matplotlib.pyplot as plt\n\nfrom tensorflow import keras\nfrom keras.models import Sequential\nfrom keras.layers import Conv2D, MaxPooling2D, Flatten, Dense, experimental, Dropout\nfrom sklearn.model_selection import train_test_split","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Data processing","execution_count":null},{"metadata":{"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","trusted":true},"cell_type":"code","source":"train_df = pd.read_csv('/kaggle/input/digit-recognizer/train.csv')\ntest_df = pd.read_csv('/kaggle/input/digit-recognizer/test.csv')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"train_df.head()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"We will split the previous dataframe: 80% for training and 20% for validation.\nThe features used for the training are the 784 pixels of each images, the labels are given by the column 'label'.\nWe will also work with numpy arrays","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"# Spliting and shuffling our dataset.\ntrain_df, val_df = train_test_split(train_df, test_size=0.2)\n\n# Forming np arrays of labels and features.\n\ntrain_labels = np.array(train_df.pop('label'))\nval_labels = np.array(val_df.pop('label'))\n\ntrain_features=np.array(train_df)\nval_features=np.array(val_df)\ntest_features=np.array(test_df)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Each images has a shape 28x28x1 (1 because we're working with black and white images), but in the dataframe the pixels are flattened. \nWe would like to reshape them back to their original shape so we can apply them as entries of our convolutional neural network.\n\nWe'll also applying one-hot encoding to our labels.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"height=28\nwidth=28\nnum_classes=10\n\ntrain_features=train_features.reshape(train_features.shape[0],height,width,1)\nval_features=val_features.reshape(val_features.shape[0],height,width,1)\n\ntrain_labels = keras.utils.to_categorical(train_labels, num_classes)\nval_labels = keras.utils.to_categorical(val_labels, num_classes)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"print('Training features shape: ',train_features.shape)\nprint('Validation features shape: ',val_features.shape)\n\nprint('Training labels shape: ', train_labels.shape)\nprint('Validation labels shape: ', val_labels.shape)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"As expected, each features is an image of shape (28,28,1),\nand each label has is a vector of size 10 (number of classes, 10 since there are 10 digits).\n\nWe can plot of the images used for training, and see its label (one-hot encoded)","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"plt.imshow(train_features[1].reshape(28, 28))\nplt.title(\"labeled one-hot vector: {}\".format(train_labels[1]))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Building the model","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"It's time to build the model.\nThe first layer we'll used is a preprocessing layer that allow us to normalize the pixels between 0 1. \nWe could do this outside the model but I wanted to try this layer so i opted for that.\nNormalizing our datas help us to train the model faster and reduces the chance of getting stuck in a local optima (example: for the sigmoid function the gradient is close to 0 for high values while it's high close to 0)\n\nWe apply several layers of Conv2D + Maxpooling2D, which help the model identifying visual features while reducing the number of features in input of of dense neural network.\nI used almost the same architecture as a model used to classify flowers: https://www.tensorflow.org/tutorials/images/classification\n\nI also added a Dropout layer to reduce overfitting. \n","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"model = Sequential([\n  experimental.preprocessing.Rescaling(1./255, input_shape=(train_features.shape[1:])),\n  Conv2D(16, 3, padding='same', activation='relu'),\n  MaxPooling2D(),\n  Conv2D(32, 3, padding='same', activation='relu'),\n  MaxPooling2D(),\n  Conv2D(64, 3, padding='same', activation='relu'),\n  MaxPooling2D(),\n  Flatten(),\n  Dense(128, activation='relu'),\n  Dropout(0.2),\n  Dense(num_classes, activation='softmax')\n])","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"The loss used is categorical crossentropy since we are in a classification problem with more that 2 classes.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"model.compile(optimizer='adam',\n              loss=keras.losses.categorical_crossentropy,\n              metrics=['accuracy'])","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"model.summary()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"epochs=20\nbatch_size=32\n\nhistory = model.fit(\n    train_features,\n    train_labels,\n    epochs=epochs,\n    batch_size=batch_size,\n    validation_data=(val_features, val_labels))\n","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"We obtain a validation accuracy of 99.00% after training. Not bad!","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"# Using the model on our test dataset and submitting the predictions ","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"test_features=test_features.reshape(test_features.shape[0],height,width,1)\npredictions = np.argmax(model.predict(test_features),axis = 1)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"predictions","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"submission_dataframe = pd.DataFrame({\"ImageId\" : range(1, test_features.shape[0]+1), \"Label\" : predictions})\nsubmission_dataframe.to_csv(\"submission.csv\", index = False)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Thank you for reading!","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"if you have any question feel free to ask!\nThis was my first Kaggle notebook.\nAn upvote would be appreciatted since i'm beginning. Thank you!","execution_count":null}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}