{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# Aim of this Notebook\n\n**Welcome everyone!**\n\n**In this notebook, I will be dealing with the Digit Recognition by using CNN.**\n\n**For the first step of this notebook, I will do some visualizations.**\n\n**For the next step, I will train a CNN models for the accomplish my aim.**\n\n**I am open to feedback and suggestions, feel free to comment your feedback and suggestions on the comment section or contact me.**\n\n**So, let's get started!**","metadata":{}},{"cell_type":"markdown","source":"# Importing Libraries","metadata":{}},{"cell_type":"code","source":"import os\nimport pandas as pd\nimport seaborn as sns\nimport numpy as np\nfrom keras.models import Sequential\nfrom keras.layers import Dense, Dropout, Flatten\nfrom keras.layers import Conv2D, MaxPooling2D\nimport matplotlib.pyplot as plt\nimport tensorflow as tf\nfrom sklearn.model_selection import train_test_split\nfrom keras.utils.np_utils import to_categorical\n\nkeras = tf.keras\nimport os\nfor dirname, _, filenames in os.walk('/kaggle/input'):\n    for filename in filenames:\n        print(os.path.join(dirname, filename))\n        ","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Reading Datasets","metadata":{}},{"cell_type":"code","source":"train = pd.read_csv('../input/digit-recognizer/train.csv')\ntest = pd.read_csv('../input/digit-recognizer/test.csv')\ntrain.head()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print('Shape of the train data:', train.shape)\nprint('Shape of the test data:', test.shape)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_unique = pd.DataFrame(train['label'].unique()).sort_values(by=0)\ntrain_unique","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**Train dataset includes all the labels from the 0 to 9.**","metadata":{}},{"cell_type":"markdown","source":"# Simple Visualization","metadata":{}},{"cell_type":"code","source":"plt.figure(figsize=(15, 8))\nsplot = sns.countplot(data=train, x='label',\n                      order=train['label'].value_counts().index,\n                      edgecolor=(0, 0, 0),\n                      linewidth=2)\n\nfor p in splot.patches:\n    splot.annotate(format(p.get_height(), '.1f'),\n                   (p.get_x() + p.get_width() / 2., p.get_height()),\n                   ha='center', va='center',\n                   xytext=(0, 9),\n                   textcoords='offset points')\nplt.ylabel('Frequency of the Labels', fontsize=14)\nplt.xlabel('Labels', fontsize=14)\nplt.xticks(fontsize=12)\nplt.yticks(fontsize=12)\nplt.title('Distribution of the Labels of the Train Dataset', fontsize=20)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**According to the figure above, the x-axis represents that the labels of the training dataset are ordered from the high frequent one to lesser. The y-axis represents that the frequency of the labels.**\n\n**As seen from the figure, the number 1 is the most frequent label in the training dataset.**\n\n**According to the graph above, the dataset labels are pretty equally distributed. There are not major frequency differences between the labels.**","metadata":{}},{"cell_type":"markdown","source":"# Feature and Label","metadata":{}},{"cell_type":"code","source":"y = train['label']  # Labels\nX = train.drop(['label'], axis=1)  # Features\nX.head()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Let's look at the some exapmles of the feature set\nfor i in range(9):\n    plt.subplot(330 + 1 + i)\n    fig = X.iloc[i].values.reshape((28, 28))\n    plt.imshow(fig)\n    plt.axis('off')\nplt.show()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Normalizing and Reshaping","metadata":{}},{"cell_type":"code","source":"# Normalizing the dataset\nX = X / 255.0\n\n# CNN expects 3D inputs, so I will convert my data tto 3D form\ny = y.values.reshape(-1, 1)\nX = X.values.reshape(-1, 28, 28, 1)\n\nprint('Shape of the X matrix:', X.shape)\nprint('Shape of the y matrix:', y.shape)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**At the end of this step, we have 3D inputs.**","metadata":{}},{"cell_type":"code","source":"# Encoding the labels\ny = to_categorical(y, num_classes=10)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Train-Test Split","metadata":{}},{"cell_type":"code","source":"# Train-Test split\ntrainX, valX, trainY, valY = train_test_split(X, y, test_size=0.2, shuffle=True, random_state=13)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# CNN Model Part-1","metadata":{}},{"cell_type":"code","source":"model = Sequential()\nmodel.add(Conv2D(32, (3, 3), activation='relu', kernel_initializer='he_uniform', input_shape=(trainX.shape[1:])))\nmodel.add(Conv2D(64, (3, 3), activation='relu'))\nmodel.add(MaxPooling2D(pool_size=(2, 2)))\nmodel.add(Dropout(0.25))\n\nmodel.add(Conv2D(64, (3, 3), activation='relu'))\nmodel.add(MaxPooling2D(pool_size=(2, 2)))\nmodel.add(Dropout(0.25))\n\nmodel.add(Flatten())\nmodel.add(Dense(128, activation='relu'))\nmodel.add(Dropout(0.3))\n\n# Output layer\nmodel.add(Dense(10, activation='softmax'))","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# CNN Model Part-2","metadata":{}},{"cell_type":"code","source":"model.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['accuracy'])  # Adam Optimizer\n\nearly_stopping = tf.keras.callbacks.EarlyStopping(monitor='loss', mode='auto', patience=2,\n                                                 restore_best_weights=True)\n\nhist = model.fit(trainX, trainY, epochs=10, batch_size=64, callbacks=[early_stopping],\n                 verbose=1, validation_data=(valX, valY))\nhist.history.keys()\nmodel.summary()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Evaluation of the model\n\nmodel.evaluate(valX, valY)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Performance Graphs","metadata":{}},{"cell_type":"code","source":"plt.figure(figsize=(15, 8))\nplt.plot(hist.history['loss'])\nplt.plot(hist.history['val_loss'])\nplt.title('Losses of the Model', fontsize=20)\nplt.legend(['Train Loss', 'Validation Loss'], loc='upper right')\nplt.xlabel('Epochs', fontsize=12)\nplt.ylabel('Losses', fontsize=12)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plt.figure(figsize=(15, 8))\nplt.plot(hist.history['accuracy'])\nplt.plot(hist.history['val_accuracy'])\nplt.title('Accuracy of the Model', fontsize=20)\nplt.legend(['Train Accuracy', 'Validation Accuracy'], loc='upper right')\nplt.xlabel('Epochs', fontsize=12)\nplt.ylabel('Losses', fontsize=12)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Test Data","metadata":{}},{"cell_type":"code","source":"test = pd.read_csv('../input/digit-recognizer/test.csv')\ntest = test / 255.0\ntest = test.values.reshape(-1, 28, 28, 1)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"predictions = model.predict(test)\npredictions = np.argmax(predictions, axis=1)\n\npredictions = pd.DataFrame(predictions)\npredictions","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"sample_sub = pd.read_csv(\"../input/digit-recognizer/sample_submission.csv\")\nsample_sub = pd.concat([sample_sub['ImageId'], predictions], axis=1)\nsample_sub.columns = ['ImageId', 'Label']\nsample_sub","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"sample_sub.to_csv('ata_submission.csv', index=False)","metadata":{"trusted":true},"execution_count":null,"outputs":[]}]}