{"cells":[{"metadata":{},"cell_type":"markdown","source":"# Handwritten Digit Recognization | MNIST Data\n\nA model to recognize and predict handwritten digits using **Tensorflow**, **Keras** and **CNN**.","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"## MNIST Data\n----\nThe MNIST database is a large database of handwritten digits that is commonly used for training various image processing systems. The database is also widely used for training and testing in the field of machine learning. It was created by \"re-mixing\" the samples from NIST's original datasets.\n\n![Mnist Data](https://external-content.duckduckgo.com/iu/?u=https%3A%2F%2Fanalyticsindiamag.com%2Fwp-content%2Fuploads%2F2017%2F12%2FMNIST-dataset.jpg&f=1&nofb=1)\n\n","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"**Tensorflow**:- \n\nTensorFlow is a free and open-source software library for dataflow and differentiable programming across a range of tasks. It is a symbolic math library, and is also used for machine learning applications such as neural networks\n\n**Keras**:-\n\nKeras is an open-source neural-network library written in Python. It is capable of running on top of TensorFlow. It focuses on being user-friendly, modular, and extensible","execution_count":null},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"cell_type":"code","source":"import numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\nimport matplotlib.pyplot as plt\nimport random\nimport tensorflow as tf\nimport keras\nfrom keras.preprocessing.image import ImageDataGenerator\nfrom keras.datasets import mnist\n\nimport os\nprint(os.listdir(\"../input\"))\n\n%matplotlib inline","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","trusted":true},"cell_type":"code","source":"df_train = pd.read_csv('/kaggle/input/digit-recognizer/train.csv')\ndf_test = pd.read_csv('/kaggle/input/digit-recognizer/test.csv')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"df_train.head()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"- Label:- it contain the actual number.","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"# Normalizing our Data\n---\nIn statistics and applications of statistics, normalization can have a range of meanings. In the simplest cases, normalization of ratings means adjusting values measured on different scales to a notionally common scale, often prior to averaging.\n\n\n![](https://external-content.duckduckgo.com/iu/?u=https%3A%2F%2Fwww.kdnuggets.com%2Fwp-content%2Fuploads%2Fconvolutional-neural-net-architecture-2.jpg&f=1&nofb=1)","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"x_train = np.array(df_train.iloc[:,1:])\nx_train = np.array([np.reshape(i, (28, 28, 1)) for i in x_train])\ny_train = np.array(df_train.iloc[:,0])","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"x_train = x_train/255.0\ny_train = keras.utils.to_categorical(y_train)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"x_test = np.array(df_test)\nx_test = np.array([np.reshape(i, (28, 28, 1)) for i in x_test])\nx_test = x_test/255.0","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Let's Visualize what do we have here.\n\nSample plots.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"# 6 random plots\n\nimg_indices = [random.randint(0,33600) for i in range(6)] \nn=0\nfig = plt.figure(figsize=[15,10])\naxes = fig.subplots(2, 3)\nfor row in range(2):\n    for col in range(3):\n        axes[row,col].imshow((x_train[img_indices[n]]).reshape((28,28)), cmap='Accent')\n        n += 1","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Splitting data for training and testing purpose","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"# Lets start with building the model\n\n<img src=\"https://blog.imarticus.org/wp-content/uploads/2020/04/deep.gif\" height=400 width=700>\n\n## Convolutional Neural Networks\n___\n    In deep learning, a convolutional neural network is a class of deep neural networks, most commonly applied to analyzing visual imagery. They are also known as shift invariant or space invariant artificial neural networks, based on their shared-weights architecture and translation invariance characteristics.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"from sklearn.model_selection import train_test_split\nX_train, X_test, Y_train, Y_test = train_test_split(x_train, y_train, test_size=0.2, stratify=y_train)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"model = keras.models.Sequential()\n\nmodel.add(keras.layers.Conv2D(filters=32, kernel_size=(3,3), kernel_initializer='random_uniform', padding='same', activation='relu', input_shape=(X_train.shape[1:])))\nmodel.add(keras.layers.Conv2D(filters=32, kernel_size=(3,3), kernel_initializer='random_uniform', padding='same', activation='relu'))\nmodel.add(keras.layers.MaxPool2D(pool_size=(2,2)))\n\nmodel.add(keras.layers.Conv2D(filters=64, kernel_size=(5,5), kernel_initializer='random_uniform', padding='same', activation='relu'))\nmodel.add(keras.layers.Conv2D(filters=64, kernel_size=(5,5), kernel_initializer='random_uniform', padding='same', activation='relu'))\nmodel.add(keras.layers.MaxPool2D(pool_size=(2,2)))\n\nmodel.add(keras.layers.Conv2D(filters=128, kernel_size=(7,7), kernel_initializer='random_uniform', padding='same', activation='relu'))\nmodel.add(keras.layers.Conv2D(filters=128, kernel_size=(7,7), kernel_initializer='random_uniform', padding='same', activation='relu'))\nmodel.add(keras.layers.MaxPool2D(pool_size=(3,3)))\n\nmodel.add(keras.layers.Conv2D(filters=256, kernel_size=(7,7), padding='same'))\nmodel.add(keras.layers.BatchNormalization())\nmodel.add(keras.layers.Activation('relu'))\n\n\nmodel.add(keras.layers.Dense(units=256, activation='relu'))\nmodel.add(keras.layers.Dropout(0.5))\nmodel.add(keras.layers.Dense(units=100, activation='relu'))\nmodel.add(keras.layers.Dropout(0.5))\nmodel.add(keras.layers.Flatten())\nmodel.add(keras.layers.Dense(units=y_train.shape[1], activation='softmax'))\n\n\nprint(model.summary())\n\nmodel.compile(loss='categorical_crossentropy',optimizer='Adam', metrics=['accuracy'])","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"from tensorflow.keras.callbacks import EarlyStopping, ReduceLROnPlateau\nfrom keras.callbacks import ModelCheckpoint\n\nes = EarlyStopping(monitor='loss', mode='min', verbose=1, patience=5)\nfilepath = \"model.h5\"\nckpt = ModelCheckpoint(filepath, monitor='loss', verbose=1, save_best_only=True, mode='min')\nrlp = ReduceLROnPlateau(monitor='loss', patience=2, factor=0.2)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Configure and train the model\nhistory = model.fit(X_train, Y_train,  batch_size=500, callbacks=[es, ckpt, rlp], epochs=100, validation_data=(X_test,Y_test))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Predicted Number Vs Actual Number\n\nLets plot and see how well is our model's prediction.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"# 40 random plots to test our model\n\nimg_indices = [random.randint(0,1000) for i in range(40)]\nn=0\nfig = plt.figure(figsize=[30,50])\naxes = fig.subplots(10, 4)\nfor row in range(10):\n    for col in range(4):\n        axes[row,col].imshow((X_test[img_indices[n]][:,:,0]).reshape((28,28)), cmap='Accent')\n        predicted_num = np.argmax(model.predict(X_test[img_indices[n]:img_indices[n]+1]))\n        actual_num = np.argmax(Y_test[img_indices[n]])\n        axes[row,col].set_title(\"{}. Predicted = {} | Actual = {}\".format(n+1, predicted_num, actual_num), fontsize=15)\n        n += 1","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Let's Save and submit what we made.","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"* Saving out predictions to `submission.csv`","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"id_img = []\nlabel = []\nfor i in range(len(x_test)):\n    id_img.append(i+1)\n    label.append(np.argmax(model.predict(x_test[i:i+1])))\n    \nimg_id = np.array(id_img)\nlabel = np.array(label)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"op_df = pd.DataFrame()\nop_df['ImageId'] = img_id\nop_df['Label'] = label\nop_df.to_csv(\"submission.csv\", index=False)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Request: - \nIf you find this kernel interesting and learns something from it please don't forget to upvote. Also, write your question in comments below lets start the discussion.\n![](https://nelottery.com/media/email_alerts/2019/12_19/research/thankyou.gif)","execution_count":null}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}