{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"name":"python","version":"3.10.12","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"},"kaggle":{"accelerator":"none","dataSources":[{"sourceId":3004,"databundleVersionId":861823,"sourceType":"competition"}],"dockerImageVersionId":30918,"isInternetEnabled":true,"language":"python","sourceType":"notebook","isGpuEnabled":false}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"## Index of Content\n- Import the libraries and load the dataset\n- Preprocess the data\n- Create the model\n- Train the model\n- Evaluate on validation data\n- prediction on test data\n- submission","metadata":{}},{"cell_type":"markdown","source":"## CNN (Convolutional Neural Network)\n\nNow let’s discuss the Convolutional Neural Networks, CNN has become famous among the recent times. CNN is part of deep, feed forward artificial neural networks that can perform a variety of task with even better time and accuracy than other classifiers, in different applications of image and video recognition, recommender system and natural language processing.\n\n","metadata":{}},{"cell_type":"markdown","source":"Use of CNN have spread as Facebook uses neural nets for their automatic tagging algorithms, google for photo search Amazon for their product recommendations, Pinterest for their home feed personalization and Instagram for search infrastructure. Image classification or object recognition is a problem is passing an image as a parameter and predicting whether a condition is satisfied or not (cat or not, dot or not), or the probability or most satisfying condition for an image. We are able to quickly recognize patterns, generalize from previous information and knowledge.\n\n","metadata":{}},{"cell_type":"markdown","source":"## Import the libraries and load the dataset","metadata":{}},{"cell_type":"code","source":"# This Python 3 environment comes with many helpful analytics libraries installed\n# It is defined by the kaggle/python Docker image: https://github.com/kaggle/docker-python\n# For example, here's several helpful packages to load\n\nimport numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\nimport seaborn as sns\nimport matplotlib.pyplot as plt\n\nfrom sklearn.model_selection import train_test_split\nfrom sklearn.metrics import confusion_matrix\n\nimport tensorflow as tf\nfrom keras.layers import Dense, MaxPool2D, Flatten, Conv2D, Dropout\nfrom keras.models import Sequential\nfrom tensorflow.keras.optimizers import Adam\n\n\n# import warnings\nimport warnings\n# filter warnings\nwarnings.filterwarnings('ignore')\n\n# Input data files are available in the read-only \"../input/\" directory\n# For example, running this (by clicking run or pressing Shift+Enter) will list all files under the input directory\n\nimport os\nfor dirname, _, filenames in os.walk('/kaggle/input'):\n    for filename in filenames:\n        print(os.path.join(dirname, filename))\n\n# You can write up to 20GB to the current directory (/kaggle/working/) that gets preserved as output when you create a version using \"Save & Run All\" \n# You can also write temporary files to /kaggle/temp/, but they won't be saved outside of the current session","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true,"execution":{"iopub.status.busy":"2025-03-17T19:08:20.719831Z","iopub.execute_input":"2025-03-17T19:08:20.720192Z","iopub.status.idle":"2025-03-17T19:08:20.732764Z","shell.execute_reply.started":"2025-03-17T19:08:20.720162Z","shell.execute_reply":"2025-03-17T19:08:20.731526Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"The data files train.csv and test.csv contain gray-scale images of hand-drawn digits, from zero through nine.\n\nEach image is 28 pixels in height and 28 pixels in width, for a total of 784 pixels in total. Each pixel has a single pixel-value associated with it, indicating the lightness or darkness of that pixel, with higher numbers meaning darker. This pixel-value is an integer between 0 and 255, inclusive.\n\nThe training data set, (train.csv), has 785 columns. The first column, called \"label\", is the digit that was drawn by the user. The rest of the columns contain the pixel-values of the associated image.\n\nEach pixel column in the training set has a name like pixelx, where x is an integer between 0 and 783, inclusive. To locate this pixel on the image, suppose that we have decomposed x as x = i * 28 + j, where i and j are integers between 0 and 27, inclusive. Then pixelx is located on row i and column j of a 28 x 28 matrix, (indexing by zero).","metadata":{}},{"cell_type":"code","source":"# read train \ntrain = pd.read_csv(\"/kaggle/input/digit-recognizer/train.csv\")\nprint(train.shape)\ntrain.head()","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-03-17T19:08:20.734316Z","iopub.execute_input":"2025-03-17T19:08:20.734830Z","iopub.status.idle":"2025-03-17T19:08:22.706359Z","shell.execute_reply.started":"2025-03-17T19:08:20.734793Z","shell.execute_reply":"2025-03-17T19:08:22.705196Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# read test \nTest = pd.read_csv(\"/kaggle/input/digit-recognizer/test.csv\")\nprint(Test.shape)\nTest.head()","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-03-17T19:08:22.708284Z","iopub.execute_input":"2025-03-17T19:08:22.708619Z","iopub.status.idle":"2025-03-17T19:08:23.965832Z","shell.execute_reply.started":"2025-03-17T19:08:22.708592Z","shell.execute_reply":"2025-03-17T19:08:23.964631Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"sns.set(palette=\"Paired\")\nplt.rcParams['figure.dpi'] = 120","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-03-17T19:08:23.967402Z","iopub.execute_input":"2025-03-17T19:08:23.967679Z","iopub.status.idle":"2025-03-17T19:08:23.973057Z","shell.execute_reply.started":"2025-03-17T19:08:23.967655Z","shell.execute_reply":"2025-03-17T19:08:23.972034Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"train_y=train['label']\ntrain_x=train.drop(labels = [\"label\"],axis = 1)\ntrain_y.head()","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-03-17T19:08:23.974030Z","iopub.execute_input":"2025-03-17T19:08:23.974432Z","iopub.status.idle":"2025-03-17T19:08:24.094740Z","shell.execute_reply.started":"2025-03-17T19:08:23.974403Z","shell.execute_reply":"2025-03-17T19:08:24.093697Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"plt.figure(figsize=(20, 10))\nfor i in range(27):\n    plt.subplot(3, 9, i + 1)\n    plt.xticks([])\n    plt.yticks([])\n    plt.grid(False)\n    plt.imshow(train_x.iloc[i].to_numpy().reshape((28,28,1)))\n    plt.title(train_y[i],size = 20)\nplt.show()","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-03-17T19:08:24.095690Z","iopub.execute_input":"2025-03-17T19:08:24.096008Z","iopub.status.idle":"2025-03-17T19:08:25.566261Z","shell.execute_reply.started":"2025-03-17T19:08:24.095973Z","shell.execute_reply":"2025-03-17T19:08:25.565050Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"a = dict(train['label'].value_counts())\na = dict(sorted(a.items()))","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-03-17T19:08:25.569351Z","iopub.execute_input":"2025-03-17T19:08:25.569635Z","iopub.status.idle":"2025-03-17T19:08:25.575089Z","shell.execute_reply.started":"2025-03-17T19:08:25.569610Z","shell.execute_reply":"2025-03-17T19:08:25.573990Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"fig, (ax1, ax2) = plt.subplots(1, 2, figsize=(14, 6))\n\nbackground_color = '#faf9f4'\nax1.set_facecolor(background_color)\nax2.set_facecolor(background_color)\n\nlabels = [f\"{key} ({value})\" for key, value in a.items()]\n\nax1.pie(a.values(), wedgeprops=dict(width=0.3, edgecolor='black'), labels=labels, radius=1)\n\nsns.countplot(ax=ax2, x=train['label'])\n\nplt.show()","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-03-17T19:08:25.577131Z","iopub.execute_input":"2025-03-17T19:08:25.577465Z","iopub.status.idle":"2025-03-17T19:08:26.023025Z","shell.execute_reply.started":"2025-03-17T19:08:25.577439Z","shell.execute_reply":"2025-03-17T19:08:26.021562Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## Preprocess the data\n\nThe image data cannot be fed directly into the model so we need to perform some operations and process the data to make it ready for our neural network. The dimension of the training data is (42000,28,28). The CNN model will require one more dimension so we reshape the matrix to shape (42000,28,28,1).","metadata":{}},{"cell_type":"code","source":"train_x = train_x/255.0\ntest = Test/255.0\nprint(\"X_train Shape  : \",train_x.shape)\nprint(\"Test Shape     : \",train_y.shape)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-03-17T19:08:26.024234Z","iopub.execute_input":"2025-03-17T19:08:26.024652Z","iopub.status.idle":"2025-03-17T19:08:26.132920Z","shell.execute_reply.started":"2025-03-17T19:08:26.024623Z","shell.execute_reply":"2025-03-17T19:08:26.131710Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"train_x = train_x.values.reshape(-1,28,28,1)\ntest = test.values.reshape(-1,28,28,1)\nprint(\"X_train Shape  : \",train_x.shape)\nprint(\"Test Shape     : \",train_y.shape)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-03-17T19:08:26.133843Z","iopub.execute_input":"2025-03-17T19:08:26.134154Z","iopub.status.idle":"2025-03-17T19:08:26.140161Z","shell.execute_reply.started":"2025-03-17T19:08:26.134130Z","shell.execute_reply":"2025-03-17T19:08:26.139165Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"x_train,x_val,y_train,y_val = train_test_split(train_x,train_y,test_size = 0.2,random_state = 12345)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-03-17T19:08:26.141156Z","iopub.execute_input":"2025-03-17T19:08:26.141464Z","iopub.status.idle":"2025-03-17T19:08:26.586580Z","shell.execute_reply.started":"2025-03-17T19:08:26.141440Z","shell.execute_reply":"2025-03-17T19:08:26.585384Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"x_train.shape,x_val.shape,y_train.shape,y_val.shape","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-03-17T19:08:26.587751Z","iopub.execute_input":"2025-03-17T19:08:26.588069Z","iopub.status.idle":"2025-03-17T19:08:26.594786Z","shell.execute_reply.started":"2025-03-17T19:08:26.588042Z","shell.execute_reply":"2025-03-17T19:08:26.593433Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## Create the model\n\nNow we will create our CNN model. A CNN model generally consists of convolutional and pooling layers. It works better for data that are represented as grid structures, this is the reason why CNN works well for image classification problems. The dropout layer is used to deactivate some of the neurons and while training, it reduces offer fitting of the model. We will then compile the model with the Adam optimizer.\n","metadata":{}},{"cell_type":"markdown","source":"## Layers of Convolutional neural network\n\nThe multiple occurring of these layers shows how deep our network is, and this formation is known as the deep neural network.","metadata":{}},{"cell_type":"code","source":"model = Sequential()\nmodel.add(Conv2D(filters = 8, kernel_size = (3,3),padding = 'Same', activation ='relu', input_shape = (28,28,1)))\nmodel.add(MaxPool2D(pool_size=(2,2)))\nmodel.add(Dropout(0.25))\nmodel.add(Conv2D(filters = 16, kernel_size = (3,3),padding = 'Same', activation ='relu'))\nmodel.add(MaxPool2D(pool_size=(2,2), strides=(2,2)))\nmodel.add(Dropout(0.25))\nmodel.add(Flatten())\nmodel.add(Dense(256, activation = \"relu\"))\nmodel.add(Dense(256, activation = \"relu\"))\nmodel.add(Dropout(0.5))\nmodel.add(Dense(10, activation = \"softmax\"))","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-03-17T19:08:26.596107Z","iopub.execute_input":"2025-03-17T19:08:26.596467Z","iopub.status.idle":"2025-03-17T19:08:26.713956Z","shell.execute_reply.started":"2025-03-17T19:08:26.596439Z","shell.execute_reply":"2025-03-17T19:08:26.712867Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"model.compile(loss ='sparse_categorical_crossentropy', optimizer=Adam(learning_rate=0.001, beta_1=0.9, beta_2=0.999),metrics =['accuracy'])","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-03-17T19:08:26.715018Z","iopub.execute_input":"2025-03-17T19:08:26.715363Z","iopub.status.idle":"2025-03-17T19:08:26.724994Z","shell.execute_reply.started":"2025-03-17T19:08:26.715327Z","shell.execute_reply":"2025-03-17T19:08:26.723951Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## Train the model\n\nThe model.fit() function of Keras will start the training of the model. It takes the training data, validation data, epochs, and batch size.","metadata":{}},{"cell_type":"code","source":"history = model.fit(\n    x_train,\n    y_train,\n    batch_size=1000,\n    epochs=30,\n    verbose=1,\n    validation_data=(x_val,y_val),\n)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-03-17T19:08:26.726148Z","iopub.execute_input":"2025-03-17T19:08:26.726581Z","iopub.status.idle":"2025-03-17T19:12:24.871458Z","shell.execute_reply.started":"2025-03-17T19:08:26.726544Z","shell.execute_reply":"2025-03-17T19:12:24.870181Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"print('Loss     : {} \\nAccuracy : {}'.format(history.history['loss'][-1],history.history['accuracy'][-1]))","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-03-17T19:12:24.873183Z","iopub.execute_input":"2025-03-17T19:12:24.873556Z","iopub.status.idle":"2025-03-17T19:12:24.879187Z","shell.execute_reply.started":"2025-03-17T19:12:24.873527Z","shell.execute_reply":"2025-03-17T19:12:24.877701Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"fig, (ax1, ax2) = plt.subplots(1, 2, figsize =(14,6))\n\nax1.plot(history.history['loss'], color='r', label=\"train_Loss\")\nax1.plot(history.history['val_loss'], color='b', label=\"val_Loss\")\nax1.legend()\nax1.set_title(\"Test Loss\")\nax1.set_ylabel(\"Loss\")\nax1.set_xlabel(\"Epochs\")\n\nax2.plot(history.history['accuracy'], color='r', label=\"train_Accuracy\")\nax2.plot(history.history['val_accuracy'], color='b', label=\"val_Accuracy\")\nax2.legend()\nax2.set_title(\"Test Accuracy\")\nax2.set_ylabel(\"Accuracy\")\nax2.set_xlabel(\"Epochs\")\nfig.tight_layout()","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-03-17T19:12:24.880423Z","iopub.execute_input":"2025-03-17T19:12:24.880753Z","iopub.status.idle":"2025-03-17T19:12:26.087174Z","shell.execute_reply.started":"2025-03-17T19:12:24.880727Z","shell.execute_reply":"2025-03-17T19:12:26.085866Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## Evaluate on validation data","metadata":{}},{"cell_type":"code","source":"Y_pred = model.predict(x_val)\nY_pred_classes = np.argmax(Y_pred,axis = 1) \nY_true = np.array(y_val)\nrows = 4\ncols = 6\nf = plt.figure(figsize=(2*cols,2*rows))\nf.suptitle(\"Validation Predictions\", fontsize=20)\nfor i in range(rows*cols): \n    f.add_subplot(rows,cols,i+1)\n    img = x_val[i]\n    img = img.reshape((28,28))\n    plt.imshow(img,\n               cmap='gray')\n    plt.axis(\"off\")\n    plt.title(\"Prediction: {}\\nTrue Value: {}\".format(Y_pred_classes[i], Y_true[i]),\n              y=-0.35,color=\"red\")\nf.tight_layout()\n    \nf.show()","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-03-17T19:12:26.088624Z","iopub.execute_input":"2025-03-17T19:12:26.089017Z","iopub.status.idle":"2025-03-17T19:12:28.679885Z","shell.execute_reply.started":"2025-03-17T19:12:26.088979Z","shell.execute_reply":"2025-03-17T19:12:28.678739Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"confusion_mtx = confusion_matrix(Y_true, Y_pred_classes) \n\nf, ax = plt.subplots(figsize=(6,6))\nsns.heatmap(confusion_mtx, annot=True,\n            linewidths=3, cmap=\"Greens\",  \n            fmt='.0f', ax=ax,\n            cbar=False,\n            annot_kws={\"size\": 16})\n\nplt.yticks(rotation=0)\nplt.xlabel(\"Predicted Label\")\nplt.ylabel(\"True Label\")\nplt.title(\"Confusion Matrix\", size=14)\nplt.show()\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-03-17T19:12:28.683550Z","iopub.execute_input":"2025-03-17T19:12:28.683868Z","iopub.status.idle":"2025-03-17T19:12:29.163587Z","shell.execute_reply.started":"2025-03-17T19:12:28.683842Z","shell.execute_reply":"2025-03-17T19:12:29.162417Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## Prediction on Test data\n\nWe have 28,000 images in our dataset which will be used to evaluate how good our model works. The testing data was not involved in the training of the data therefore, it is new data for our model. The MNIST dataset is well balanced so we can get around 98-99% accuracy.","metadata":{}},{"cell_type":"code","source":"Y_pred = model.predict(test)\nY_pred_classes = np.argmax(Y_pred,axis = 1) \nrows = 4\ncols = 6\nf = plt.figure(figsize=(2*cols,2*rows))\nf.suptitle(\"Predictions\", fontsize=20)\nfor i in range(rows*cols): \n    f.add_subplot(rows,cols,i+1)\n    img = test[i]\n    img = img.reshape((28,28))\n    plt.imshow(img,\n               cmap='gray')\n    plt.axis(\"off\")\n    plt.title(\"Prediction: {}\".format(Y_pred_classes[i]),\n              y=-0.35,color=\"red\")\nf.tight_layout()\n    \nf.show()","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-03-17T19:12:29.164806Z","iopub.execute_input":"2025-03-17T19:12:29.165122Z","iopub.status.idle":"2025-03-17T19:12:35.033476Z","shell.execute_reply.started":"2025-03-17T19:12:29.165096Z","shell.execute_reply":"2025-03-17T19:12:35.032343Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## Submission","metadata":{}},{"cell_type":"code","source":"submission = pd.DataFrame({'ImageId': range(1,28001), 'Label': Y_pred_classes})\nsubmission.to_csv('my_submission.csv', index=False)\nprint(\"Your submission was successfully saved!\")","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-03-17T19:12:35.034749Z","iopub.execute_input":"2025-03-17T19:12:35.035160Z","iopub.status.idle":"2025-03-17T19:12:35.062413Z","shell.execute_reply.started":"2025-03-17T19:12:35.035122Z","shell.execute_reply":"2025-03-17T19:12:35.061299Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"submission","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-03-17T19:12:35.063639Z","iopub.execute_input":"2025-03-17T19:12:35.064013Z","iopub.status.idle":"2025-03-17T19:12:35.073915Z","shell.execute_reply.started":"2025-03-17T19:12:35.063983Z","shell.execute_reply":"2025-03-17T19:12:35.072808Z"}},"outputs":[],"execution_count":null}]}