{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"name":"python","version":"3.10.12","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"},"kaggle":{"accelerator":"none","dataSources":[{"sourceId":3004,"databundleVersionId":861823,"sourceType":"competition"}],"dockerImageVersionId":30822,"isInternetEnabled":true,"language":"python","sourceType":"notebook","isGpuEnabled":false}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"## **Introduction**\nThis notebook provides a step-by-step guide for building and evaluating a Convolutional Neural Network (CNN) for digit recognition using the MNIST dataset. We will preprocess the data, define and train the CNN model, evaluate its performance, and generate predictions for submission to a Kaggle competition.","metadata":{}},{"cell_type":"markdown","source":"## **Step 1: Import Libaries**","metadata":{}},{"cell_type":"code","source":"# Importing TensorFlow library\nimport tensorflow as tf\n\n#Importing Pandas library for data manipulation \nimport pandas as pd \n\n# Importing the layers and models modules from TensorFlow's Keras API for building neural network architectures\nfrom tensorflow.keras import layers, models\n\n# Importing the Input layer from TensorFlow's Keras API to define model input shapes\nfrom tensorflow.keras.layers import Input\n\n# Importing the train_test_split function from scikit-learn for splitting datasets into training and testing subsets\nfrom sklearn.model_selection import train_test_split\n\n\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2024-12-24T15:36:52.538139Z","iopub.execute_input":"2024-12-24T15:36:52.538501Z","iopub.status.idle":"2024-12-24T15:37:04.430186Z","shell.execute_reply.started":"2024-12-24T15:36:52.538472Z","shell.execute_reply":"2024-12-24T15:37:04.428914Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## **Step 2: Load and Process MNIST data**\n- Load the dataset and separated images from target labels.\n- Split data into training and development sets **(10% of training data used for development)**.\n- Normalize pixel values from 0–255 to 0–1.\n- Reshape images for CNN compatibility (grayscale images).\n\n\n\n\n\n","metadata":{}},{"cell_type":"markdown","source":"### **Load the Dataset**","metadata":{}},{"cell_type":"code","source":"test_set = pd.read_csv(\"/kaggle/input/digit-recognizer/test.csv\") #Modify according to your file path\ntrain_set = pd.read_csv(\"/kaggle/input/digit-recognizer/train.csv\") #Modify according to your file path\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2024-12-24T15:43:59.831070Z","iopub.execute_input":"2024-12-24T15:43:59.831455Z","iopub.status.idle":"2024-12-24T15:44:05.080197Z","shell.execute_reply.started":"2024-12-24T15:43:59.831426Z","shell.execute_reply":"2024-12-24T15:44:05.079077Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"### **Separate Features and Labels**","metadata":{}},{"cell_type":"code","source":"#Extract labels from the training dataset \ny_train_full = train_set['label']\n\n#Remove the label column from the training dataset to get features \nX_train_full = train_set.drop('label', axis=1)\n\n#Use the test dataset as it is (no labels provided)\nX_test = test_set","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2024-12-24T15:44:55.305816Z","iopub.execute_input":"2024-12-24T15:44:55.306162Z","iopub.status.idle":"2024-12-24T15:44:55.435920Z","shell.execute_reply.started":"2024-12-24T15:44:55.306135Z","shell.execute_reply":"2024-12-24T15:44:55.434893Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"### **Split Data for Training and Development**","metadata":{}},{"cell_type":"code","source":"# Split the full training dataset into a smaller training set and a development (validation) set\n# - `X_train_full`: Full training set features\n# - `y_train_full`: Full training set labels\n# - `test_size=0.1`: Specifies that 10% of the data will be allocated to the development set\n# - `random_state=42`: Ensures reproducibility of the split by fixing the random seed\n# - `stratify=y_train_full`: Ensures the split preserves the class distribution of the labels\nX_train,X_dev,y_train,y_dev = train_test_split(X_train_full,y_train_full, test_size=0.1,random_state=42, stratify=y_train_full)","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"### **Normalize Pixel Values**","metadata":{}},{"cell_type":"code","source":"# Normalize pixel values from the range [0, 255] to [0, 1]\n# - This scaling helps improve the performance and stability of neural networks during training\n\nX_train = X_train / 255.0\nX_test = X_test / 255.0\nX_dev = X_dev / 255.0","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2024-12-24T15:46:39.210930Z","iopub.execute_input":"2024-12-24T15:46:39.211306Z","iopub.status.idle":"2024-12-24T15:46:39.236265Z","shell.execute_reply.started":"2024-12-24T15:46:39.211279Z","shell.execute_reply":"2024-12-24T15:46:39.234604Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"### **Reshape for CNN Compatibility** ","metadata":{}},{"cell_type":"code","source":"# Add a channel dimension to the datasets\n# - This is required for compatibility with convolutional neural networks (CNNs)\n# - Original shape: (28, 28) (grayscale image with no explicit channel dimension)\n# - New shape: (28, 28, 1) (grayscale image with a single channel dimension)\n# - `tf.newaxis` adds a new axis to the data array at the specified position\nX_train = X_train.values.reshape(-1, 28, 28, 1)\nX_test = X_test.values.reshape(-1, 28, 28, 1)\nX_dev = X_dev.values.reshape(-1, 28, 28, 1)","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## **Step 3: Define the CNN Model**\n\n- Define a sequential model model_mnist using `tf.keras.models.Sequential`.\n    - **Input:** 28x28 grayscale images (1 channel).\n    - **Conv Layer 1:** 32 filters, (3x3) kernel, `ReLU` activation.\n    - **Max-Pool 1:** (2x2) pool size.\n    -  **Conv Layer 2:** 64 filters, (3x3) kernel, `ReLU` activation.\n    -  **Max-Pool 2:** (2x2) pool size.\n    -  **Flatten:** Converts 2D features to 1D vector.\n    -  **Dense Layer:** 128 neurons, `ReLU` activation.\n    -  **Output:** 10 neurons, `Softmax` activation.","metadata":{}},{"cell_type":"code","source":"model_mnist = models.Sequential([\n    # Input layer specifying the shape of the input data (28x28 grayscale images with 1 channel)\n    Input(shape=(28,28,1)),\n\n    # First convolutional layer\n    # - 32 filters with a kernel size of (3, 3)\n    # - Activation function: ReLU (introduces non-linearity)\n    layers.Conv2D(32,(3,3),activation='relu'),\n\n    # First max-pooling layer\n    # - Pool size: (2, 2)\n    # - Reduces the spatial dimensions by taking the maximum value in each (2x2) window\n    layers.MaxPooling2D((2,2)),\n\n    # Second convolutional layer\n    # - 64 filters with a kernel size of (3, 3)\n    # - Activation function: ReLU\n    layers.Conv2D(64,(3,3),activation='relu'),\n\n    # Second max-pooling layer\n    # - Pool size: (2, 2)\n    layers.MaxPooling2D(2,2),\n\n    # Flatten layer\n    # - Converts the 2D feature maps into a 1D feature vector to prepare for the dense layers\n    layers.Flatten(),\n\n    # Fully connected (dense) layer\n    # - 128 neurons\n    # - Activation function: ReLU\n    layers.Dense(128, activation='relu'),\n\n    # Output layer\n    # - 10 neurons (corresponding to the 10 possible classes in MNIST)\n    # - Activation function: Softmax (produces probabilities for each class)\n    layers.Dense(10,activation='softmax')\n  ])","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## **Step 4: Compile & Train the CNN Model**\n- Compile the model using the **Adam optimizer**.\n- Use `Sparse Categorical Cross-Entropy` loss function to efficiently compute the loss for this multi-class classification problem.\n- Train the model with`batch_size = 32` to balance computational efficiency with gradient stability and `epochs=30` to ensure sufficient training to minimize loss without overfitting.\n","metadata":{}},{"cell_type":"markdown","source":"### **Compile the Model**","metadata":{}},{"cell_type":"code","source":"# - `optimizer='adam'`: Adam optimizer is used for training, which combines the advantages of RMSProp and momentum,\n#   and adapts the learning rate during training for faster convergence.\n# - `loss='sparse_categorical_crossentropy'`: This loss function is suitable for multi-class classification problems\n#   where the labels are integers (not one-hot encoded).\n# - `metrics=['accuracy']`: The accuracy metric is used to evaluate the model's performance during training and testing.\nmodel_mnist.compile(optimizer='adam',\n              loss='sparse_categorical_crossentropy',metrics=['accuracy'])","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"### **Train the Model**","metadata":{}},{"cell_type":"code","source":"#Train the CNN Model\nhistory = model_mnist.fit(X_train,y_train, epochs=30, batch_size=32, validation_data=(X_dev,y_dev))","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"","metadata":{}},{"cell_type":"markdown","source":"## **Step 5: Evaluate the Model and Make Predictions**\n\n- Evaluate the model on the development set to assess performance.\n- Generate predictions on the test set by:\n    - Predicting **class probabilities** using the trained model.\n    - Converting probabilities to **class labels** (using `argmax`).\n    - Saving the predictions to a **CSV file** for further analysis.\n","metadata":{}},{"cell_type":"markdown","source":"### **Evaluate the Model on the Development Set**","metadata":{}},{"cell_type":"code","source":"# Evaluate the model on the development/validation set\n# - `X_dev`: development set features\n# - `y_dev`: development set labels\n# - Returns a list containing the loss and the specified metrics (e.g., accuracy)\nmodel_results = model_mnist.evaluate(X_dev,y_dev)\n\n# Print the results with descriptive labels\nprint(f\"Development Set Loss: {model_results[0]:.4f}\")  # Print the loss value from the development set evaluation\nprint(f\"Development Set Accuracy: {model_results[1]:.4f}\")  # Print the accuracy value from the development set evaluation\n","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"### **Generate Predictions from the Test Set**","metadata":{}},{"cell_type":"code","source":"#Predict class probabilities for the test set \npredictions = model_mnist.predict(X_test)\n\n#Convert probabilities to class labels (0-9)\npredicted_labels = predictions.argmax(axis=1)\n","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"### **Save Predictions to A CSV File** ","metadata":{}},{"cell_type":"code","source":"#Create a Dataframe with ImageId and predicted labels \nresults_df = pd.DataFrame({\n    'ImageId': test_set.index + 1,\n    'Label': predicted_labels \n\n})\n\n# Save results to a CSV File  \nresults_df.to_csv('results_mnist.csv',index=False)","metadata":{"trusted":true},"outputs":[],"execution_count":null}]}