{"metadata":{"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"name":"python","version":"3.12.13","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# Digit Recognizer - CNN with Early Stopping\n\n**Dataset:** [Kaggle Digit Recognizer](https://www.kaggle.com/c/digit-recognizer)  \n**Task:** Classify handwritten digits (0-9) from 28x28 grayscale images.  \n**Method:** Convolutional Neural Network (CNN) with early stopping to prevent overfitting.","metadata":{}},{"cell_type":"markdown","source":"## 1. Setup & Data Loading","metadata":{}},{"cell_type":"code","source":"import numpy as np\nimport pandas as pd\nimport matplotlib.pyplot as plt\nfrom sklearn.model_selection import train_test_split\nfrom tensorflow import keras\nfrom tensorflow.keras import layers\nfrom tensorflow.keras.callbacks import EarlyStopping\n\ntrain_df = pd.read_csv('/kaggle/input/competitions/digit-recognizer/train.csv')\ntest_df = pd.read_csv('/kaggle/input/competitions/digit-recognizer/test.csv')\n\nprint(f\"Train shape: {train_df.shape}\")\nprint(f\"Test shape: {test_df.shape}\")","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-07-28T12:11:11.108102Z","iopub.execute_input":"2026-07-28T12:11:11.108531Z","iopub.status.idle":"2026-07-28T12:11:14.692070Z","shell.execute_reply.started":"2026-07-28T12:11:11.108482Z","shell.execute_reply":"2026-07-28T12:11:14.691170Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## 2. Preprocessing\n\n- Separate labels from pixel data\n- Normalize pixel values to [0, 1]\n- Reshape to 28x28x1 for CNN input","metadata":{}},{"cell_type":"code","source":"y = train_df['label'].values\nX = train_df.drop('label', axis=1).values / 255.0\nX_test = test_df.values / 255.0\n\nX = X.reshape(-1, 28, 28, 1)\nX_test = X_test.reshape(-1, 28, 28, 1)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-07-28T12:11:14.693593Z","iopub.execute_input":"2026-07-28T12:11:14.694240Z","iopub.status.idle":"2026-07-28T12:11:14.995746Z","shell.execute_reply.started":"2026-07-28T12:11:14.694193Z","shell.execute_reply":"2026-07-28T12:11:14.994470Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## 3. Train / Validation Split\n\nWe hold out 20% of the training data for validation. This lets us monitor generalization during training and decide when to stop.","metadata":{}},{"cell_type":"code","source":"X_train, X_val, y_train, y_val = train_test_split(\n    X, y, test_size=0.2, random_state=42, stratify=y\n)\n\nprint(f\"Training samples: {X_train.shape[0]}\")\nprint(f\"Validation samples: {X_val.shape[0]}\")","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-07-28T12:11:14.996764Z","iopub.execute_input":"2026-07-28T12:11:14.997026Z","iopub.status.idle":"2026-07-28T12:11:15.461856Z","shell.execute_reply.started":"2026-07-28T12:11:14.997000Z","shell.execute_reply":"2026-07-28T12:11:15.460825Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## 4. Baseline: Fully Connected Neural Network\n\nBefore CNN, we try a simple dense network. It treats the image as a flat vector of 784 pixels — no spatial awareness.","metadata":{}},{"cell_type":"code","source":"fc_model = keras.Sequential([\n    layers.Flatten(input_shape=(28, 28, 1)),\n    layers.Dense(256, activation=\"relu\"),\n    layers.Dropout(0.3),\n    layers.Dense(128, activation=\"relu\"),\n    layers.Dropout(0.3),\n    layers.Dense(10, activation=\"softmax\")\n])\n\nfc_model.compile(\n    optimizer=\"adam\",\n    loss=\"sparse_categorical_crossentropy\",\n    metrics=[\"accuracy\"]\n)\n\nfc_model.summary()","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-07-28T12:11:15.464804Z","iopub.execute_input":"2026-07-28T12:11:15.465101Z","iopub.status.idle":"2026-07-28T12:11:15.585664Z","shell.execute_reply.started":"2026-07-28T12:11:15.465074Z","shell.execute_reply":"2026-07-28T12:11:15.584912Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"early_stop_fc = EarlyStopping(\n    monitor=\"val_loss\",\n    patience=5,\n    restore_best_weights=True,\n    verbose=1\n)\n\nfc_history = fc_model.fit(\n    X_train, y_train,\n    epochs=50,\n    batch_size=128,\n    validation_data=(X_val, y_val),\n    callbacks=[early_stop_fc]\n)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-07-28T12:11:15.586753Z","iopub.execute_input":"2026-07-28T12:11:15.587040Z","iopub.status.idle":"2026-07-28T12:11:45.939895Z","shell.execute_reply.started":"2026-07-28T12:11:15.587013Z","shell.execute_reply":"2026-07-28T12:11:45.938934Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"fc_val_loss, fc_val_acc = fc_model.evaluate(X_val, y_val, verbose=0)\nprint(f\"Fully Connected NN - Validation Accuracy: {fc_val_acc:.4f}\")","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-07-28T12:11:45.941119Z","iopub.execute_input":"2026-07-28T12:11:45.941906Z","iopub.status.idle":"2026-07-28T12:11:46.535529Z","shell.execute_reply.started":"2026-07-28T12:11:45.941870Z","shell.execute_reply":"2026-07-28T12:11:46.534563Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## 5. CNN Model\n\nNow we use a CNN which understands spatial structure. Unlike the dense network, CNN sees the image as a 2D grid and detects patterns like edges, curves, and shapes.","metadata":{}},{"cell_type":"markdown","source":"## 4. Build CNN Model\n\nArchitecture:\n- 2 Convolutional blocks (Conv2D → ReLU → MaxPool → Dropout)\n- Dense head with dropout for regularization\n- Softmax output for 10 classes","metadata":{}},{"cell_type":"code","source":"model = keras.Sequential([\n    layers.Conv2D(32, 3, activation='relu', input_shape=(28, 28, 1)),\n    layers.MaxPooling2D(2),\n    layers.Dropout(0.25),\n\n    layers.Conv2D(64, 3, activation='relu'),\n    layers.MaxPooling2D(2),\n    layers.Dropout(0.25),\n\n    layers.Flatten(),\n    layers.Dense(128, activation='relu'),\n    layers.Dropout(0.5),\n    layers.Dense(10, activation='softmax')\n])\n\nmodel.compile(\n    optimizer='adam',\n    loss='sparse_categorical_crossentropy',\n    metrics=['accuracy']\n)\n\nmodel.summary()","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-07-28T12:11:46.536771Z","iopub.execute_input":"2026-07-28T12:11:46.537637Z","iopub.status.idle":"2026-07-28T12:11:46.618277Z","shell.execute_reply.started":"2026-07-28T12:11:46.537607Z","shell.execute_reply":"2026-07-28T12:11:46.617442Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## 5. Training with Early Stopping\n\n**Early Stopping** monitors `val_loss`. If it doesn't improve for `patience` epochs, training stops and the best weights are restored. This prevents overfitting — the model stops before it memorizes the training data.","metadata":{}},{"cell_type":"code","source":"early_stop = EarlyStopping(\n    monitor='val_loss',\n    patience=5,\n    restore_best_weights=True,\n    verbose=1\n)\n\nhistory = model.fit(\n    X_train, y_train,\n    epochs=50,\n    batch_size=128,\n    validation_data=(X_val, y_val),\n    callbacks=[early_stop]\n)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-07-28T12:11:46.619341Z","iopub.execute_input":"2026-07-28T12:11:46.620011Z","iopub.status.idle":"2026-07-28T12:15:37.847893Z","shell.execute_reply.started":"2026-07-28T12:11:46.619983Z","shell.execute_reply":"2026-07-28T12:15:37.846994Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## 6. Training Graphs\n\nPlotting loss and accuracy for both training and validation sets helps us visually confirm that early stopping triggered at the right point.","metadata":{}},{"cell_type":"code","source":"fig, (ax1, ax2) = plt.subplots(1, 2, figsize=(12, 4))\n\nax1.plot(history.history['loss'], label='Train Loss')\nax1.plot(history.history['val_loss'], label='Val Loss')\nax1.set_title('Loss')\nax1.set_xlabel('Epoch')\nax1.legend()\n\nax2.plot(history.history['accuracy'], label='Train Acc')\nax2.plot(history.history['val_accuracy'], label='Val Acc')\nax2.set_title('Accuracy')\nax2.set_xlabel('Epoch')\nax2.legend()\n\nplt.tight_layout()\nplt.show()","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-07-28T12:15:37.849221Z","iopub.execute_input":"2026-07-28T12:15:37.849640Z","iopub.status.idle":"2026-07-28T12:15:38.275972Z","shell.execute_reply.started":"2026-07-28T12:15:37.849602Z","shell.execute_reply":"2026-07-28T12:15:38.274983Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## 7. Final Test & Submission\n\nGenerate predictions on the test set and create the submission file.","metadata":{}},{"cell_type":"code","source":"val_loss, val_acc = model.evaluate(X_val, y_val, verbose=0)\nprint(f\"Validation Accuracy: {val_acc:.4f}\")\n\npredictions = model.predict(X_test)\npredicted_labels = np.argmax(predictions, axis=1)\n\nsubmission = pd.DataFrame({\n    'ImageId': range(1, len(predicted_labels) + 1),\n    'Label': predicted_labels\n})\nsubmission.to_csv('submission.csv', index=False)\nprint(f\"Submission saved: {submission.shape[0]} predictions\")\nsubmission.head()","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-07-28T12:15:38.278097Z","iopub.execute_input":"2026-07-28T12:15:38.278418Z","iopub.status.idle":"2026-07-28T12:15:44.493150Z","shell.execute_reply.started":"2026-07-28T12:15:38.278380Z","shell.execute_reply":"2026-07-28T12:15:44.492445Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## Comparison: Fully Connected vs CNN","metadata":{}},{"cell_type":"code","source":"fig, (ax1, ax2) = plt.subplots(1, 2, figsize=(12, 4))\n\n# Loss comparison\nax1.plot(fc_history.history[\"val_loss\"], label=\"FC - Val Loss\", linestyle=\"--\")\nax1.plot(history.history[\"val_loss\"], label=\"CNN - Val Loss\")\nax1.set_title(\"Validation Loss Comparison\")\nax1.set_xlabel(\"Epoch\")\nax1.legend()\n\n# Accuracy comparison\nax2.plot(fc_history.history[\"val_accuracy\"], label=\"FC - Val Acc\", linestyle=\"--\")\nax2.plot(history.history[\"val_accuracy\"], label=\"CNN - Val Acc\")\nax2.set_title(\"Validation Accuracy Comparison\")\nax2.set_xlabel(\"Epoch\")\nax2.legend()\n\nplt.tight_layout()\nplt.show()","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-07-28T12:15:44.494122Z","iopub.execute_input":"2026-07-28T12:15:44.494431Z","iopub.status.idle":"2026-07-28T12:15:44.866087Z","shell.execute_reply.started":"2026-07-28T12:15:44.494397Z","shell.execute_reply":"2026-07-28T12:15:44.865233Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"print(\"=\" * 40)\nprint(\"MODEL COMPARISON\")\nprint(\"=\" * 40)\nprint(f\"Fully Connected NN: {fc_val_acc:.4f} accuracy\")\nprint(f\"CNN:                {val_acc:.4f} accuracy\")\nprint(f\"Improvement:        +{(val_acc - fc_val_acc)*100:.2f}%\")\nprint(\"=\" * 40)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-07-28T12:15:44.867136Z","iopub.execute_input":"2026-07-28T12:15:44.867549Z","iopub.status.idle":"2026-07-28T12:15:44.873673Z","shell.execute_reply.started":"2026-07-28T12:15:44.867520Z","shell.execute_reply":"2026-07-28T12:15:44.872754Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## 8. Sample Predictions","metadata":{}},{"cell_type":"code","source":"fig, axes = plt.subplots(2, 5, figsize=(12, 5))\nindices = np.random.choice(len(X_test), 10, replace=False)\n\nfor i, ax in enumerate(axes.flat):\n    ax.imshow(X_test[indices[i]].reshape(28, 28), cmap=\"gray\")\n    ax.set_title(f\"Pred: {predicted_labels[indices[i]]}\")\n    ax.axis(\"off\")\n\nplt.tight_layout()\nplt.show()","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-07-28T12:15:44.874845Z","iopub.execute_input":"2026-07-28T12:15:44.875165Z","iopub.status.idle":"2026-07-28T12:15:45.629505Z","shell.execute_reply.started":"2026-07-28T12:15:44.875141Z","shell.execute_reply":"2026-07-28T12:15:45.628587Z"}},"outputs":[],"execution_count":null}]}