{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"name":"python","version":"3.12.12","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"},"kaggle":{"accelerator":"nvidiaTeslaT4","dataSources":[{"sourceType":"competition","sourceId":3649,"databundleVersionId":46718}],"isInternetEnabled":true,"language":"python","sourceType":"notebook","isGpuEnabled":true}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"<h1 style=\"color:#1f77b4;\">Building and Training a CNN for Image Classification Using the CIFAR-10 Dataset&nbsp;&nbsp;</h1>\n\nThis notebook presents an end-to-end **deep learning image classification project** using the **CIFAR-10 object recognition dataset** and a custom **PyTorch Convolutional Neural Network (CNN)**. The central purpose of the project is to build a model that can learn visual patterns directly from raw image pixels and assign each image to one of ten mutually exclusive object categories. The workflow moves through the full practical machine-learning pipeline: confirming the local Kaggle file structure, reading metadata, mapping class labels, creating a stratified training and validation split, extracting image files from compressed archives, preprocessing image tensors, training a CNN, evaluating validation performance, visualizing prediction behavior, and producing a Kaggle-style `submission.csv` file.\n\nThe project is designed as both a modeling exercise and a reproducible workflow. CIFAR-10 images are intentionally small, only **32 × 32 pixels with three RGB color channels**, so the task requires the model to learn compact features such as edges, color regions, textures, wheels, wings, animal faces, legs, and background context. A CNN is appropriate for this problem because convolutional filters can scan local neighborhoods of pixels and learn spatially organized features without manually engineering image descriptors. As the network becomes deeper, early layers tend to learn low-level patterns such as corners and color contrasts, while later layers combine those patterns into more class-specific visual structures.\n\n### Dataset description\n\n**CIFAR-10** is an established computer-vision benchmark for object recognition in small natural images. The original CIFAR-10 dataset was collected by **Alex Krizhevsky, Vinod Nair, and Geoffrey Hinton** as a labeled subset of the larger 80 Million Tiny Images dataset. It contains **60,000 color images** distributed evenly across **10 object classes**, with **6,000 images per class**. The traditional benchmark split contains **50,000 training images** and **10,000 test images**. Each image is a low-resolution RGB image with shape **32 × 32 × 3**.\n\nThe Kaggle competition version used in this notebook provides the data in archive and metadata form. The important files are:\n\n- `train.7z`: compressed folder containing the labeled training images in PNG format.\n- `test.7z`: compressed folder containing the competition test images in PNG format.\n- `trainLabels.csv`: training metadata with image `id` values and string labels.\n- `sampleSubmission.csv`: the required submission structure, with one row per test image and columns `id,label`.\n\nKaggle preserves the labeled training set and asks participants to predict labels for the test image IDs. In the Kaggle competition setting, the test archive contains **300,000 images** because Kaggle added **290,000 junk images** that are ignored during scoring. Participants still submit predictions for all test IDs, while the leaderboard evaluates the official scoring subset using **classification accuracy**, or the percentage of labels predicted correctly.\n\nThe ten official labels are:\n\n`airplane`, `automobile`, `bird`, `cat`, `deer`, `dog`, `frog`, `horse`, `ship`, and `truck`.\n\nThe classes are mutually exclusive, so this is a **single-label multi-class classification** problem rather than a multi-label problem. The distinction between `automobile` and `truck` is especially important: automobiles include passenger vehicles such as sedans and SUVs, while trucks refer to larger truck-type vehicles. Because the model is trained with one correct class per image, the final output layer produces ten class scores, and the predicted label is selected as the class with the highest score.\n\n### Project pipeline overview\n\nThis notebook follows a structured development pipeline:\n\n1. Configure libraries, reproducibility controls, hardware detection, and training parameters.\n2. Read CIFAR-10 Kaggle metadata and verify that all required files are available.\n3. Convert string labels into numeric class IDs and create stratified train/validation splits.\n4. Extract only the required image files from the `.7z` archives.\n5. Build PyTorch `Dataset` and `DataLoader` objects for training, validation, and testing.\n6. Visualize sample images to confirm that image extraction, label mapping, and transforms are working.\n7. Define a custom CNN architecture with convolution, batch normalization, pooling, dropout, and dense layers.\n8. Train the model, monitor learning curves, save the best validation checkpoint, and adjust learning rate when progress plateaus.\n9. Evaluate validation performance with accuracy, a classification report, and a confusion matrix.\n10. Generate qualitative prediction examples and create a Kaggle-style `submission.csv` file.\n\nPrimary dataset citations: Krizhevsky (2009), *Learning Multiple Layers of Features from Tiny Images*; and Cukierski (2013), *CIFAR-10 — Object Recognition in Images* on Kaggle.","metadata":{}},{"cell_type":"markdown","source":"## Step 0. Library Imports, Reproducibility, and Low-Compute Configuration","metadata":{}},{"cell_type":"code","source":"## Dependency notes\n##!pip install py7zr\n##!pip uninstall -y torch torchvision torchaudio\n'''\n!pip install -q --no-cache-dir --no-deps \\\n    torch==2.3.1+cu118 \\\n    torchvision==0.18.1+cu118 \\\n    torchaudio==2.3.1+cu118 \\\n    --extra-index-url https://download.pytorch.org/whl/cu118 \n\n'''","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-05-01T22:08:16.176450Z","iopub.execute_input":"2026-05-01T22:08:16.177168Z","iopub.status.idle":"2026-05-01T22:08:16.182813Z","shell.execute_reply.started":"2026-05-01T22:08:16.177136Z","shell.execute_reply":"2026-05-01T22:08:16.181717Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"## Verify \nimport torch, torchvision\n\nprint(\"Torch:\", torch.__version__)\nprint(\"Torchvision:\", torchvision.__version__)\nprint(\"CUDA:\", torch.version.cuda)\nprint(\"CUDA available:\", torch.cuda.is_available())\nprint(\"GPU:\", torch.cuda.get_device_name(0) if torch.cuda.is_available() else \"CPU\")","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-05-01T21:25:31.064328Z","iopub.execute_input":"2026-05-01T21:25:31.064529Z","iopub.status.idle":"2026-05-01T21:25:45.042909Z","shell.execute_reply.started":"2026-05-01T21:25:31.064506Z","shell.execute_reply":"2026-05-01T21:25:45.041978Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"__author__ = \"Will Hinton\"\n__email__ = \"willhint@gmail.com\"\n__website__ = \"whinton0.github.com/py\"","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-05-01T21:28:29.573673Z","iopub.execute_input":"2026-05-01T21:28:29.574042Z","iopub.status.idle":"2026-05-01T21:28:29.578635Z","shell.execute_reply.started":"2026-05-01T21:28:29.574012Z","shell.execute_reply":"2026-05-01T21:28:29.577756Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"import os\nimport random\nimport warnings\nimport shutil\nfrom pathlib import Path\nfrom typing import Iterable, List, Tuple\n\nimport numpy as np\nimport pandas as pd\nimport matplotlib.pyplot as plt\n\nfrom PIL import Image\n\n# import torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nfrom torch.utils.data import Dataset, DataLoader\nfrom torchvision import transforms\n\nfrom sklearn.model_selection import train_test_split\nfrom sklearn.metrics import accuracy_score, classification_report, confusion_matrix\n\nwarnings.filterwarnings(\"ignore\")\n\nSEED = 42\nrandom.seed(SEED)\nnp.random.seed(SEED)\ntorch.manual_seed(SEED)\nif torch.cuda.is_available():\n    torch.cuda.manual_seed_all(SEED)\n\ndevice = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\nprint(f\"Using device: {device}\")\n\n\n# -------------------------------------------------------------------------\n# GPU / FULL-DATA CONTROLS\n# -------------------------------------------------------------------------\nSAMPLE_FRACTION = 1.00\nTEST_SAMPLE_FRACTION = 1.00\n\nBATCH_SIZE = 128          # Safe starting point for P100\nNUM_EPOCHS = 30           # Reasonable full-training run\nLEARNING_RATE = 1e-3\n\nNUM_WORKERS = 2           # Good Kaggle-safe starting value\nPIN_MEMORY = True         # Speeds CPU -> GPU transfer\nPERSISTENT_WORKERS = True # Useful when NUM_WORKERS > 0\n\nIMAGE_SIZE = 32\nNUM_CLASSES = 10\n\nprint(\"Sample fraction:\", SAMPLE_FRACTION)\nprint(\"Test sample fraction:\", TEST_SAMPLE_FRACTION)\nprint(\"Batch size:\", BATCH_SIZE)\nprint(\"Epochs:\", NUM_EPOCHS)\nprint(\"Workers:\", NUM_WORKERS)\nprint(\"Pin memory:\", PIN_MEMORY)\nprint(\"Persistent workers:\", PERSISTENT_WORKERS)\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-05-01T21:29:12.458984Z","iopub.execute_input":"2026-05-01T21:29:12.459846Z","iopub.status.idle":"2026-05-01T21:29:14.020415Z","shell.execute_reply.started":"2026-05-01T21:29:12.459803Z","shell.execute_reply":"2026-05-01T21:29:14.019649Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## Step 1. Define Local File Paths and Read CSV Metadata","metadata":{}},{"cell_type":"code","source":"DATA_DIR = Path(\"/kaggle/input/competitions/cifar-10\")\nTRAIN_ARCHIVE = DATA_DIR / \"train.7z\"\nTEST_ARCHIVE = DATA_DIR / \"test.7z\"\nTRAIN_LABELS_CSV = DATA_DIR / \"trainLabels.csv\"\nSAMPLE_SUBMISSION_CSV = DATA_DIR / \"sampleSubmission.csv\"\n\nEXTRACT_ROOT = DATA_DIR / \"cifar10_extracted_sample\"\nTRAIN_DIR = EXTRACT_ROOT / \"train\"\nTEST_DIR = EXTRACT_ROOT / \"test\"\n\nrequired_files = [TRAIN_ARCHIVE, TEST_ARCHIVE, TRAIN_LABELS_CSV, SAMPLE_SUBMISSION_CSV]\nmissing = [str(p) for p in required_files if not p.exists()]\nif missing:\n    raise FileNotFoundError(\n        \"Missing required local files: \" + \", \".join(missing) +\n        \"\\nPlace train.7z, test.7z, trainLabels.csv, and sampleSubmission.csv in the notebook directory.\"\n    )\n\ntrain_labels_df = pd.read_csv(TRAIN_LABELS_CSV)\nsample_submission_df = pd.read_csv(SAMPLE_SUBMISSION_CSV)\n\nprint(\"trainLabels.csv shape:\", train_labels_df.shape)\nprint(\"sampleSubmission.csv shape:\", sample_submission_df.shape)\ndisplay(train_labels_df.head())\ndisplay(sample_submission_df.head())","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-05-01T21:29:27.747216Z","iopub.execute_input":"2026-05-01T21:29:27.748355Z","iopub.status.idle":"2026-05-01T21:29:27.966920Z","shell.execute_reply.started":"2026-05-01T21:29:27.748294Z","shell.execute_reply":"2026-05-01T21:29:27.966279Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## Step 2. Create Adjustable Stratified Training Sample and Test Sample\n\nFor local CPU development, the notebook samples only a small percentage of the full Kaggle dataset. The training sample is **stratified by class** to preserve class balance as much as possible.","metadata":{"execution":{"iopub.status.busy":"2026-04-30T04:04:17.135380Z","iopub.execute_input":"2026-04-30T04:04:17.136166Z","iopub.status.idle":"2026-04-30T04:04:17.143913Z","shell.execute_reply.started":"2026-04-30T04:04:17.136132Z","shell.execute_reply":"2026-04-30T04:04:17.142619Z"}}},{"cell_type":"code","source":"class_names = sorted(train_labels_df[\"label\"].unique().tolist())\nclass_to_idx = {name: idx for idx, name in enumerate(class_names)}\nidx_to_class = {idx: name for name, idx in class_to_idx.items()}\n\ntrain_labels_df[\"label_idx\"] = train_labels_df[\"label\"].map(class_to_idx)\n\n# -------------------------------------------------------------------------\n# Full or sampled training set\n# -------------------------------------------------------------------------\nif SAMPLE_FRACTION >= 1.00:\n    train_sample_df = train_labels_df.copy()\nelse:\n    n_train_sample = max(NUM_CLASSES, int(len(train_labels_df) * SAMPLE_FRACTION))\n\n    train_sample_df, _ = train_test_split(\n        train_labels_df,\n        train_size=n_train_sample,\n        stratify=train_labels_df[\"label\"],\n        random_state=SEED,\n    )\n\ntrain_sample_df = train_sample_df.sort_values(\"id\").reset_index(drop=True)\n\n# -------------------------------------------------------------------------\n# Train / validation split\n# -------------------------------------------------------------------------\nVALIDATION_FRACTION = 0.20\n\ntrain_df, val_df = train_test_split(\n    train_sample_df,\n    test_size=VALIDATION_FRACTION,\n    stratify=train_sample_df[\"label\"],\n    random_state=SEED,\n)\n\ntrain_df = train_df.sort_values(\"id\").reset_index(drop=True)\nval_df = val_df.sort_values(\"id\").reset_index(drop=True)\n\n# -------------------------------------------------------------------------\n# Full or sampled Kaggle test set\n# -------------------------------------------------------------------------\nif TEST_SAMPLE_FRACTION >= 1.00:\n    test_sample_df = sample_submission_df.copy()\nelse:\n    n_test_sample = max(1, int(len(sample_submission_df) * TEST_SAMPLE_FRACTION))\n    test_sample_df = sample_submission_df.sample(\n        n=n_test_sample,\n        random_state=SEED\n    )\n\ntest_sample_df = test_sample_df.sort_values(\"id\").reset_index(drop=True)\n\nprint(\"Class mapping:\", class_to_idx)\nprint(\"Training sample size:\", len(train_sample_df))\nprint(\"Train split size:\", len(train_df))\nprint(\"Validation split size:\", len(val_df))\nprint(\"Test sample size:\", len(test_sample_df))\n\nprint(\"\\nSampled training class counts:\")\ndisplay(train_sample_df[\"label\"].value_counts().sort_index().to_frame(\"count\"))\n\nprint(\"\\nTrain split class counts:\")\ndisplay(train_df[\"label\"].value_counts().sort_index().to_frame(\"count\"))\n\nprint(\"\\nValidation split class counts:\")\ndisplay(val_df[\"label\"].value_counts().sort_index().to_frame(\"count\"))","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-05-01T21:29:37.204658Z","iopub.execute_input":"2026-05-01T21:29:37.205064Z","iopub.status.idle":"2026-05-01T21:29:37.380967Z","shell.execute_reply.started":"2026-05-01T21:29:37.205032Z","shell.execute_reply":"2026-05-01T21:29:37.380210Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"!pip install py7zr","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-05-01T21:31:17.549559Z","iopub.execute_input":"2026-05-01T21:31:17.550652Z","iopub.status.idle":"2026-05-01T21:31:25.359097Z","shell.execute_reply.started":"2026-05-01T21:31:17.550599Z","shell.execute_reply":"2026-05-01T21:31:25.357985Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"from pathlib import Path\nfrom typing import Iterable\nimport py7zr\n\nEXTRACT_ROOT = Path(\"/kaggle/temp/cifar10_extracted_sample\")\nTRAIN_DIR = EXTRACT_ROOT / \"train\"\nTEST_DIR = EXTRACT_ROOT / \"test\"\n\ndef extract_selected_7z(archive_path: Path, output_dir: Path, targets: Iterable[str]) -> None:\n    output_dir.mkdir(parents=True, exist_ok=True)\n    targets = sorted({str(t).replace(\"\\\\\", \"/\") for t in targets})\n\n    with py7zr.SevenZipFile(archive_path, \"r\") as archive:\n        available = {name.replace(\"\\\\\", \"/\") for name in archive.getnames()}\n        targets = sorted(set(targets) & available)\n\n        if not targets:\n            raise FileNotFoundError(f\"No requested files found in {archive_path}\")\n\n        archive.extract(path=output_dir, targets=targets)\n        print(f\"Extracted {len(targets):,} files from {archive_path.name} → {output_dir}\")\n\ntrain_ids = sorted(set(train_df[\"id\"]).union(val_df[\"id\"]))\ntest_ids = sorted(test_sample_df[\"id\"])\n\ntrain_targets = [f\"train/{i}.png\" for i in train_ids]\ntest_targets = [f\"test/{i}.png\" for i in test_ids]\n\nif not (TRAIN_DIR / f\"{train_ids[0]}.png\").exists():\n    extract_selected_7z(TRAIN_ARCHIVE, EXTRACT_ROOT, train_targets)\nelse:\n    print(\"Train images already extracted.\")\n\nif not (TEST_DIR / f\"{test_ids[0]}.png\").exists():\n    extract_selected_7z(TEST_ARCHIVE, EXTRACT_ROOT, test_targets)\nelse:\n    print(\"Test images already extracted.\")\n\nprint(\"Train example:\", (TRAIN_DIR / f\"{train_ids[0]}.png\").exists())\nprint(\"Test example :\", (TEST_DIR / f\"{test_ids[0]}.png\").exists())","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-05-01T21:31:48.565625Z","iopub.execute_input":"2026-05-01T21:31:48.566010Z","iopub.status.idle":"2026-05-01T21:35:01.739059Z","shell.execute_reply.started":"2026-05-01T21:31:48.565980Z","shell.execute_reply":"2026-05-01T21:35:01.737957Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## Step 3. Extract Only the Sampled Images from the 7-Zip Archives\n\nThis cell avoids extracting the full Kaggle archive when developing locally. It extracts only the sampled `train/{id}.png` and `test/{id}.png` files needed for this notebook run.","metadata":{}},{"cell_type":"markdown","source":"## Step 4. Define Image Dataset Class, Transforms, and DataLoaders","metadata":{}},{"cell_type":"code","source":"class CIFAR10ImageDataset(Dataset):\n    def __init__(self, df: pd.DataFrame, image_dir: Path, transform=None, is_test: bool = False):\n        self.df = df.reset_index(drop=True)\n        self.image_dir = Path(image_dir)\n        self.transform = transform\n        self.is_test = is_test\n\n    def __len__(self):\n        return len(self.df)\n\n    def __getitem__(self, idx: int):\n        row = self.df.iloc[idx]\n        image_id = int(row[\"id\"])\n        image_path = self.image_dir / f\"{image_id}.png\"\n\n        if not image_path.exists():\n            raise FileNotFoundError(f\"Image not found: {image_path}\")\n\n        image = Image.open(image_path).convert(\"RGB\")\n        if self.transform:\n            image = self.transform(image)\n\n        if self.is_test:\n            return image, image_id\n        else:\n            label = int(row[\"label_idx\"])\n            return image, label\n\ntrain_transform = transforms.Compose([\n    transforms.RandomHorizontalFlip(p=0.5),\n    transforms.ToTensor(),\n    transforms.Normalize(mean=(0.4914, 0.4822, 0.4465), std=(0.2470, 0.2435, 0.2616)),\n])\n\nval_test_transform = transforms.Compose([\n    transforms.ToTensor(),\n    transforms.Normalize(mean=(0.4914, 0.4822, 0.4465), std=(0.2470, 0.2435, 0.2616)),\n])\n\ntrain_dataset = CIFAR10ImageDataset(train_df, TRAIN_DIR, transform=train_transform, is_test=False)\nval_dataset = CIFAR10ImageDataset(val_df, TRAIN_DIR, transform=val_test_transform, is_test=False)\ntest_dataset = CIFAR10ImageDataset(test_sample_df, TEST_DIR, transform=val_test_transform, is_test=True)\n\ntrain_loader = DataLoader(train_dataset, batch_size=BATCH_SIZE, shuffle=True, num_workers=NUM_WORKERS)\nval_loader = DataLoader(val_dataset, batch_size=BATCH_SIZE, shuffle=False, num_workers=NUM_WORKERS)\ntest_loader = DataLoader(test_dataset, batch_size=BATCH_SIZE, shuffle=False, num_workers=NUM_WORKERS)\n\nprint(\"Train batches:\", len(train_loader))\nprint(\"Validation batches:\", len(val_loader))\nprint(\"Test batches:\", len(test_loader))","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-05-01T21:35:43.551027Z","iopub.execute_input":"2026-05-01T21:35:43.551675Z","iopub.status.idle":"2026-05-01T21:35:43.567330Z","shell.execute_reply.started":"2026-05-01T21:35:43.551637Z","shell.execute_reply":"2026-05-01T21:35:43.566617Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## Step 5. Visualize Sample Images","metadata":{}},{"cell_type":"code","source":"def unnormalize_for_display(tensor_img):\n    mean = torch.tensor([0.4914, 0.4822, 0.4465]).view(3, 1, 1)\n    std = torch.tensor([0.2470, 0.2435, 0.2616]).view(3, 1, 1)\n    img = tensor_img.cpu() * std + mean\n    return img.clamp(0, 1)\n\nimages, labels = next(iter(train_loader))\nn_show = min(8, images.size(0))\nplt.figure(figsize=(12, 3))\nfor i in range(n_show):\n    plt.subplot(1, n_show, i + 1)\n    img = unnormalize_for_display(images[i]).permute(1, 2, 0).numpy()\n    plt.imshow(img)\n    plt.title(idx_to_class[int(labels[i])])\n    plt.axis(\"off\")\nplt.tight_layout()\nplt.show()","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-05-01T21:35:53.284385Z","iopub.execute_input":"2026-05-01T21:35:53.284991Z","iopub.status.idle":"2026-05-01T21:35:54.227574Z","shell.execute_reply.started":"2026-05-01T21:35:53.284960Z","shell.execute_reply":"2026-05-01T21:35:54.226755Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## Step 6. Build a Lightweight CNN Model\n\nThis CNN is intentionally compact for CPU development. It uses convolution, ReLU activation, max pooling, dropout, and fully connected layers.","metadata":{}},{"cell_type":"code","source":"class CIFARDeepCNN(nn.Module):\n    def __init__(self, num_classes: int = 10):\n        super().__init__()\n\n        self.features = nn.Sequential(\n            # Block 1: 3 x 32 x 32 -> 64 x 16 x 16\n            nn.Conv2d(3, 64, kernel_size=3, padding=1, bias=False),\n            nn.BatchNorm2d(64),\n            nn.ReLU(inplace=True),\n            nn.Conv2d(64, 64, kernel_size=3, padding=1, bias=False),\n            nn.BatchNorm2d(64),\n            nn.ReLU(inplace=True),\n            nn.MaxPool2d(2),\n            nn.Dropout(0.20),\n\n            # Block 2: 64 x 16 x 16 -> 128 x 8 x 8\n            nn.Conv2d(64, 128, kernel_size=3, padding=1, bias=False),\n            nn.BatchNorm2d(128),\n            nn.ReLU(inplace=True),\n            nn.Conv2d(128, 128, kernel_size=3, padding=1, bias=False),\n            nn.BatchNorm2d(128),\n            nn.ReLU(inplace=True),\n            nn.MaxPool2d(2),\n            nn.Dropout(0.30),\n\n            # Block 3: 128 x 8 x 8 -> 256 x 4 x 4\n            nn.Conv2d(128, 256, kernel_size=3, padding=1, bias=False),\n            nn.BatchNorm2d(256),\n            nn.ReLU(inplace=True),\n            nn.Conv2d(256, 256, kernel_size=3, padding=1, bias=False),\n            nn.BatchNorm2d(256),\n            nn.ReLU(inplace=True),\n            nn.MaxPool2d(2),\n            nn.Dropout(0.40),\n        )\n\n        self.classifier = nn.Sequential(\n            nn.Flatten(),\n            nn.Linear(256 * 4 * 4, 512),\n            nn.ReLU(inplace=True),\n            nn.BatchNorm1d(512),\n            nn.Dropout(0.50),\n            nn.Linear(512, num_classes),\n        )\n\n    def forward(self, x):\n        return self.classifier(self.features(x))\n\n\nmodel = CIFARDeepCNN(num_classes=NUM_CLASSES).to(device)\n\ncriterion = nn.CrossEntropyLoss()\n\noptimizer = torch.optim.AdamW(\n    model.parameters(),\n    lr=LEARNING_RATE,\n    weight_decay=1e-4\n)\n\nscheduler = torch.optim.lr_scheduler.ReduceLROnPlateau(\n    optimizer,\n    mode=\"max\",\n    factor=0.5,\n    patience=3\n)\n\nprint(model)\nprint(\"Trainable parameters:\", sum(p.numel() for p in model.parameters() if p.requires_grad))","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-05-01T21:36:01.144465Z","iopub.execute_input":"2026-05-01T21:36:01.145356Z","iopub.status.idle":"2026-05-01T21:36:01.482935Z","shell.execute_reply.started":"2026-05-01T21:36:01.145319Z","shell.execute_reply":"2026-05-01T21:36:01.481847Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## Step 7. Train the CNN and Monitor Loss/Accuracy","metadata":{}},{"cell_type":"code","source":"def train_one_epoch(model, loader, criterion, optimizer, device):\n    model.train()\n    running_loss, correct, total = 0.0, 0, 0\n\n    for images, labels in loader:\n        images = images.to(device, non_blocking=True)\n        labels = labels.to(device, non_blocking=True)\n\n        optimizer.zero_grad(set_to_none=True)\n        outputs = model(images)\n        loss = criterion(outputs, labels)\n\n        loss.backward()\n        optimizer.step()\n\n        running_loss += loss.item() * images.size(0)\n        correct += (outputs.argmax(1) == labels).sum().item()\n        total += labels.size(0)\n\n    return running_loss / total, correct / total\n\n\n@torch.no_grad()\ndef evaluate(model, loader, criterion, device):\n    model.eval()\n    running_loss, correct, total = 0.0, 0, 0\n    all_labels, all_preds = [], []\n\n    for images, labels in loader:\n        images = images.to(device, non_blocking=True)\n        labels = labels.to(device, non_blocking=True)\n\n        outputs = model(images)\n        loss = criterion(outputs, labels)\n\n        preds = outputs.argmax(1)\n        running_loss += loss.item() * images.size(0)\n        correct += (preds == labels).sum().item()\n        total += labels.size(0)\n\n        all_labels.extend(labels.cpu().numpy())\n        all_preds.extend(preds.cpu().numpy())\n\n    return running_loss / total, correct / total, np.array(all_labels), np.array(all_preds)\n\n\nhistory = []\nbest_val_acc = 0.0\nbest_model_path = \"best_cifar_cnn.pt\"\n\nfor epoch in range(1, NUM_EPOCHS + 1):\n    train_loss, train_acc = train_one_epoch(\n        model, train_loader, criterion, optimizer, device\n    )\n\n    val_loss, val_acc, _, _ = evaluate(\n        model, val_loader, criterion, device\n    )\n\n    scheduler.step(val_acc)\n\n    if val_acc > best_val_acc:\n        best_val_acc = val_acc\n        torch.save(model.state_dict(), best_model_path)\n\n    history.append({\n        \"epoch\": epoch,\n        \"train_loss\": train_loss,\n        \"train_acc\": train_acc,\n        \"val_loss\": val_loss,\n        \"val_acc\": val_acc,\n        \"lr\": optimizer.param_groups[0][\"lr\"],\n    })\n\n    print(\n        f\"Epoch {epoch:02d}/{NUM_EPOCHS} | \"\n        f\"Train Loss: {train_loss:.4f} | Train Acc: {train_acc:.4f} | \"\n        f\"Val Loss: {val_loss:.4f} | Val Acc: {val_acc:.4f} | \"\n        f\"LR: {optimizer.param_groups[0]['lr']:.6f}\"\n    )\n\nhistory_df = pd.DataFrame(history)\ndisplay(history_df)\n\nprint(f\"Best validation accuracy: {best_val_acc:.4f}\")\nprint(f\"Best model saved to: {best_model_path}\")","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-05-01T21:36:12.968487Z","iopub.execute_input":"2026-05-01T21:36:12.969193Z","iopub.status.idle":"2026-05-01T21:46:17.057670Z","shell.execute_reply.started":"2026-05-01T21:36:12.969162Z","shell.execute_reply":"2026-05-01T21:46:17.056825Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## Step 8. Plot Training History","metadata":{}},{"cell_type":"code","source":"plt.figure(figsize=(8, 5))\nplt.plot(history_df[\"epoch\"], history_df[\"train_loss\"], marker=\"o\", label=\"Train Loss\")\nplt.plot(history_df[\"epoch\"], history_df[\"val_loss\"], marker=\"o\", label=\"Validation Loss\")\nplt.xlabel(\"Epoch\")\nplt.ylabel(\"Loss\")\nplt.title(\"CNN Training and Validation Loss\")\nplt.legend()\nplt.grid(True, alpha=0.3)\nplt.show()\n\nplt.figure(figsize=(8, 5))\nplt.plot(history_df[\"epoch\"], history_df[\"train_acc\"], marker=\"o\", label=\"Train Accuracy\")\nplt.plot(history_df[\"epoch\"], history_df[\"val_acc\"], marker=\"o\", label=\"Validation Accuracy\")\nplt.xlabel(\"Epoch\")\nplt.ylabel(\"Accuracy\")\nplt.title(\"CNN Training and Validation Accuracy\")\nplt.legend()\nplt.grid(True, alpha=0.3)\nplt.show()","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-05-01T21:58:44.895747Z","iopub.execute_input":"2026-05-01T21:58:44.896297Z","iopub.status.idle":"2026-05-01T21:58:45.227409Z","shell.execute_reply.started":"2026-05-01T21:58:44.896258Z","shell.execute_reply":"2026-05-01T21:58:45.226635Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## Step 9. Evaluate the Model on the Validation Set","metadata":{}},{"cell_type":"code","source":"val_loss, val_acc, y_true, y_pred = evaluate(model, val_loader, criterion, device)\nprint(f\"Validation loss: {val_loss:.4f}\")\nprint(f\"Validation accuracy: {val_acc:.4f}\")\n\nprint(\"\\nClassification Report:\")\nprint(classification_report(\n    y_true,\n    y_pred,\n    labels=list(range(NUM_CLASSES)),\n    target_names=[idx_to_class[i] for i in range(NUM_CLASSES)],\n    zero_division=0,\n))\n\ncm = confusion_matrix(y_true, y_pred, labels=list(range(NUM_CLASSES)))\nplt.figure(figsize=(8, 7))\nplt.imshow(cm, interpolation=\"nearest\")\nplt.title(\"Validation Confusion Matrix\")\nplt.colorbar()\nplt.xticks(range(NUM_CLASSES), [idx_to_class[i] for i in range(NUM_CLASSES)], rotation=45, ha=\"right\")\nplt.yticks(range(NUM_CLASSES), [idx_to_class[i] for i in range(NUM_CLASSES)])\nplt.xlabel(\"Predicted Label\")\nplt.ylabel(\"True Label\")\nfor i in range(NUM_CLASSES):\n    for j in range(NUM_CLASSES):\n        plt.text(j, i, cm[i, j], ha=\"center\", va=\"center\", fontsize=8)\nplt.tight_layout()\nplt.show()","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-05-01T21:58:53.952610Z","iopub.execute_input":"2026-05-01T21:58:53.952932Z","iopub.status.idle":"2026-05-01T21:58:58.115757Z","shell.execute_reply.started":"2026-05-01T21:58:53.952906Z","shell.execute_reply":"2026-05-01T21:58:58.114811Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## Step 10. Visualize Validation Predictions","metadata":{}},{"cell_type":"code","source":"model.eval()\nimages, labels = next(iter(val_loader))\nwith torch.no_grad():\n    outputs = model(images.to(device))\n    preds = outputs.argmax(dim=1).cpu()\n\nn_show = min(8, images.size(0))\nplt.figure(figsize=(14, 3.5))\nfor i in range(n_show):\n    plt.subplot(1, n_show, i + 1)\n    img = unnormalize_for_display(images[i]).permute(1, 2, 0).numpy()\n    plt.imshow(img)\n    true_label = idx_to_class[int(labels[i])]\n    pred_label = idx_to_class[int(preds[i])]\n    plt.title(f\"True: {true_label} Pred: {pred_label}\", fontsize=9)\n    plt.axis(\"off\")\nplt.tight_layout()\nplt.show()","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-05-01T21:59:03.892013Z","iopub.execute_input":"2026-05-01T21:59:03.892508Z","iopub.status.idle":"2026-05-01T21:59:04.563061Z","shell.execute_reply.started":"2026-05-01T21:59:03.892472Z","shell.execute_reply":"2026-05-01T21:59:04.562065Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## Step 11. Generate Kaggle-Style Predictions and Save `submission.csv`\n\nFor local development, this cell creates predictions for the sampled test set. For an official Kaggle submission, set `TEST_SAMPLE_FRACTION = 1.00`, ensure all test images are extracted or accessible, rerun prediction, and submit the full `submission.csv`.","metadata":{}},{"cell_type":"code","source":"@torch.no_grad()\ndef predict_test(model, loader, device):\n    model.eval()\n    rows = []\n    for images, image_ids in loader:\n        images = images.to(device)\n        outputs = model(images)\n        preds = outputs.argmax(dim=1).cpu().numpy()\n        for image_id, pred_idx in zip(image_ids.numpy(), preds):\n            rows.append({\"id\": int(image_id), \"label\": idx_to_class[int(pred_idx)]})\n    return pd.DataFrame(rows).sort_values(\"id\").reset_index(drop=True)\n\nsubmission_df = predict_test(model, test_loader, device)\nsubmission_path = Path(\"submission.csv\")\nsubmission_df.to_csv(submission_path, index=False)\n\nprint(f\"Saved: {submission_path.resolve()}\")\nprint(\"Submission rows:\", len(submission_df))\ndisplay(submission_df.head())","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-05-01T21:59:11.897229Z","iopub.execute_input":"2026-05-01T21:59:11.898038Z","iopub.status.idle":"2026-05-01T22:00:59.886756Z","shell.execute_reply.started":"2026-05-01T21:59:11.898000Z","shell.execute_reply":"2026-05-01T22:00:59.885969Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## Step 12. Descriptive Evaluation of the CNN Workflow\n\nThis notebook demonstrates a complete small-scale CNN image classification pipeline for the CIFAR-10 Kaggle competition. The workflow loads the Kaggle metadata files, samples a small stratified subset of training images, extracts only the required image files from the local `.7z` archives, normalizes the images, builds a lightweight CNN, trains the model, evaluates validation performance, visualizes predictions, and produces a Kaggle-style `final_submission.csv` file.\n\nBecause the starting configuration uses a 1% sample, validation accuracy should be interpreted as a **pipeline validation result**, not as the best achievable CIFAR-10 model performance. The purpose of this notebook is to confirm that the data ingestion, image preprocessing, CNN architecture, training loop, evaluation logic, and submission pipeline all work in a low-compute CPU environment. Once verified, the same notebook can be scaled by increasing `SAMPLE_FRACTION`, increasing `TEST_SAMPLE_FRACTION`, raising `BATCH_SIZE`, and training for more epochs on a GPU-enabled environment.","metadata":{}},{"cell_type":"markdown","source":"## Step. Scaling Notes for GPU/TPU or Kaggle Notebook Execution\n\nRecommended changes for a stronger final run:\n\n1. Increase `SAMPLE_FRACTION` from `0.01` to `0.10`, `0.25`, `0.50`, and eventually `1.00`.\n2. Increase `TEST_SAMPLE_FRACTION` to `1.00` before creating the final Kaggle submission.\n3. Increase `BATCH_SIZE` from `8` to `64` or `128` if GPU memory allows.\n4. Increase `NUM_EPOCHS` from `3` to `20+` for stronger convergence.\n5. Consider a deeper CNN or a transfer-learning model.\n6. Add stronger image augmentation such as random crop, padding, color jitter, and random erasing.\n\nThe key advantage of this notebook design is that the same code path works for both small local CPU development and later larger-scale GPU execution.","metadata":{}},{"cell_type":"code","source":"print(\"Notebook project complete.\")","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-04-30T06:29:42.184795Z","iopub.execute_input":"2026-04-30T06:29:42.185303Z","iopub.status.idle":"2026-04-30T06:29:42.190436Z","shell.execute_reply.started":"2026-04-30T06:29:42.185262Z","shell.execute_reply":"2026-04-30T06:29:42.189413Z"}},"outputs":[],"execution_count":null}]}