{"metadata":{"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"name":"python","version":"3.13.15","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"}},"nbformat_minor":4,"nbformat":4,"cells":[{"id":"b4d9b372","cell_type":"markdown","source":"# CIFAR-10 Challenge: DSEGNet-12, a custom SE-residual ConvNet trained from scratch\n\nDSEG660 challenge assignment. Everything here is trained from **random initialization** (no pre-trained weights anywhere).\n\n| Requirement | What this notebook does |\n|---|---|\n| Own network design | `DSEGNet-12`, defined from scratch below (not an off-the-shelf architecture) |\n| Advanced optimizer | **AdamW** (decoupled weight decay, no decay on BatchNorm/bias parameters) |\n| Cosine schedule with warmup | Linear **warmup** for 5 epochs, then **cosine annealing** to near zero, updated every step |\n| Advanced augmentation | **CutMix** (p = 0.5, alpha = 1.0) on top of random crop + horizontal flip |\n\nExtras: label smoothing, an exponential moving average (EMA) of the weights, mixed precision, and flip test-time augmentation.\n\n**Before running:** Settings → Accelerator = GPU (T4 x2 or P100), Internet = On, and the CIFAR-10 competition data attached (Add Input → Competitions → CIFAR-10). Then *Save Version → Save & Run All*.","metadata":{}},{"id":"2252f9e2","cell_type":"code","source":"import os, glob, time, math, shutil, subprocess, sys, copy\nimport numpy as np\nimport pandas as pd\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nfrom PIL import Image\nfrom concurrent.futures import ProcessPoolExecutor\nimport matplotlib.pyplot as plt\n\n# ---------------- configuration ----------------\nSEED              = 42\nEPOCHS            = 150\nWARMUP_EPOCHS     = 5\nBATCH_SIZE        = 512\nPEAK_LR           = 2e-3      # AdamW peak learning rate (reached at the end of warmup)\nMIN_LR_FRAC       = 0.005     # cosine floor, as a fraction of PEAK_LR\nWEIGHT_DECAY      = 0.05      # decoupled (AdamW); not applied to BatchNorm / bias\nBETAS             = (0.9, 0.999)\nLABEL_SMOOTHING   = 0.1\nCUTMIX_PROB       = 0.5       # share of batches that get CutMix\nCUTMIX_ALPHA      = 1.0\nCUTMIX_OFF_LAST   = 10        # last N epochs train on un-mixed images\nEMA_DECAY         = 0.999\nWIDTHS            = (64, 192, 384, 768)   # stem, stage1, stage2, stage3\nFALLBACK_VAL_SIZE = 5000      # only used if the official test set cannot be downloaded\nN_TRAIN, N_TEST   = 50_000, 300_000\n\nCLASSES = [\"airplane\", \"automobile\", \"bird\", \"cat\", \"deer\",\n           \"dog\", \"frog\", \"horse\", \"ship\", \"truck\"]   # same order as torchvision CIFAR10\n\nOUT_DIR = \"/kaggle/working\" if os.path.isdir(\"/kaggle/working\") else \".\"\nWORK    = \"/kaggle/temp/cifar10\" if os.path.isdir(\"/kaggle/temp\") else \"/tmp/cifar10\"\nos.makedirs(WORK, exist_ok=True)\n\ntorch.manual_seed(SEED); np.random.seed(SEED)\ntorch.backends.cudnn.benchmark = True\ndevice  = \"cuda\" if torch.cuda.is_available() else \"cpu\"\nuse_amp = device == \"cuda\"\nprint(\"device:\", device, \"|\", torch.cuda.get_device_name(0) if use_amp else \"CPU only (training will be very slow)\")","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-10-10T15:31:48.74006Z","iopub.execute_input":"2026-10-10T15:31:48.740482Z","iopub.status.idle":"2026-10-10T15:32:04.423358Z","shell.execute_reply.started":"2026-10-10T15:31:48.740446Z","shell.execute_reply":"2026-10-10T15:32:04.422488Z"}},"outputs":[],"execution_count":null},{"id":"b7d2b7af","cell_type":"markdown","source":"## 1. Extract and load the Kaggle data","metadata":{}},{"id":"48bdfd66","cell_type":"code","source":"def find(name):\n    hits = glob.glob(f\"/kaggle/input/**/{name}\", recursive=True)\n    assert hits, f\"{name} not found under /kaggle/input. Attach the CIFAR-10 competition data.\"\n    return hits[0]\n\ndef extract_7z(archive, expected):\n    folder = os.path.join(WORK, os.path.basename(archive)[:-3])      # train.7z -> WORK/train\n    if os.path.isdir(folder) and len(os.listdir(folder)) == expected:\n        print(f\"{folder} already extracted\"); return folder\n    t = time.time()\n    if shutil.which(\"7z\") is None:\n        subprocess.run(\"apt-get install -y -q p7zip-full > /dev/null 2>&1\", shell=True)\n    if shutil.which(\"7z\"):\n        subprocess.run([\"7z\", \"x\", \"-y\", f\"-o{WORK}\", archive], check=True, stdout=subprocess.DEVNULL)\n    else:\n        subprocess.run([sys.executable, \"-m\", \"pip\", \"install\", \"-q\", \"py7zr\"], check=True)\n        import py7zr\n        with py7zr.SevenZipFile(archive) as z: z.extractall(WORK)\n    n = len(os.listdir(folder))\n    assert n == expected, f\"expected {expected} files in {folder}, found {n}\"\n    print(f\"extracted {n} files to {folder} in {time.time()-t:.0f}s\"); return folder\n\ndef load_png(path):\n    return np.asarray(Image.open(path).convert(\"RGB\"), dtype=np.uint8)\n\ndef load_folder(folder, n, cache):\n    if os.path.exists(cache): return np.load(cache)\n    paths = [f\"{folder}/{i}.png\" for i in range(1, n + 1)]          # ids are 1..n\n    with ProcessPoolExecutor(os.cpu_count()) as ex:\n        arr = np.stack(list(ex.map(load_png, paths, chunksize=1000)))\n    np.save(cache, arr); return arr\n\ntrain_dir = extract_7z(find(\"train.7z\"), N_TRAIN)\ntest_dir  = extract_7z(find(\"test.7z\"),  N_TEST)\n\nt = time.time()\nx_train = load_folder(train_dir, N_TRAIN, f\"{WORK}/x_train.npy\")\nx_test  = load_folder(test_dir,  N_TEST,  f\"{WORK}/x_test.npy\")\nlabels  = pd.read_csv(find(\"trainLabels.csv\")).sort_values(\"id\")\ny_train = labels[\"label\"].map({c: i for i, c in enumerate(CLASSES)}).values.astype(np.int64)\nprint(\"train\", x_train.shape, \"test\", x_test.shape, f\"loaded in {time.time()-t:.0f}s\")\nprint(\"class counts:\", np.bincount(y_train))","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-10-10T15:39:54.784422Z","iopub.execute_input":"2026-10-10T15:39:54.785108Z","iopub.status.idle":"2026-10-10T15:42:05.871978Z","shell.execute_reply.started":"2026-10-10T15:39:54.785052Z","shell.execute_reply":"2026-10-10T15:42:05.871075Z"}},"outputs":[],"execution_count":null},{"id":"7b2945b6","cell_type":"markdown","source":"## 2. Validation data\n\nThe assignment notes that the Kaggle score equals accuracy on the official CIFAR-10 test set. So the model trains on **all 50,000** Kaggle training images and the official 10,000-image test set (downloaded with torchvision, labels used only for evaluation, never for training) gives the score estimate. If the download fails, it falls back to holding out 5,000 training images.","metadata":{}},{"id":"59118352","cell_type":"code","source":"val_source = None\ntry:\n    import torchvision\n    off = torchvision.datasets.CIFAR10(root=WORK, train=False, download=True)\n    x_tr, y_tr = x_train, y_train\n    x_val, y_val = off.data, np.array(off.targets, dtype=np.int64)\n    val_source = \"official CIFAR-10 test set (10,000 images)\"\nexcept Exception as e:\n    print(\"Official test set unavailable:\", repr(e)[:200])\nif val_source is None:\n    rng = np.random.RandomState(SEED); idx = rng.permutation(len(x_train))\n    v, tr = idx[:FALLBACK_VAL_SIZE], idx[FALLBACK_VAL_SIZE:]\n    x_tr, y_tr, x_val, y_val = x_train[tr], y_train[tr], x_train[v], y_train[v]\n    val_source = f\"held-out training images ({FALLBACK_VAL_SIZE})\"\nprint(\"training on\", len(x_tr), \"images | validating on\", val_source)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-10-10T15:43:04.563819Z","iopub.execute_input":"2026-10-10T15:43:04.564567Z","iopub.status.idle":"2026-10-10T16:04:53.545106Z","shell.execute_reply.started":"2026-10-10T15:43:04.564535Z","shell.execute_reply":"2026-10-10T16:04:53.544271Z"}},"outputs":[],"execution_count":null},{"id":"7ec527b3","cell_type":"markdown","source":"## 3. Model: DSEGNet-12 (own design)\n\nA compact residual network designed for 32×32 inputs:\n\n* **Stem**: 3×3 conv, 64 channels, at full 32×32 resolution.\n* **Three stages**, each opening with a *transition* (3×3 conv that widens the channels, BatchNorm, ReLU, 2×2 max-pool to halve resolution), followed by **SE-residual blocks** (two 3×3 conv-BN layers, a squeeze-and-excitation gate that reweights channels, and an identity shortcut).\n  * Stage 1: 64 → 192 channels, 32×32 → 16×16, 1 SE-residual block\n  * Stage 2: 192 → 384 channels, 16×16 → 8×8, 2 SE-residual blocks (the deepest stage, where features are most class-specific but still spatially rich)\n  * Stage 3: 384 → 768 channels, 8×8 → 4×4, 1 SE-residual block\n* **Head**: concatenated global average + global max pooling (1,536 features), dropout 0.2, linear layer to 10 classes.\n\n12 convolutional layers + 1 linear layer. Max-pool downsampling (instead of strided convs) keeps the network shallow and fast; SE gates add channel attention at almost no cost; the last BatchNorm in every residual branch starts at zero so each block begins as an identity map, which stabilizes early training with an adaptive optimizer. All conv weights use Kaiming-normal random initialization.","metadata":{}},{"id":"8c67d52a","cell_type":"code","source":"class ConvBNAct(nn.Sequential):\n    def __init__(self, cin, cout, pool=False):\n        layers = [nn.Conv2d(cin, cout, 3, padding=1, bias=False), nn.BatchNorm2d(cout), nn.ReLU(inplace=True)]\n        if pool: layers.append(nn.MaxPool2d(2))\n        super().__init__(*layers)\n\nclass SqueezeExcite(nn.Module):\n    def __init__(self, c, reduction=8):\n        super().__init__()\n        self.fc1 = nn.Conv2d(c, c // reduction, 1)\n        self.fc2 = nn.Conv2d(c // reduction, c, 1)\n    def forward(self, x):\n        s = x.mean((2, 3), keepdim=True)\n        return x * torch.sigmoid(self.fc2(F.relu(self.fc1(s), inplace=True)))\n\nclass SEResidual(nn.Module):\n    def __init__(self, c):\n        super().__init__()\n        self.body = nn.Sequential(\n            ConvBNAct(c, c),\n            nn.Conv2d(c, c, 3, padding=1, bias=False), nn.BatchNorm2d(c),\n            SqueezeExcite(c))\n        nn.init.zeros_(self.body[2].weight)          # zero-init last BN gamma: block starts as identity\n    def forward(self, x):\n        return F.relu(x + self.body(x))\n\nclass DSEGNet(nn.Module):\n    def __init__(self, widths=(64, 192, 384, 768), num_classes=10, dropout=0.2):\n        super().__init__()\n        c0, c1, c2, c3 = widths\n        self.stem   = ConvBNAct(3, c0)\n        self.stage1 = nn.Sequential(ConvBNAct(c0, c1, pool=True), SEResidual(c1))\n        self.stage2 = nn.Sequential(ConvBNAct(c1, c2, pool=True), SEResidual(c2), SEResidual(c2))\n        self.stage3 = nn.Sequential(ConvBNAct(c2, c3, pool=True), SEResidual(c3))\n        self.drop   = nn.Dropout(dropout)\n        self.fc     = nn.Linear(2 * c3, num_classes)\n        for m in self.modules():                     # random initialization, no pre-trained weights\n            if isinstance(m, nn.Conv2d) and m.kernel_size == (3, 3):\n                nn.init.kaiming_normal_(m.weight, mode=\"fan_out\", nonlinearity=\"relu\")\n    def forward(self, x):\n        x = self.stage3(self.stage2(self.stage1(self.stem(x))))\n        x = torch.cat([F.adaptive_avg_pool2d(x, 1), F.adaptive_max_pool2d(x, 1)], 1).flatten(1)\n        return self.fc(self.drop(x))\n\nmodel = DSEGNet(WIDTHS).to(device).to(memory_format=torch.channels_last)\nn_params = sum(p.numel() for p in model.parameters())\nn_conv = sum(isinstance(m, nn.Conv2d) and m.kernel_size == (3, 3) for m in model.modules())\nprint(f\"DSEGNet: {n_params/1e6:.2f}M parameters, {n_conv} conv3x3 layers + 1 linear\")","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-10-10T16:05:09.869019Z","iopub.execute_input":"2026-10-10T16:05:09.869865Z","iopub.status.idle":"2026-10-10T16:05:10.738751Z","shell.execute_reply.started":"2026-10-10T16:05:09.869835Z","shell.execute_reply":"2026-10-10T16:05:10.738057Z"}},"outputs":[],"execution_count":null},{"id":"330fbe37","cell_type":"markdown","source":"## 4. GPU augmentation: random crop + flip, then CutMix\n\nThe whole training set lives on the GPU as uint8 and every batch is augmented there (fast even with Kaggle's few CPU cores).\n\n* **Random crop** (reflect-pad 4 px, crop 32×32) and **horizontal flip** on every image.\n* **CutMix** on 50% of batches: a random box from a shuffled partner image is pasted in, and the loss is mixed in proportion to the pasted area. It is switched off for the last 10 epochs so the model finishes on clean images.","metadata":{}},{"id":"d5aae16e","cell_type":"code","source":"MEAN = torch.tensor([0.4914, 0.4822, 0.4465], device=device).view(1, 3, 1, 1) * 255\nSTD  = torch.tensor([0.2470, 0.2435, 0.2616], device=device).view(1, 3, 1, 1) * 255\n\ndef normalize(xb):                       # uint8 NHWC -> normalized float NCHW (channels_last)\n    x = (xb.permute(0, 3, 1, 2).float() - MEAN) / STD\n    return x.contiguous(memory_format=torch.channels_last)\n\ndef random_crop_flip(x, pad=4):\n    B, C, H, W = x.shape\n    xp = F.pad(x, (pad, pad, pad, pad), mode=\"reflect\")\n    i = torch.randint(0, 2 * pad + 1, (B,), device=x.device)\n    j = torch.randint(0, 2 * pad + 1, (B,), device=x.device)\n    rows = (i[:, None] + torch.arange(H, device=x.device))[:, :, None]\n    cols = (j[:, None] + torch.arange(W, device=x.device))[:, None, :]\n    out = xp[torch.arange(B, device=x.device)[:, None, None], :, rows, cols].permute(0, 3, 1, 2)\n    flip = torch.rand(B, device=x.device) < 0.5\n    out = torch.where(flip[:, None, None, None], out.flip(3), out)\n    return out.contiguous(memory_format=torch.channels_last)\n\ndef cutmix(x, y, alpha=1.0):\n    B, _, H, W = x.shape\n    lam = np.random.beta(alpha, alpha)\n    ch, cw = int(H * math.sqrt(1 - lam)), int(W * math.sqrt(1 - lam))\n    cy, cx = np.random.randint(H), np.random.randint(W)\n    y1, y2 = max(cy - ch // 2, 0), min(cy + ch // 2, H)\n    x1, x2 = max(cx - cw // 2, 0), min(cx + cw // 2, W)\n    perm = torch.randperm(B, device=x.device)\n    x[:, :, y1:y2, x1:x2] = x[perm, :, y1:y2, x1:x2]\n    lam = 1 - (y2 - y1) * (x2 - x1) / (H * W)       # actual area kept from the original image\n    return x, y, y[perm], lam\n\n# quick visual check of the augmentation\n_xb = normalize(torch.from_numpy(x_tr[:8]).to(device))\n_xa, _, _, _ = cutmix(random_crop_flip(_xb), torch.from_numpy(y_tr[:8]).to(device))\n_show = ((_xa.float() * STD + MEAN).clamp(0, 255) / 255).permute(0, 2, 3, 1).cpu().numpy()\nfig, ax = plt.subplots(1, 8, figsize=(14, 2))\nfor k in range(8): ax[k].imshow(_show[k]); ax[k].axis(\"off\")\nplt.suptitle(\"crop + flip + CutMix\"); plt.show()","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-10-10T16:05:23.017494Z","iopub.execute_input":"2026-10-10T16:05:23.01777Z","iopub.status.idle":"2026-10-10T16:05:24.512684Z","shell.execute_reply.started":"2026-10-10T16:05:23.017748Z","shell.execute_reply":"2026-10-10T16:05:24.511951Z"}},"outputs":[],"execution_count":null},{"id":"5f9b747c","cell_type":"markdown","source":"## 5. Optimizer, schedule, EMA\n\n* **AdamW**, peak LR 2e-3, betas (0.9, 0.999), decoupled weight decay 0.05 on conv/linear weights only.\n* **Learning-rate schedule**: linear warmup from ~0 to the peak over the first 5 epochs, then cosine annealing down to 0.5% of the peak by the final step (updated every iteration).\n* **Label smoothing** 0.1, **mixed precision** (fp16 autocast + gradient scaling).\n* **EMA** of the weights (decay 0.999, with a short ramp-up) is the model used for the final predictions.","metadata":{}},{"id":"c93f8106","cell_type":"code","source":"decay, no_decay = [], []\nfor n, p in model.named_parameters():\n    (no_decay if p.ndim <= 1 else decay).append(p)        # no weight decay on BN / bias\noptimizer = torch.optim.AdamW([{\"params\": decay, \"weight_decay\": WEIGHT_DECAY},\n                               {\"params\": no_decay, \"weight_decay\": 0.0}],\n                              lr=PEAK_LR, betas=BETAS)\n\nsteps_per_epoch = len(x_tr) // BATCH_SIZE                 # drop the last partial batch\ntotal_steps  = EPOCHS * steps_per_epoch\nwarmup_steps = WARMUP_EPOCHS * steps_per_epoch\n\ndef lr_factor(step):\n    if step < warmup_steps:                               # linear warmup\n        return (step + 1) / warmup_steps\n    t = (step - warmup_steps) / max(1, total_steps - warmup_steps)\n    return MIN_LR_FRAC + (1 - MIN_LR_FRAC) * 0.5 * (1 + math.cos(math.pi * t))   # cosine annealing\nscheduler = torch.optim.lr_scheduler.LambdaLR(optimizer, lr_factor)\n\nclass EMA:\n    def __init__(self, model):\n        self.model = copy.deepcopy(model).eval()\n        for p in self.model.parameters(): p.requires_grad_(False)\n    @torch.no_grad()\n    def update(self, model, step):\n        d = min(EMA_DECAY, (1 + step) / (10 + step))\n        for e, m in zip(self.model.state_dict().values(), model.state_dict().values()):\n            if e.dtype.is_floating_point: e.mul_(d).add_(m.detach(), alpha=1 - d)\n            else: e.copy_(m)\nema = EMA(model)\n\ncriterion = nn.CrossEntropyLoss(label_smoothing=LABEL_SMOOTHING)\nscaler = torch.amp.GradScaler(\"cuda\", enabled=use_amp)\n\n# plot the schedule so it can go in the report\n_lrs = [PEAK_LR * lr_factor(s) for s in range(total_steps)]\nplt.figure(figsize=(6, 2.5)); plt.plot(np.arange(total_steps) / steps_per_epoch, _lrs)\nplt.xlabel(\"epoch\"); plt.ylabel(\"learning rate\"); plt.title(\"Warmup + cosine annealing\")\nplt.tight_layout(); plt.savefig(f\"{OUT_DIR}/lr_schedule.png\", dpi=120); plt.show()","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-10-10T16:05:30.621552Z","iopub.execute_input":"2026-10-10T16:05:30.622372Z","iopub.status.idle":"2026-10-10T16:05:30.898453Z","shell.execute_reply.started":"2026-10-10T16:05:30.622341Z","shell.execute_reply":"2026-10-10T16:05:30.897738Z"}},"outputs":[],"execution_count":null},{"id":"e544fe12","cell_type":"markdown","source":"## 6. Train","metadata":{}},{"id":"b383e8de","cell_type":"code","source":"@torch.no_grad()\ndef predict_probs(net, x_uint8, bs=1024, tta=True):\n    net.eval(); out = []\n    for k in range(0, len(x_uint8), bs):\n        xb = normalize(torch.from_numpy(x_uint8[k:k + bs]).to(device, non_blocking=True))\n        with torch.autocast(device_type=device, dtype=torch.float16, enabled=use_amp):\n            p = net(xb).float().softmax(1)\n            if tta: p = p + net(xb.flip(3)).float().softmax(1)\n        out.append(p.cpu())\n    return torch.cat(out).numpy()\n\ndef accuracy(net, tta=True):\n    return (predict_probs(net, x_val, tta=tta).argmax(1) == y_val).mean()\n\nX_gpu = torch.from_numpy(x_tr).to(device)                # uint8, ~150 MB\nY_gpu = torch.from_numpy(y_tr).to(device)\nhistory = {\"epoch\": [], \"loss\": [], \"val\": [], \"val_ema\": []}\nstep, t0 = 0, time.time()\n\nfor epoch in range(EPOCHS):\n    model.train(); te = time.time(); run_loss = 0.0\n    use_cutmix_epoch = epoch < EPOCHS - CUTMIX_OFF_LAST\n    perm = torch.randperm(len(X_gpu), device=device)\n    for k in range(steps_per_epoch):\n        idx = perm[k * BATCH_SIZE:(k + 1) * BATCH_SIZE]\n        x = random_crop_flip(normalize(X_gpu[idx])); y = Y_gpu[idx]\n        with torch.autocast(device_type=device, dtype=torch.float16, enabled=use_amp):\n            if use_cutmix_epoch and np.random.rand() < CUTMIX_PROB:\n                x, ya, yb, lam = cutmix(x, y, CUTMIX_ALPHA)\n                out = model(x)\n                loss = lam * criterion(out, ya) + (1 - lam) * criterion(out, yb)\n            else:\n                loss = criterion(model(x), y)\n        optimizer.zero_grad(set_to_none=True)\n        scaler.scale(loss).backward()\n        scaler.step(optimizer); scaler.update(); scheduler.step()\n        ema.update(model, step); step += 1\n        run_loss += loss.item() if k % 20 == 0 else 0.0\n    ep_loss = run_loss / len(range(0, steps_per_epoch, 20))\n\n    if epoch == 0:\n        print(f\"first epoch took {time.time()-te:.0f}s -> full run roughly {(time.time()-te)*EPOCHS/60:.0f} min\")\n    if (epoch + 1) % 10 == 0 or epoch >= EPOCHS - 5:\n        va, ve = accuracy(model, tta=False), accuracy(ema.model, tta=False)\n        history[\"epoch\"].append(epoch + 1); history[\"loss\"].append(ep_loss)\n        history[\"val\"].append(va); history[\"val_ema\"].append(ve)\n        print(f\"epoch {epoch+1:3d}/{EPOCHS} | loss {ep_loss:.3f} | lr {scheduler.get_last_lr()[0]:.2e} \"\n              f\"| val {va:.4f} | val(EMA) {ve:.4f} | {(time.time()-t0)/60:.1f} min\")\n\ntorch.save(ema.model.state_dict(), f\"{OUT_DIR}/dsegnet_ema.pt\")\nprint(f\"training done in {(time.time()-t0)/60:.1f} min\")","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-10-10T16:05:37.39936Z","iopub.execute_input":"2026-10-10T16:05:37.400261Z","iopub.status.idle":"2026-10-10T17:10:03.943527Z","shell.execute_reply.started":"2026-10-10T16:05:37.400221Z","shell.execute_reply":"2026-10-10T17:10:03.942735Z"}},"outputs":[],"execution_count":null},{"id":"a52fc81a","cell_type":"markdown","source":"## 7. Final accuracy estimate (this is the expected Kaggle score)","metadata":{}},{"id":"08b3696a","cell_type":"code","source":"final = ema.model\nacc_plain = accuracy(final, tta=False)\nacc_tta   = accuracy(final, tta=True)\nprint(f\"Validation on {val_source}\")\nprint(f\"  EMA model, no TTA      : {acc_plain:.4f}\")\nprint(f\"  EMA model, flip TTA    : {acc_tta:.4f}   <- expected Kaggle score\")\n\nfig, ax = plt.subplots(1, 2, figsize=(11, 3.5))\nax[0].plot(history[\"epoch\"], history[\"loss\"]); ax[0].set_title(\"training loss\"); ax[0].set_xlabel(\"epoch\")\nax[1].plot(history[\"epoch\"], history[\"val\"], label=\"model\")\nax[1].plot(history[\"epoch\"], history[\"val_ema\"], label=\"EMA\")\nax[1].set_title(\"validation accuracy\"); ax[1].set_xlabel(\"epoch\"); ax[1].legend()\nplt.tight_layout(); plt.savefig(f\"{OUT_DIR}/training_curves.png\", dpi=120); plt.show()","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-10-10T17:10:26.412256Z","iopub.execute_input":"2026-10-10T17:10:26.412548Z","iopub.status.idle":"2026-10-10T17:10:31.720405Z","shell.execute_reply.started":"2026-10-10T17:10:26.412527Z","shell.execute_reply":"2026-10-10T17:10:31.719678Z"}},"outputs":[],"execution_count":null},{"id":"ecfd9174","cell_type":"markdown","source":"## 8. Predict the 300,000-image Kaggle test set and write `submission.csv`","metadata":{}},{"id":"7c62cdeb","cell_type":"code","source":"t = time.time()\npred = predict_probs(final, x_test, bs=2048, tta=True).argmax(1)\nsub = pd.DataFrame({\"id\": np.arange(1, len(x_test) + 1), \"label\": [CLASSES[i] for i in pred]})\nsub.to_csv(f\"{OUT_DIR}/submission.csv\", index=False)\nprint(f\"wrote {len(sub)} rows in {time.time()-t:.0f}s\"); print(sub.head())\nprint(sub[\"label\"].value_counts())","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-10-10T17:11:17.730953Z","iopub.execute_input":"2026-10-10T17:11:17.731561Z","iopub.status.idle":"2026-10-10T17:13:03.172604Z","shell.execute_reply.started":"2026-10-10T17:11:17.731535Z","shell.execute_reply":"2026-10-10T17:13:03.17188Z"}},"outputs":[],"execution_count":null},{"id":"58a4a9d9","cell_type":"markdown","source":"## Submit\n\n1. **Save Version** → **Save & Run All (Commit)**.\n2. When it finishes, open the version → **Output** tab → **Submit to Competition** (shows as a late submission).\n3. Screenshot the score from **My Submissions** for the report.","metadata":{}}]}