{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"name":"python","version":"3.7.6","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# Cassava Leaf Disease: ResNeXt50 training (replication of the 1st place solution)\n\n**Competition:** Cassava Leaf Disease Classification. Each photo shows a cassava leaf, and the task is to predict one of 5 classes: four diseases (CBB, CBSD, CGM, CMD) or healthy. The labels are noisy, which shapes many of the choices below.\n\n**What this notebook is?** \nA replication of the training script for the ResNeXt50 component of the 1st place ensemble. The winning team's own settings are used: `resnext50_32x4d`, 512x512 images, plain cross-entropy, Adam with lr 1e-4, ReduceLROnPlateau on the validation loss, 15 epochs, and the checkpoint with the best validation accuracy.\n\n**Scope (what I trained and what I did not):**\n* Trained by me: The ResNeXt50 model, **fold 0 of 5** only.\n* Not retrained: The ViT, EfficientNetB4 and CropNet models of the full ensemble. In my clone-and-submit notebook I used the authors' released weights for those.\n- Reason: One fold at 512 px takes about 6 hours on a free Kaggle T4 (measured: about 25 minutes per epoch), and the full ensemble has 12 models.\n\n**Changes from the original script:**\n* Pretrained ImageNet weights are loaded from an attached dataset, because the notebook runs offline and cannot download them.\n* Only fold 0 is trained (`trn_fold = [0]`) to fit the GPU quota.\n\n**Results:** See the table in the last cell.","metadata":{}},{"cell_type":"markdown","source":"## 1. Paths\nWhere the competition images live. `OUTPUT_DIR` is the working folder, where the log, the out-of-fold predictions (`oof_df.csv`) and the best weights (`*_fold0_best.pth`) are saved.","metadata":{}},{"cell_type":"code","source":"# ====================================================\n# directory settings\n# ====================================================\n\nimport os\n\nOUTPUT_DIR = './'\nif not os.path.exists(OUTPUT_DIR):\n    os.makedirs(OUTPUT_DIR)\n\nTRAIN_PATH = '../input/cassava-leaf-disease-classification/train_images'\nTEST_PATH = '../input/cassava-leaf-disease-classification/test_images'","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-10-14T09:41:25.088237Z","iopub.execute_input":"2026-10-14T09:41:25.088589Z","iopub.status.idle":"2026-10-14T09:41:25.09506Z","shell.execute_reply.started":"2026-10-14T09:41:25.088551Z","shell.execute_reply":"2026-10-14T09:41:25.093987Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## 2. Configuration (CFG)\nAll the hyperparameters in one place.\n* `model_name = resnext50_32x4d`: ResNeXt50 with 32 groups of width 4.\n*  `size = 512`: Images are resized to 512x512. Larger images keep small lesions visible.\n*  `epochs = 15`, `lr = 1e-4`, `batch_size = 16`: The winner's settings.\n*  `factor / patience / eps`: Settings of the learning-rate scheduler (the rate is multiplied by 0.2 if validation loss does not improve for 5 epochs).\n*  `trn_fold = [0]`: The winner trained folds 0 to 4. I train only fold 0 because of the GPU quota.\n*  `seed = 42`: Fixed for repeatability.","metadata":{}},{"cell_type":"code","source":"# ====================================================\n# CFG\n# ====================================================\n\nclass CFG:\n    print_freq=100\n    num_workers = 4\n    model_name = 'resnext50_32x4d'\n    size = 512\n    epochs = 15\n    factor = 0.2\n    patience = 5\n    eps = 1e-6\n    lr = 1e-4\n    min_lr = 1e-6\n    batch_size = 16\n    weight_decay = 1e-6\n    gradient_accumulation_steps = 1\n    max_grad_norm = 1000\n    seed = 42\n    target_size = 5\n    target_col = 'label'\n    n_fold = 5\n    trn_fold = [0]","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-10-14T09:41:25.096855Z","iopub.execute_input":"2026-10-14T09:41:25.097274Z","iopub.status.idle":"2026-10-14T09:41:25.109147Z","shell.execute_reply.started":"2026-10-14T09:41:25.097235Z","shell.execute_reply":"2026-10-14T09:41:25.108001Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## 3. Libraries\n* `timm` provides the ResNeXt50 architecture. It is loaded from an attached copy of the library (`pytorch-image-models`) because the notebook is offline.\n* `albumentations` does the image augmentation, `cv2` reads images, and `torch` trains the model.","metadata":{}},{"cell_type":"code","source":"# ====================================================\n# libraries\n# ====================================================\n\nimport sys\nsys.path.append('../input/pytorch-image-models/pytorch-image-models-master')\nimport os\nimport math\nimport time\nimport random\nimport shutil\nfrom pathlib import Path\nfrom contextlib import contextmanager\nfrom collections import defaultdict, Counter\nimport scipy as sp\nimport numpy as np\nimport pandas as pd\nfrom sklearn import preprocessing\nfrom sklearn.metrics import accuracy_score\nfrom tqdm.auto import tqdm\nfrom functools import partial\nimport cv2\nfrom PIL import Image\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nfrom torch.optim import Adam, SGD\nimport torchvision.models as models\nfrom torch.nn.parameter import Parameter\nfrom torch.utils.data import DataLoader, Dataset\nfrom torch.optim.lr_scheduler import ReduceLROnPlateau\nfrom albumentations import (Compose, Normalize, Resize, RandomResizedCrop, HorizontalFlip, VerticalFlip, ShiftScaleRotate, Transpose)\nfrom albumentations.pytorch import ToTensorV2\nfrom albumentations import ImageOnlyTransform\nimport timm\nimport warnings \nwarnings.filterwarnings('ignore')\ndevice = torch.device('cuda' if torch.cuda.is_available() else 'cpu')\nfrom matplotlib import pyplot as plt\nimport joblib","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-10-14T09:41:25.111067Z","iopub.execute_input":"2026-10-14T09:41:25.111442Z","iopub.status.idle":"2026-10-14T09:41:32.251351Z","shell.execute_reply.started":"2026-10-14T09:41:25.111412Z","shell.execute_reply":"2026-10-14T09:41:32.250564Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## 4. Utilities\n* `get_score`: The metric, plain accuracy, as in the competition.\n* `init_logger`: Writes progress to the screen and to `train.log`.\n* `seed_torch`: Fixes the random seeds. Note that `cudnn.deterministic = True` makes training repeatable but a little slower.","metadata":{}},{"cell_type":"code","source":"# ====================================================\n# utils\n# ====================================================\n\ndef get_score(y_true, y_pred):\n    return accuracy_score(y_true, y_pred)\n\n@contextmanager\ndef timer(name):\n    t0 = time.time()\n    LOGGER.info(f'[{name}] start')\n    yield\n    LOGGER.info(f'[{name}] done in {time.time() - t0:.0f} s.')\n\ndef init_logger(log_file=OUTPUT_DIR+'train.log'):\n    from logging import getLogger, INFO, FileHandler,  Formatter,  StreamHandler\n    logger = getLogger(__name__)\n    logger.setLevel(INFO)\n    handler1 = StreamHandler()\n    handler1.setFormatter(Formatter(\"%(message)s\"))\n    handler2 = FileHandler(filename=log_file)\n    handler2.setFormatter(Formatter(\"%(message)s\"))\n    logger.addHandler(handler1)\n    logger.addHandler(handler2)\n    return logger\n\nLOGGER = init_logger()\n\ndef seed_torch(seed=42):\n    random.seed(seed)\n    os.environ['PYTHONHASHSEED'] = str(seed)\n    np.random.seed(seed)\n    torch.manual_seed(seed)\n    torch.cuda.manual_seed(seed)\n    torch.backends.cudnn.deterministic = True\n\nseed_torch(seed=CFG.seed)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-10-14T09:41:32.253926Z","iopub.execute_input":"2026-10-14T09:41:32.254333Z","iopub.status.idle":"2026-10-14T09:41:32.267142Z","shell.execute_reply.started":"2026-10-14T09:41:32.254289Z","shell.execute_reply":"2026-10-14T09:41:32.266229Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## 5. Dataset\n`TrainDataset` reads one image from disk, converts it from BGR to RGB, applies the transform, and returns the image with its label. The fold split comes from the winners' `validation_data.csv`, so my validation set is the same as theirs.","metadata":{}},{"cell_type":"code","source":"# ====================================================\n# dataset\n# ====================================================\n\nclass TrainDataset(Dataset):\n    def __init__(self, df, transform=None):\n        self.df = df\n        self.file_names = df['image_id'].values\n        self.labels = df['label'].values\n        self.transform = transform\n        \n    def __len__(self):\n        return len(self.df)\n\n    def __getitem__(self, idx):\n        file_name = self.file_names[idx]\n        file_path = f'{TRAIN_PATH}/{file_name}'\n        image = cv2.imread(file_path)\n        image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)\n        if self.transform:\n            augmented = self.transform(image=image)\n            image = augmented['image']\n        label = torch.tensor(self.labels[idx]).long()\n        return image, label","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-10-14T09:41:32.268876Z","iopub.execute_input":"2026-10-14T09:41:32.269283Z","iopub.status.idle":"2026-10-14T09:41:32.279009Z","shell.execute_reply.started":"2026-10-14T09:41:32.269247Z","shell.execute_reply":"2026-10-14T09:41:32.27809Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## 6. Augmentation\n**Training:** Random resized crop, transpose, horizontal and vertical flips, and small shift/scale/rotate, then normalisation with ImageNet statistics. Leaf photos have no fixed orientation, so flips and transposes create realistic new views.\n\n**Validation:** Only resize and normalise, so the score is not affected by randomness. The winners used the same for inference.","metadata":{}},{"cell_type":"code","source":"# ====================================================\n# transformations\n# ====================================================\n\ndef get_transforms(*, data):\n    \n    if data == 'train':\n        return Compose([\n            RandomResizedCrop(CFG.size, CFG.size),\n            Transpose(p=0.5),\n            HorizontalFlip(p=0.5),\n            VerticalFlip(p=0.5),\n            ShiftScaleRotate(p=0.5),\n            Normalize(\n                mean=[0.485, 0.456, 0.406],\n                std=[0.229, 0.224, 0.225],\n            ),\n            ToTensorV2(),\n        ])\n    \n    elif data == 'valid':\n       return Compose([\n           Resize(CFG.size, CFG.size),\n           Normalize(\n               mean=[0.485, 0.456, 0.406],\n               std=[0.229, 0.224, 0.225],\n           ),\n           ToTensorV2(),\n       ])","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-10-14T09:41:32.280152Z","iopub.execute_input":"2026-10-14T09:41:32.280456Z","iopub.status.idle":"2026-10-14T09:41:32.29061Z","shell.execute_reply.started":"2026-10-14T09:41:32.280402Z","shell.execute_reply":"2026-10-14T09:41:32.289791Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## 7. Model: ResNeXt50 with offline pretrained weights\n`CustomResNext` builds `resnext50_32x4d` from timm, loads ImageNet-pretrained weights from a local file, and replaces the final 1000-class layer with a 5-class layer.\n\n**Engineering challenge:** The original code downloads the pretrained weights automatically, but Kaggle's offline mode blocks downloads. I downloaded the `.pth` file separately, uploaded it as a dataset, and loaded it with `load_state_dict`. The following cells check that the file exists and that the model outputs a (1, 5) tensor, so a wrong path fails early and not an hour into training.","metadata":{}},{"cell_type":"code","source":"# ====================================================\n# MODEL INITIALIZATION - LOAD PRETRAINED WEIGHTS LOCALLY\n# ====================================================\n\nimport os\nimport torch\nimport torch.nn as nn\n\nPRETRAINED_PATH = (\n    \"/kaggle/input/datasets/ananyaamadhursharma/\"\n    \"resnext50-pretrained-weights/pretrained_weights/\"\n    \"resnext50_32x4d_ra-d733960d.pth\"\n)\n\nclass CustomResNext(nn.Module):\n\n    def __init__(\n        self,\n        model_name=\"resnext50_32x4d\",\n        pretrained=False\n    ):\n        super().__init__()\n\n        # Create the model without downloading anything.\n        self.model = timm.create_model(\n            model_name,\n            pretrained=False\n        )\n\n        # Load pretrained ImageNet weights from Kaggle.\n        if pretrained:\n\n            if not os.path.isfile(PRETRAINED_PATH):\n                raise FileNotFoundError(\n                    \"Weights file not found at: \"\n                    + PRETRAINED_PATH\n                )\n\n            checkpoint = torch.load(\n                PRETRAINED_PATH,\n                map_location=\"cpu\"\n            )\n\n            # Handle common checkpoint formats.\n            if isinstance(checkpoint, dict):\n                for key in (\"state_dict\", \"model_state_dict\"):\n                    if key in checkpoint and isinstance(\n                        checkpoint[key], dict\n                    ):\n                        checkpoint = checkpoint[key]\n                        break\n\n            # Remove the prefix if saved using DataParallel.\n            if (\n                isinstance(checkpoint, dict)\n                and len(checkpoint) > 0\n                and all(\n                    key.startswith(\"module.\")\n                    for key in checkpoint.keys()\n                )\n            ):\n                checkpoint = {\n                    key[len(\"module.\"):]: value\n                    for key, value in checkpoint.items()\n                }\n\n            # Load the pretrained weights.\n            self.model.load_state_dict(\n                checkpoint,\n                strict=True\n            )\n\n            print(\"Pretrained ResNeXt50 weights loaded successfully.\")\n\n        # Replace the 1000-class ImageNet head with 5 classes.\n        n_features = self.model.fc.in_features\n\n        self.model.fc = nn.Linear(\n            n_features,\n            CFG.target_size\n        )\n\n    def forward(self, x):\n        return self.model(x)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-10-14T09:41:32.292013Z","iopub.execute_input":"2026-10-14T09:41:32.292427Z","iopub.status.idle":"2026-10-14T09:41:32.307925Z","shell.execute_reply.started":"2026-10-14T09:41:32.292398Z","shell.execute_reply":"2026-10-14T09:41:32.30655Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"import os\n\nprint(\"Checking pretrained weights...\")\n\nprint(\"File exists:\", os.path.isfile(PRETRAINED_PATH))\nprint(\"Path:\", PRETRAINED_PATH)\n\nif os.path.isfile(PRETRAINED_PATH):\n    print(\n        \"File size (MB):\",\n        round(os.path.getsize(PRETRAINED_PATH) / (1024 * 1024), 2)\n    )\nelse:\n    print(\"\\nSearching attached Kaggle inputs...\")\n\n    for root, dirs, files in os.walk(\"/kaggle/input\"):\n        for filename in files:\n            if filename == \"resnext50_32x4d_ra-d733960d.pth\":\n                print(\"Found file:\", os.path.join(root, filename))","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-10-14T09:41:32.309364Z","iopub.execute_input":"2026-10-14T09:41:32.309755Z","iopub.status.idle":"2026-10-14T09:41:32.333256Z","shell.execute_reply.started":"2026-10-14T09:41:32.309717Z","shell.execute_reply":"2026-10-14T09:41:32.332376Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# ====================================================\n# TEST MODEL INITIALIZATION\n# ====================================================\n\nassert os.path.isfile(PRETRAINED_PATH), (\n    \"Pretrained weights file is missing.\"\n)\n\ntest_model = CustomResNext(\n    CFG.model_name,\n    pretrained=True\n).to(device)\n\ntest_model.eval()\n\nwith torch.no_grad():\n    test_input = torch.randn(\n        1, 3, CFG.size, CFG.size\n    ).to(device)\n\n    test_output = test_model(test_input)\n\nprint(\"Device:\", device)\nprint(\"Input shape:\", tuple(test_input.shape))\nprint(\"Output shape:\", tuple(test_output.shape))\nprint(\"Expected output shape:\", (1, CFG.target_size))\n\nassert test_output.shape == (1, CFG.target_size)\n\nprint(\"MODEL TEST PASSED\")","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-10-14T09:41:32.335322Z","iopub.execute_input":"2026-10-14T09:41:32.335626Z","iopub.status.idle":"2026-10-14T09:41:41.176551Z","shell.execute_reply.started":"2026-10-14T09:41:32.335598Z","shell.execute_reply":"2026-10-14T09:41:41.175679Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## 8. Training and validation functions\n* `train_fn`: One pass over the training set. Forward pass, cross-entropy loss, backward pass, gradient clipping, optimizer step.\n* `valid_fn`: One pass over the validation fold with no gradients. It returns the loss and the softmax probabilities.\n* Note: `gradient_accumulation_steps = 1`, so every batch updates the weights. The accumulation branch in this code (inherited from the original) would not train correctly if it were set above 1, so I left it at 1.","metadata":{}},{"cell_type":"code","source":"# ====================================================\n# helper functions\n# ====================================================\n\nclass AverageMeter(object):\n    \"\"\"Computes and stores the average and current value\"\"\"\n    def __init__(self):\n        self.reset()\n\n    def reset(self):\n        self.val = 0\n        self.avg = 0\n        self.sum = 0\n        self.count = 0\n\n    def update(self, val, n=1):\n        self.val = val\n        self.sum += val * n\n        self.count += n\n        self.avg = self.sum / self.count\n\n\ndef asMinutes(s):\n    m = math.floor(s / 60)\n    s -= m * 60\n    return '%dm %ds' % (m, s)\n\n\ndef timeSince(since, percent):\n    now = time.time()\n    s = now - since\n    es = s / (percent)\n    rs = es - s\n    return '%s (remain %s)' % (asMinutes(s), asMinutes(rs))\n\n\ndef train_fn(train_loader, model, criterion, optimizer, epoch, scheduler, device):\n    batch_time = AverageMeter()\n    data_time = AverageMeter()\n    losses = AverageMeter()\n    scores = AverageMeter()\n    model.train()\n    start = end = time.time()\n    global_step = 0\n    for step, (images, labels) in enumerate(train_loader):\n        data_time.update(time.time() - end)\n        images = images.to(device)\n        labels = labels.to(device)\n        batch_size = labels.size(0)\n        y_preds = model(images)\n        loss = criterion(y_preds, labels)\n        losses.update(loss.item(), batch_size)\n        if CFG.gradient_accumulation_steps > 1:\n            loss = loss / CFG.gradient_accumulation_steps\n        else:\n            loss.backward()\n        grad_norm = torch.nn.utils.clip_grad_norm_(model.parameters(), CFG.max_grad_norm)\n        if (step + 1) % CFG.gradient_accumulation_steps == 0:\n            optimizer.step()\n            optimizer.zero_grad()\n            global_step += 1\n        batch_time.update(time.time() - end)\n        end = time.time()\n        if step % CFG.print_freq == 0 or step == (len(train_loader)-1):\n            print('Epoch: [{0}][{1}/{2}] '\n                  'Data {data_time.val:.3f} ({data_time.avg:.3f}) '\n                  'Elapsed {remain:s} '\n                  'Loss: {loss.val:.4f}({loss.avg:.4f}) '\n                  'Grad: {grad_norm:.4f}  '\n                  .format(\n                   epoch+1, step, len(train_loader), batch_time=batch_time,\n                   data_time=data_time, loss=losses,\n                   remain=timeSince(start, float(step+1)/len(train_loader)),\n                   grad_norm=grad_norm,\n                   ))\n    return losses.avg\n\ndef valid_fn(valid_loader, model, criterion, device):\n    batch_time = AverageMeter()\n    data_time = AverageMeter()\n    losses = AverageMeter()\n    scores = AverageMeter()\n    model.eval()\n    preds = []\n    start = end = time.time()\n    for step, (images, labels) in enumerate(valid_loader):\n        data_time.update(time.time() - end)\n        images = images.to(device)\n        labels = labels.to(device)\n        batch_size = labels.size(0)\n        with torch.no_grad():\n            y_preds = model(images)\n        loss = criterion(y_preds, labels)\n        losses.update(loss.item(), batch_size)\n        preds.append(y_preds.softmax(1).to('cpu').numpy())\n        if CFG.gradient_accumulation_steps > 1:\n            loss = loss / CFG.gradient_accumulation_steps\n        batch_time.update(time.time() - end)\n        end = time.time()\n        if step % CFG.print_freq == 0 or step == (len(valid_loader)-1):\n            print('EVAL: [{0}/{1}] '\n                  'Data {data_time.val:.3f} ({data_time.avg:.3f}) '\n                  'Elapsed {remain:s} '\n                  'Loss: {loss.val:.4f}({loss.avg:.4f}) '\n                  .format(\n                   step, len(valid_loader), batch_time=batch_time,\n                   data_time=data_time, loss=losses,\n                   remain=timeSince(start, float(step+1)/len(valid_loader)),\n                   ))\n    predictions = np.concatenate(preds)\n    return losses.avg, predictions","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-10-14T09:41:41.178148Z","iopub.execute_input":"2026-10-14T09:41:41.178455Z","iopub.status.idle":"2026-10-14T09:41:41.202216Z","shell.execute_reply.started":"2026-10-14T09:41:41.178426Z","shell.execute_reply":"2026-10-14T09:41:41.20119Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## 9. Training loop\nFor each epoch: train, validate, step the scheduler on the validation loss, compute accuracy, and **save the weights only when validation accuracy improves**. After the last epoch the best checkpoint is reloaded, which is how the winners chose their final model. The saved file also contains the validation predictions, which become `oof_df.csv`.","metadata":{}},{"cell_type":"code","source":"# ====================================================\n# train loop\n# ====================================================\n\ndef train_loop(folds, fold):\n\n    LOGGER.info(f\"========== fold: {fold} training ==========\")\n\n    trn_idx = folds[folds['fold'] != fold].index\n    val_idx = folds[folds['fold'] == fold].index\n\n    train_folds = folds.loc[trn_idx].reset_index(drop=True)\n    valid_folds = folds.loc[val_idx].reset_index(drop=True)\n\n    train_dataset = TrainDataset(train_folds, transform=get_transforms(data='train'))\n    valid_dataset = TrainDataset(valid_folds, transform=get_transforms(data='valid'))\n\n    train_loader = DataLoader(train_dataset, batch_size=CFG.batch_size, \n                              shuffle=True, num_workers=CFG.num_workers, pin_memory=True, drop_last=True)\n    valid_loader = DataLoader(valid_dataset, batch_size=CFG.batch_size, \n                              shuffle=False, num_workers=CFG.num_workers, pin_memory=True, drop_last=False)\n    \n    model = CustomResNext(CFG.model_name, pretrained=True)\n    model.to(device)\n\n    optimizer = Adam(model.parameters(), lr=CFG.lr, weight_decay=CFG.weight_decay, amsgrad=False)\n    scheduler = ReduceLROnPlateau(optimizer, mode='min', factor=CFG.factor, patience=CFG.patience, verbose=True, eps=CFG.eps)\n  \n    criterion = nn.CrossEntropyLoss()\n\n    best_score = 0.\n    best_loss = np.inf\n    \n    for epoch in range(CFG.epochs):\n        start_time = time.time()\n        avg_loss = train_fn(train_loader, model, criterion, optimizer, epoch, scheduler, device)\n        avg_val_loss, preds = valid_fn(valid_loader, model, criterion, device)\n        valid_labels = valid_folds[CFG.target_col].values\n        scheduler.step(avg_val_loss)\n        score = get_score(valid_labels, preds.argmax(1))\n        elapsed = time.time() - start_time\n        LOGGER.info(f'Epoch {epoch+1} - avg_train_loss: {avg_loss:.4f}  avg_val_loss: {avg_val_loss:.4f}  time: {elapsed:.0f}s')\n        LOGGER.info(f'Epoch {epoch+1} - Accuracy: {score}')\n        if score > best_score:\n            best_score = score\n            LOGGER.info(f'Epoch {epoch+1} - Save Best Score: {best_score:.4f} Model')\n            torch.save({'model': model.state_dict(), 'preds': preds}, OUTPUT_DIR+f'{CFG.model_name}_fold{fold}_best.pth')\n    \n    check_point = torch.load(OUTPUT_DIR+f'{CFG.model_name}_fold{fold}_best.pth')\n    valid_folds[[str(c) for c in range(5)]] = check_point['preds']\n    valid_folds['preds'] = check_point['preds'].argmax(1)\n\n    return valid_folds","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true,"execution":{"iopub.status.busy":"2026-10-14T09:41:41.204805Z","iopub.execute_input":"2026-10-14T09:41:41.205205Z","iopub.status.idle":"2026-10-14T09:41:41.222525Z","shell.execute_reply.started":"2026-10-14T09:41:41.205175Z","shell.execute_reply":"2026-10-14T09:41:41.221289Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## 10. Run\nTrains the folds listed in `trn_fold` and prints the fold and overall accuracy. In my runs only fold 0 is trained, so the \"CV\" score printed at the end is the fold 0 score.","metadata":{}},{"cell_type":"code","source":"# ====================================================\n# main function\n# ====================================================\n\ndef main():\n\n    def get_result(result_df):\n        preds = result_df['preds'].values\n        labels = result_df[CFG.target_col].values\n        score = get_score(labels, preds)\n        LOGGER.info(f'Score: {score:<.5f}')\n\n    oof_df = pd.DataFrame()\n\n    for fold in CFG.trn_fold:\n        _oof_df = train_loop(folds, fold)\n        oof_df = pd.concat([oof_df, _oof_df])\n\n        LOGGER.info(f\"========== fold: {fold} result ==========\")\n        get_result(_oof_df)\n\n    LOGGER.info(f\"========== CV ==========\")\n    get_result(oof_df)\n\n    oof_df.to_csv(OUTPUT_DIR + 'oof_df.csv', index=False)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-10-14T09:41:41.224244Z","iopub.execute_input":"2026-10-14T09:41:41.224634Z","iopub.status.idle":"2026-10-14T09:41:41.247598Z","shell.execute_reply.started":"2026-10-14T09:41:41.224592Z","shell.execute_reply":"2026-10-14T09:41:41.246648Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# Load training data\ntrain = pd.read_csv('../input/cassava-leaf-disease-classification/train.csv')\n# Split into folds for cross validation - we used the same split for all the models we trained!\nfolds = train.merge(\n    pd.read_csv(\"../input/cassava-leaf-disease-resnext/validation_data.csv\")[[\"image_id\", \"fold\"]], on=\"image_id\")\n\nif __name__ == '__main__':\n    main()","metadata":{"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","trusted":true,"execution":{"iopub.status.busy":"2026-10-14T09:41:41.248848Z","iopub.execute_input":"2026-10-14T09:41:41.24917Z","iopub.status.idle":"2026-10-14T10:08:27.422325Z","shell.execute_reply.started":"2026-10-14T09:41:41.249146Z","shell.execute_reply":"2026-10-14T10:08:27.421146Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## Results\n\n| Run | Image size | Epochs | Best fold 0 accuracy | Time per epoch |\n|---|---|---|---|---|\n| This notebook (faithful) | 512 | 15 | see the `Score:` | see the `time:` values in the log |\n| Earlier reduced run | 384 | 8 | 0.87388 | about 775 s |\n\nCompared with the winners: they trained 5 folds and ensembled many models, so my single-fold result is expected to be lower than their ensemble score.","metadata":{}}]}