{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"name":"python","version":"3.11.13","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"},"kaggle":{"accelerator":"gpu","dataSources":[{"sourceId":112509,"databundleVersionId":14254895,"sourceType":"competition"}],"dockerImageVersionId":31192,"isInternetEnabled":true,"language":"python","sourceType":"notebook","isGpuEnabled":true}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# 🌱 Understanding the Problem: Pasture Biomass from Images\n\nBefore writing any code, let’s step back and clearly understand **what we are trying to predict**, **how the data is organized**, and **how everything fits together**.\n\n---\n## 📌 What is the task?\n\nAt its core, this competition is about **looking at an image of pasture and estimating how much usable feed is present**.\n\nGiven a single pasture image, our model must **predict numerical values (in grams)** for **five different biomass components**.  \nThis makes the problem a:\n\n- **Computer vision task** (input = RGB images)\n- **Regression problem** (output = real-valued numbers)\n- **Multi-output prediction** (5 values per image)\n\nWe are *not* classifying images.  \nWe are *not* detecting objects.  \nWe are learning to **estimate quantities from visual patterns** like color, density, texture, and structure of grass.\n\n---\n\n## 📌 What are we predicting?\n\nEach image is associated with **five different biomass components**, all measured in grams:\n\n- **Dry_Green_g**  \n  Dry mass of green vegetation, excluding clover. This reflects actively growing pasture that animals can graze.\n\n- **Dry_Dead_g**  \n  Dry mass of dead plant material. High values often indicate older or less nutritious pasture.\n\n- **Dry_Clover_g**  \n  Dry mass of clover present in the pasture. Clover is important because it improves feed quality and soil nitrogen.\n\n- **GDM_g (Green Dry Matter)**  \n  The dry mass of green vegetation. This is a key indicator of how much fresh, usable feed is available.\n\n- **Dry_Total_g**  \n  Total dry biomass, combining all components. This is the most important target for overall grazing decisions.\n\nAlthough these values are related, they capture **different aspects of pasture condition**, so the model must learn more than just “how dense the grass looks”.\n\n---\n\n## 📌 How is the data organized?\n\nAt first glance, the dataset can feel confusing.\n\nEach image appears **five times** in the CSV files — once for each biomass component.  \nSo instead of one row per image, the raw data is stored in a **long format**.\n\nHowever, when training a neural network, this is **not the most convenient structure**.\n\nDuring training:\n- We want to **load each image only once**\n- And predict **all five biomass values at the same time**\n\nTo do this, we reshape the data so that:\n- One row corresponds to **one image**\n- The target becomes a **5-element vector**\n\nThis allows the model to:\n- See the image once per iteration\n- Learn the relationships between the five biomass components\n- Train more efficiently and correctly\n\nUnderstanding this restructuring step is **crucial** for both training and inference.\n\n---\n\n##  📌  Big picture: our approach\n\nIn this notebook, we follow a practical deep learning pipeline:\n\n- Use a **pretrained ConvNeXt model** to extract visual features\n- Replace the classification head with a **regression head** (5 outputs)\n- Apply image augmentations to improve generalization\n- Train using **K-Fold cross-validation**\n- Evaluate using the official weighted R² metric\n- Save the best model from each fold for final predictions\n\nThe goal is to keep the solution **clear and reproducible**\n\n---\n\n🎯 *With the problem clearly defined, let’s move on to exploring the dataset itself.*\n","metadata":{}},{"cell_type":"markdown","source":"# 1.  Imports and setup\n\nThis cell contains all the required libraries for the notebook,\n","metadata":{}},{"cell_type":"code","source":"import warnings\nwarnings.filterwarnings(\"ignore\")\n\nimport os\nimport random\nimport numpy as np\nimport pandas as pd\nfrom PIL import Image\n\nimport torch\nimport torch.nn as nn\nfrom torch.utils.data import Dataset, DataLoader, Subset\n\nimport timm\nfrom torchvision import transforms\nfrom sklearn.model_selection import KFold\nfrom torch.cuda.amp import autocast, GradScaler\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-12-22T20:40:41.044699Z","iopub.execute_input":"2025-12-22T20:40:41.045175Z","iopub.status.idle":"2025-12-22T20:40:41.050753Z","shell.execute_reply.started":"2025-12-22T20:40:41.045159Z","shell.execute_reply":"2025-12-22T20:40:41.049716Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"# 2. Configuration and evaluation metric\n\nThis section defines configures are the variables that we will be using through the notebook and also defines function for our evaluation metric\n","metadata":{}},{"cell_type":"code","source":"class Config:\n    train_path = \"/kaggle/input/csiro-biomass/train.csv\"\n    test_path = \"/kaggle/input/csiro-biomass/test\"\n    sample_sub = \"/kaggle/input/csiro-biomass/sample_submission.csv\"\n    root_dir = \"/kaggle/input/csiro-biomass\"\n    sample_id = \"ID1001187975__Dry_Clover_g\"\n    TARGET_NAMES = [\n        \"Dry_Green_g\",\n        \"Dry_Dead_g\",\n        \"Dry_Clover_g\",\n        \"GDM_g\",\n        \"Dry_Total_g\",\n    ]\n\n\nconfig = Config()\n\n\ndef weighted_r2_score(y_true, y_pred):\n        weights = np.array([0.1, 0.1, 0.1, 0.2, 0.5])\n        r2s = []\n    \n        for i in range(5):\n            yt, yp = y_true[:, i], y_pred[:, i]\n            ss_res = np.sum((yt - yp) ** 2)\n            ss_tot = np.sum((yt - yt.mean()) ** 2)\n            r2 = 1 - ss_res / ss_tot if ss_tot > 0 else 0\n            r2s.append(r2)\n    \n        r2s = np.array(r2s)\n        return np.sum(r2s * weights) / weights.sum(), r2s\n\n\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-12-22T20:40:41.238651Z","iopub.execute_input":"2025-12-22T20:40:41.238921Z","iopub.status.idle":"2025-12-22T20:40:41.246593Z","shell.execute_reply.started":"2025-12-22T20:40:41.238900Z","shell.execute_reply":"2025-12-22T20:40:41.244856Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"# 3. Load the training data\n\nWe start by loading the training CSV file into a Pandas DataFrame.  \n\n","metadata":{}},{"cell_type":"code","source":"train_df = pd.read_csv(config.train_path)\n\ntrain_df.head(5)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-12-22T20:40:41.248061Z","iopub.execute_input":"2025-12-22T20:40:41.248270Z","iopub.status.idle":"2025-12-22T20:40:41.330616Z","shell.execute_reply.started":"2025-12-22T20:40:41.248250Z","shell.execute_reply":"2025-12-22T20:40:41.329446Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## 4. Reshape the training DataFrame\n\nSince each image has five rows (one per biomass target), we pivot the DataFrame so that **each image has a single row** with all five targets as separate columns.  \n\nThis makes it convenient to feed one image into the model and predict all targets at once.\n","metadata":{}},{"cell_type":"code","source":"df = train_df.pivot(\n    index=\"image_path\",\n    columns=\"target_name\", \n    values=\"target\"\n).reset_index()\n\ndf.columns.name = None \n\ndf.head(5)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-12-22T20:40:41.427490Z","iopub.execute_input":"2025-12-22T20:40:41.427771Z","iopub.status.idle":"2025-12-22T20:40:41.489514Z","shell.execute_reply.started":"2025-12-22T20:40:41.427753Z","shell.execute_reply":"2025-12-22T20:40:41.488419Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"# 5. Create a custom Dataset\n\nWe define a PyTorch Dataset to load images and their corresponding **five biomass targets**.  \n\n","metadata":{}},{"cell_type":"code","source":"class BiomassDataset(Dataset):\n    def __init__(self, df, root_dir, transform):\n        self.df = df.reset_index(drop=True)\n        self.root_dir = root_dir\n        self.transform = transform\n\n    def __len__(self):\n        return len(self.df)\n\n    def __getitem__(self, idx):\n        row = self.df.iloc[idx]\n        img = Image.open(os.path.join(self.root_dir, row[\"image_path\"])).convert(\"RGB\")\n        img = self.transform(img)\n\n        target = torch.tensor([\n            row[\"Dry_Green_g\"],\n            row[\"Dry_Dead_g\"],\n            row[\"Dry_Clover_g\"],\n            row[\"GDM_g\"],\n            row[\"Dry_Total_g\"]\n        ], dtype=torch.float32)\n\n        return img, target\n\n\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-12-22T20:40:41.490774Z","iopub.execute_input":"2025-12-22T20:40:41.491042Z","iopub.status.idle":"2025-12-22T20:40:41.498881Z","shell.execute_reply.started":"2025-12-22T20:40:41.491024Z","shell.execute_reply":"2025-12-22T20:40:41.496874Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"# 6. Model Architecture: ConvNeXt Regressor\n\nWe use a pre-trained ConvNeXt Tiny backbone to extract features from pasture images, followed by global pooling and a linear layer to predict all five biomass targets simultaneously. This allows the model to capture visual patterns relevant for biomass estimation efficiently.\n","metadata":{}},{"cell_type":"code","source":"class ConvNeXtRegressor(nn.Module):\n    def __init__(self, num_targets=5):\n        super().__init__()\n\n        self.backbone = timm.create_model(\n            \"convnext_tiny\",\n            pretrained=True,\n            features_only=False\n        )\n        self.backbone.head = nn.Identity()\n\n        self.pool = nn.AdaptiveAvgPool2d(1)\n        self.head = nn.Linear(768, num_targets)\n\n    def forward(self, x):\n        x = self.backbone.forward_features(x)\n        x = self.pool(x).flatten(1)\n        return self.head(x)\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-12-22T20:40:41.610229Z","iopub.execute_input":"2025-12-22T20:40:41.610510Z","iopub.status.idle":"2025-12-22T20:40:41.617632Z","shell.execute_reply.started":"2025-12-22T20:40:41.610489Z","shell.execute_reply":"2025-12-22T20:40:41.615886Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"# 7. Training and Validation\n\nWe define functions for one training epoch and for validation. Training uses mixed-precision to speed up computation and reduce memory usage. Validation computes the loss and the weighted R² score to track model performance on all five biomass targets simultaneously.\n","metadata":{}},{"cell_type":"code","source":"def train_one_epoch(model, loader, optimizer, criterion, scaler, device):\n    model.train()\n    total_loss = 0\n\n    for x, y in loader:\n        x, y = x.to(device), y.to(device)\n        optimizer.zero_grad()\n\n        with autocast():\n            preds = model(x)\n            loss = criterion(preds, y)\n\n        scaler.scale(loss).backward()\n        scaler.step(optimizer)\n        scaler.update()\n\n        total_loss += loss.item() * x.size(0)\n\n    return total_loss / len(loader.dataset)\n\n\ndef validate(model, loader, criterion, device):\n    model.eval()\n    total_loss = 0\n    preds, trues = [], []\n\n    with torch.no_grad():\n        for x, y in loader:\n            x, y = x.to(device), y.to(device)\n            p = model(x)\n\n            total_loss += criterion(p, y).item() * x.size(0)\n            preds.append(p.cpu())\n            trues.append(y.cpu())\n\n    preds = torch.cat(preds).numpy()\n    trues = torch.cat(trues).numpy()\n\n    wr2, _ = weighted_r2_score(trues, preds)\n    return total_loss / len(loader.dataset), wr2\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-12-22T20:40:41.639313Z","iopub.execute_input":"2025-12-22T20:40:41.639651Z","iopub.status.idle":"2025-12-22T20:40:41.648065Z","shell.execute_reply.started":"2025-12-22T20:40:41.639633Z","shell.execute_reply":"2025-12-22T20:40:41.647008Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"# 8. Training Loop\n\nWe use 5-fold cross-validation to train and evaluate our model. Images are augmented for training to improve generalization, while validation images are only resized and normalized. Each fold trains the ConvNeXt model with Smooth L1 loss, AdamW optimizer, and a cosine annealing scheduler. We track weighted R² on the validation set and save the best model per fold.\n","metadata":{}},{"cell_type":"code","source":"device = \"cuda\" if torch.cuda.is_available() else \"cpu\"\n\ntrain_tfms = transforms.Compose([\n    transforms.Resize((224, 224)),\n    transforms.RandomHorizontalFlip(),\n    transforms.RandomVerticalFlip(),\n    transforms.ColorJitter(0.1, 0.1, 0.1, 0.05),\n    transforms.ToTensor(),\n    transforms.Normalize([0.485, 0.456, 0.406],\n                         [0.229, 0.224, 0.225])\n])\n\nval_tfms = transforms.Compose([\n    transforms.Resize((224, 224)),\n    transforms.ToTensor(),\n    transforms.Normalize([0.485, 0.456, 0.406],\n                         [0.229, 0.224, 0.225])\n])\n\n\nkf = KFold(n_splits=5, shuffle=True, random_state=42)\n\nfor fold, (tr_idx, va_idx) in enumerate(kf.split(df)):\n    print(f\"\\n===== Fold {fold+1} =====\")\n\n    train_ds = BiomassDataset(df.iloc[tr_idx], config.root_dir, train_tfms)\n    val_ds   = BiomassDataset(df.iloc[va_idx], config.root_dir, val_tfms)\n\n    train_loader = DataLoader(train_ds, batch_size=8, shuffle=True, num_workers=2)\n    val_loader   = DataLoader(val_ds, batch_size=8, shuffle=False, num_workers=2)\n\n    model = ConvNeXtRegressor().to(device)\n    criterion = nn.SmoothL1Loss()\n    optimizer = torch.optim.AdamW(model.parameters(), lr=1e-4)\n    scheduler = torch.optim.lr_scheduler.CosineAnnealingLR(optimizer, T_max=10)\n    scaler = GradScaler()\n\n    best_r2 = -1\n\n    for epoch in range(10):\n        train_loss = train_one_epoch(model, train_loader, optimizer, criterion, scaler, device)\n        val_loss, val_r2 = validate(model, val_loader, criterion, device)\n        scheduler.step()\n\n        print(f\"Epoch {epoch+1}: Train {train_loss:.4f} | Val {val_loss:.4f} | R2 {val_r2:.4f}\")\n\n        if val_r2 > best_r2:\n            best_r2 = val_r2\n            torch.save(model.state_dict(), f\"best_convnext_fold{fold}.pth\")\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-12-22T20:40:41.795873Z","iopub.execute_input":"2025-12-22T20:40:41.796484Z","execution_failed":"2025-12-22T20:41:45.126Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"# 🎬 Conclusion\n\nThis notebook demonstrates how to train a ConvNeXt-based model for predicting pasture biomass across five key components. Remember, this notebook is **only for training**. For running inference and generating submission files, check out the [Inference Notebook](https://www.kaggle.com/code/chikonzeroselemani/csir0-gettingstarted-inference) with a ready-to-use workflow.  \n\nIf you found this notebook helpful, consider giving it an **upvote** 🌱\n","metadata":{}}]}