{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"name":"python","version":"3.12.12","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"},"kaggle":{"accelerator":"none","dataSources":[{"sourceId":112509,"databundleVersionId":14254895,"sourceType":"competition"},{"sourceId":14348906,"sourceType":"datasetVersion","datasetId":9161955}],"dockerImageVersionId":31234,"isInternetEnabled":false,"language":"python","sourceType":"notebook","isGpuEnabled":false}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"\n

\n🌱 Understanding the Problem: Pasture Biomass from Images\n

\n\n
\n\n

\n Before writing any code, it’s important to step back and clearly understand what this problem is about, \n what we are trying to predict, and how the data is structured. This forms the foundation for every design \n choice we make later.\n

\n\n
\n

\n πŸ“Œ What is the task?\n

\n

\n At its core, this competition asks us to look at an image of pasture and estimate how much usable feed is present.\n

\n \n

\n This makes the problem a computer vision regression task with \n multiple outputs. We are not classifying images or detecting objects β€” \n we are learning to estimate quantities from visual patterns like color, texture, and density.\n

\n
\n\n
\n

\n πŸ“Œ What are we predicting?\n

\n\n\n \n \n \n \n\n\n \n \n \n\n\n \n \n \n\n\n \n \n \n\n\n \n \n \n
\n Dry_Green_g\n \n Dry mass of green vegetation (excluding clover). Reflects actively growing pasture available for grazing.\n
\n Dry_Dead_g\n \n Dry mass of dead plant material. Higher values often indicate older or less nutritious pasture.\n
\n Dry_Clover_g\n \n Dry mass of clover in the pasture. Clover improves feed quality and contributes to soil nitrogen.\n
\n GDM_g\n \n Green Dry Matter β€” a key indicator of fresh, usable feed available to livestock.\n
\n Dry_Total_g\n \n Total dry biomass combining all components. This summarizes overall pasture availability.\n
\n
\n\n
\n

\n πŸ“Œ How is the data organized?\n

\n

\n Each image appears multiple times in the raw CSV files β€” once for each biomass component.\n So instead of one row per image, the data is stored in a long format.\n

\n

\n For modeling, this is reshaped so that:\n

\n \n

\n This restructuring allows the model to see each image once and learn the relationships \n between the different biomass components.\n

\n
\n\n

\n πŸ“Œ With a clear understanding of the task and data, we can now move forward and build a model \n that learns to estimate pasture biomass directly from images.\n

\n\n
\n","metadata":{}},{"cell_type":"markdown","source":"\n

\n🧠 Approach\n

\n\n
\n\n

\n The pipeline for predicting pasture biomass is designed to leverage visual patterns effectively while providing accurate multi-target regression:\n

\n\n \n\n\n
\n","metadata":{}},{"cell_type":"markdown","source":"\n

\n 1. Importing Libraries\n

\n\n
\n

In this section, we import all the essential libraries

\n \n
","metadata":{}},{"cell_type":"code","source":"import warnings\nwarnings.filterwarnings(\"ignore\")\n\nimport os\nimport random\nimport numpy as np\nimport pandas as pd\nfrom PIL import Image\nfrom tqdm import tqdm\nimport matplotlib.pyplot as plt\n\nimport torch\nimport torch.nn as nn\nfrom torch.utils.data import Dataset, DataLoader, Subset\n\nimport timm\nfrom torchvision import transforms\nfrom sklearn.model_selection import KFold\nfrom torch.cuda.amp import autocast, GradScaler\n\ndevice = \"cuda\" if torch.cuda.is_available() else \"cpu\"","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-12-31T00:02:25.573687Z","iopub.execute_input":"2025-12-31T00:02:25.574327Z","iopub.status.idle":"2025-12-31T00:02:43.210372Z","shell.execute_reply.started":"2025-12-31T00:02:25.574293Z","shell.execute_reply":"2025-12-31T00:02:43.209281Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"\n

\n 2. Configuration and evaluation metric\n\n

\n\n
\n

\nThis section defines configures are the variables that we will be using through out the notebook and also defines function for our evaluation metric

\n \n
","metadata":{}},{"cell_type":"code","source":"class Config:\n train_path = \"/kaggle/input/csiro-biomass/train.csv\"\n test_path = \"/kaggle/input/csiro-biomass/test\"\n sample_sub = \"/kaggle/input/csiro-biomass/sample_submission.csv\"\n root_dir = \"/kaggle/input/csiro-biomass\"\n sample_id = \"ID1001187975__Dry_Clover_g\"\n\n NUM_EPOCHS = 50\n PATIENCE = 8\n\n TARGETS = [\n \"Dry_Green_g\",\n \"GDM_g\",\n \"Dry_Total_g\",\n \"Dry_Dead_g\",\n \"Dry_Clover_g\",\n ]\n\n pre_trained_models = \"/kaggle/input/my-pretrained-models\"\n\n\n\nconfig = Config()\n\n\ndef weighted_r2_score(y_true, y_pred):\n weights = np.array([0.1, 0.1, 0.1, 0.2, 0.5])\n r2s = []\n\n for i in range(5):\n yt, yp = y_true[:, i], y_pred[:, i]\n ss_res = np.sum((yt - yp) ** 2)\n ss_tot = np.sum((yt - yt.mean()) ** 2)\n r2 = 1 - ss_res / ss_tot if ss_tot > 0 else 0\n r2s.append(r2)\n\n r2s = np.array(r2s)\n return np.sum(r2s * weights) / weights.sum(), r2s\n\n\ndef calc_metric(outputs, targets):\n y_true = np.column_stack((\n targets[:, 0],\n targets[:, 1],\n targets[:, 2],\n targets[:, 2] - targets[:, 1],\n targets[:, 1] - targets[:, 0],\n ))\n\n y_pred = np.column_stack((\n outputs[:, 0],\n outputs[:, 1],\n outputs[:, 2],\n outputs[:, 2] - outputs[:, 1],\n outputs[:, 1] - outputs[:, 0],\n ))\n\n return weighted_r2_score(y_true, y_pred)[0]\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-12-31T00:02:43.212228Z","iopub.execute_input":"2025-12-31T00:02:43.212859Z","iopub.status.idle":"2025-12-31T00:02:43.223718Z","shell.execute_reply.started":"2025-12-31T00:02:43.212817Z","shell.execute_reply":"2025-12-31T00:02:43.222778Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"\n

\n3. Reshaping the Training Data\n

\n\n
\n\n

\n The raw CSV is stored in a long format where each image appears multiple times in the CSV files β€” once for each biomass component. \n For efficient multi-output regression, we pivot the dataframe so that:\n

\n\n \n\n

\n This transformation enables loading each image once per iteration while predicting all biomass components jointly.\n

\n\n
\n","metadata":{}},{"cell_type":"code","source":"train_df = pd.read_csv(config.train_path)\n\ndf = train_df.pivot(\n index=\"image_path\",\n columns=\"target_name\",\n values=\"target\"\n).reset_index()\n\ndf.columns.name = None","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-12-31T00:02:43.224827Z","iopub.execute_input":"2025-12-31T00:02:43.225242Z","iopub.status.idle":"2025-12-31T00:02:43.326508Z","shell.execute_reply.started":"2025-12-31T00:02:43.225204Z","shell.execute_reply":"2025-12-31T00:02:43.325434Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"\n

\n4. Custom Dataset \n

\n\n
\n\n

\n We define a PyTorch Dataset that loads each pasture image, splits it vertically into left and right halves, and returns both views along with the regression targets.\n

\n\n \n\n\n
\n","metadata":{}},{"cell_type":"code","source":"class BiomassDataset(Dataset):\n def __init__(self, df, root_dir, transform, train=True):\n self.train = train\n self.root_dir = root_dir\n self.transform = transform\n\n if self.train:\n self.df = df.reset_index(drop=True)\n else:\n self.items = list(df)\n\n def __len__(self):\n return len(self.df) if self.train else len(self.items)\n\n def __getitem__(self, idx):\n if self.train:\n row = self.df.iloc[idx]\n img_path = row[\"image_path\"]\n else:\n img_path, sub_df = self.items[idx]\n\n img = Image.open(os.path.join(self.root_dir, img_path)).convert(\"RGB\")\n\n w, h = img.size\n img_left = img.crop((0, 0, w // 2, h))\n img_right = img.crop((w // 2, 0, w, h))\n\n img_left = self.transform(img_left)\n img_right = self.transform(img_right)\n\n if self.train:\n target = torch.tensor([\n row['Dry_Green_g'],\n row['GDM_g'],\n row['Dry_Total_g'],\n ], dtype=torch.float32)\n\n\n return img_left, img_right, target\n else:\n return img_left, img_right, sub_df\n\n\ndef collate_fn(batch):\n lefts = torch.stack([b[0] for b in batch])\n rights = torch.stack([b[1] for b in batch])\n thirds = [b[2] for b in batch]\n\n if torch.is_tensor(thirds[0]):\n thirds = torch.stack(thirds)\n\n return lefts, rights, thirds\n\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-12-31T00:02:43.328617Z","iopub.execute_input":"2025-12-31T00:02:43.329064Z","iopub.status.idle":"2025-12-31T00:02:43.339435Z","shell.execute_reply.started":"2025-12-31T00:02:43.329036Z","shell.execute_reply":"2025-12-31T00:02:43.338373Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"\n

\n5. ConvNeXt Regressor Model\n

\n\n
\n\n

\n This model adapts a ConvNeXt-Tiny backbone into a dual-view regression network for pasture biomass estimation.\n

\n\n \n\n\n
\n","metadata":{}},{"cell_type":"code","source":"class ConvNeXtRegressor(nn.Module):\n def __init__(self):\n super().__init__()\n\n self.backbone = timm.create_model(\n \"convnext_tiny\",\n pretrained=False,\n features_only=False\n )\n self.backbone.head = nn.Identity()\n\n self.pool = nn.AdaptiveAvgPool2d(1)\n\n self.fc = nn.Sequential(\n nn.Linear(768 * 2, 256),\n nn.ReLU(),\n nn.Dropout(0.2),\n nn.Linear(256, 64),\n nn.ReLU()\n )\n\n self.head_green = nn.Linear(64, 1)\n self.head_clover = nn.Linear(64, 1)\n self.head_dead = nn.Linear(64, 1)\n\n self.softplus = nn.Softplus(beta=1.0)\n\n def forward(self, x_left, x_right):\n f_left = self.backbone.forward_features(x_left)\n f_right = self.backbone.forward_features(x_right)\n\n f = torch.cat([f_left, f_right], dim=1)\n f = self.pool(f).flatten(1)\n f = self.fc(f)\n\n green = self.softplus(self.head_green(f))\n clover = self.softplus(self.head_clover(f))\n dead = self.softplus(self.head_dead(f))\n\n logits = torch.cat([green, dead, clover], dim=1)\n return logits\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-12-31T00:02:43.340492Z","iopub.execute_input":"2025-12-31T00:02:43.340791Z","iopub.status.idle":"2025-12-31T00:02:43.360228Z","shell.execute_reply.started":"2025-12-31T00:02:43.340765Z","shell.execute_reply":"2025-12-31T00:02:43.359258Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"\n

\n6. Training & Validation Pipeline\n

\n\n
\n\n

\n This block defines the core utilities used to optimize the model, evaluate performance, and control training dynamics.\n

\n\n \n\n\n\n
\n","metadata":{}},{"cell_type":"code","source":"def train_one_epoch(model, loader, optimizer, criterion, scaler, device):\n model.train()\n total_loss = 0\n\n for x_left, x_right, y in loader:\n x_left, x_right, y = x_left.to(device), x_right.to(device), y.to(device)\n optimizer.zero_grad()\n\n with autocast():\n preds = model(x_left, x_right)\n loss = criterion(preds, y)\n\n scaler.scale(loss).backward()\n scaler.step(optimizer)\n scaler.update()\n\n total_loss += loss.item()\n\n return total_loss / len(loader.dataset)\n\n\ndef validate(model, loader, criterion, device):\n model.eval()\n total_loss = 0\n preds, trues = [], []\n\n with torch.no_grad():\n for x_left, x_right, y in loader:\n x_left, x_right, y = x_left.to(device), x_right.to(device), y.to(device)\n p = model(x_left, x_right)\n\n total_loss += criterion(p, y).item()\n preds.append(p.cpu())\n trues.append(y.cpu())\n\n preds = torch.cat(preds).numpy()\n trues = torch.cat(trues).numpy()\n\n wr2 = calc_metric(preds, trues)\n return total_loss / len(loader.dataset), wr2\n\n\nclass EarlyStopping:\n def __init__(self, patience=10, min_delta=1e-4):\n self.patience = patience\n self.min_delta = min_delta\n self.best = None\n self.counter = 0\n self.stop = False\n\n def step(self, score):\n if self.best is None:\n self.best = score\n elif score < self.best + self.min_delta:\n self.counter += 1\n if self.counter >= self.patience:\n self.stop = True\n else:\n self.best = score\n self.counter = 0\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-12-31T00:02:43.361402Z","iopub.execute_input":"2025-12-31T00:02:43.361843Z","iopub.status.idle":"2025-12-31T00:02:43.380593Z","shell.execute_reply.started":"2025-12-31T00:02:43.361811Z","shell.execute_reply":"2025-12-31T00:02:43.379591Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"\n

\n7. Training loop\n

\n\n
\n\n

\n This section implements the full training pipeline using K-Fold cross-validation to obtain robust and generalizable models.\n

\n\n \n\n

\n This strategy yields multiple models from different data splits, later combined as an ensemble during inference.\n

\n\n
\n","metadata":{}},{"cell_type":"code","source":"# train_tfms = transforms.Compose([\n# transforms.Resize((224, 224)),\n# transforms.RandomHorizontalFlip(p=0.5),\n# transforms.RandomVerticalFlip(p=0.5),\n# transforms.ColorJitter(\n# brightness=0.1,\n# contrast=0.1,\n# saturation=0.1,\n# hue=0.05\n# ),\n# transforms.ToTensor(),\n# transforms.Normalize(\n# mean=[0.485, 0.456, 0.406],\n# std=[0.229, 0.224, 0.225]\n# )\n# ])\n\n# val_tfms = transforms.Compose([\n# transforms.Resize((224, 224)),\n# transforms.ToTensor(),\n# transforms.Normalize(\n# [0.485, 0.456, 0.406],\n# [0.229, 0.224, 0.225]\n# )\n# ])\n\n# kf = KFold(n_splits=5, shuffle=True, random_state=42)\n\n# all_fold_histories = []\n\n# for fold, (tr_idx, va_idx) in enumerate(kf.split(df)):\n# print(f\"\\n===== Fold {fold+1} =====\")\n\n# train_ds = BiomassDataset(df.iloc[tr_idx], config.root_dir, train_tfms)\n# val_ds = BiomassDataset(df.iloc[va_idx], config.root_dir, val_tfms)\n\n# train_loader = DataLoader(\n# train_ds, batch_size=8, shuffle=True, num_workers=2\n# )\n# val_loader = DataLoader(\n# val_ds, batch_size=8, shuffle=False, num_workers=2\n# )\n\n# model = ConvNeXtRegressor().to(device)\n# criterion = nn.SmoothL1Loss()\n# optimizer = torch.optim.AdamW(model.parameters(), lr=1e-4)\n# scheduler = torch.optim.lr_scheduler.CosineAnnealingLR(\n# optimizer, T_max=config.NUM_EPOCHS\n# )\n# scaler = GradScaler()\n\n# early_stopper = EarlyStopping(patience=config.PATIENCE)\n\n# best_r2 = -1e9\n# history = {\"train_loss\": [], \"val_loss\": [], \"val_r2\": []}\n\n# for epoch in range(config.NUM_EPOCHS):\n# train_loss = train_one_epoch(\n# model, train_loader, optimizer, criterion, scaler, device\n# )\n# val_loss, val_r2 = validate(\n# model, val_loader, criterion, device\n# )\n# scheduler.step()\n\n# history[\"train_loss\"].append(train_loss)\n# history[\"val_loss\"].append(val_loss)\n# history[\"val_r2\"].append(val_r2)\n\n# print(\n# f\"Epoch {epoch+1:03d}: \"\n# f\"Train {train_loss:.4f} | \"\n# f\"Val {val_loss:.4f} | \"\n# f\"R2 {val_r2:.4f}\"\n# )\n\n# if val_r2 > best_r2:\n# best_r2 = val_r2\n# torch.save(\n# model.state_dict(),\n# f\"best_convnext_fold{fold}.pth\"\n# )\n\n# early_stopper.step(val_r2)\n# if early_stopper.stop:\n# print(f\"πŸ›‘ Early stopping at epoch {epoch+1}\")\n# break\n\n# all_fold_histories.append(history)\n\n# epochs = range(1, len(history[\"train_loss\"]) + 1)\n\n# plt.figure(figsize=(12, 4))\n\n# plt.subplot(1, 2, 1)\n# plt.plot(epochs, history[\"train_loss\"], label=\"Train Loss\")\n# plt.plot(epochs, history[\"val_loss\"], label=\"Val Loss\")\n# plt.xlabel(\"Epoch\")\n# plt.ylabel(\"Loss\")\n# plt.title(f\"Fold {fold+1} Loss\")\n# plt.legend()\n\n# plt.subplot(1, 2, 2)\n# plt.plot(epochs, history[\"val_r2\"], label=\"Val Weighted R2\")\n# plt.xlabel(\"Epoch\")\n# plt.ylabel(\"Weighted R2\")\n# plt.title(f\"Fold {fold+1} R2\")\n# plt.legend()\n\n# plt.show()\n\n# print(f\"βœ… Best R2 for fold {fold+1}: {best_r2:.4f}\")\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-12-31T00:02:43.381830Z","iopub.execute_input":"2025-12-31T00:02:43.382271Z","iopub.status.idle":"2025-12-31T00:02:43.404796Z","shell.execute_reply.started":"2025-12-31T00:02:43.382232Z","shell.execute_reply":"2025-12-31T00:02:43.403748Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"\n

\n8. Inference & Submission\n

\n\n
\n\n

\n This block performs final prediction on the test set and prepares the Kaggle submission using an ensemble of the best models from all folds.\n

\n\n \n\n\n
\n","metadata":{}},{"cell_type":"code","source":"models = []\n\ntest_df = pd.read_csv(os.path.join(config.root_dir, \"test.csv\"))\ngroup_df = test_df.groupby(\"image_path\")\n\ntest_tfms = transforms.Compose([\n transforms.Resize((224, 224)),\n transforms.ToTensor(),\n transforms.Normalize(\n [0.485, 0.456, 0.406],\n [0.229, 0.224, 0.225]\n )\n])\n\ntest_ds = BiomassDataset(\n group_df, config.root_dir, test_tfms, train=False\n)\n\ntest_loader = DataLoader(\n test_ds,\n batch_size=16,\n shuffle=False,\n num_workers=2,\n collate_fn=collate_fn\n)\n\nmodel_paths = sorted(\n [p for p in os.listdir(config.pre_trained_models) if p.endswith(\".pth\")]\n)\n\nfor p in model_paths:\n m = ConvNeXtRegressor().to(device)\n state = torch.load(\n os.path.join(config.pre_trained_models, p),\n map_location=device\n )\n m.load_state_dict(state)\n m.eval()\n models.append(m)\n\nprint(f\"βœ… Loaded {len(models)} models\")\n\npreds = []\n\nwith torch.no_grad():\n for imgs_left, imgs_right, df_batch in tqdm(test_loader):\n imgs_left = imgs_left.to(device)\n imgs_right = imgs_right.to(device)\n\n preds_sum = torch.zeros(\n (imgs_left.size(0), 3), device=device\n )\n\n for m in models:\n preds_sum += m(imgs_left, imgs_right)\n\n preds_avg = preds_sum / len(models)\n preds_avg = preds_avg.cpu().numpy()\n\n preds_5 = np.column_stack((\n preds_avg[:, 0],\n preds_avg[:, 1],\n preds_avg[:, 2],\n preds_avg[:, 2] - preds_avg[:, 1],\n preds_avg[:, 1] - preds_avg[:, 0],\n \n ))\n\n for i, df_img in enumerate(df_batch):\n row_preds = preds_5[i]\n image_id = df_img[\"sample_id\"].iloc[0].split(\"__\")[0]\n\n for j, target_name in enumerate(config.TARGETS):\n preds.append({\n \"sample_id\": f\"{image_id}__{target_name}\",\n \"target\": float(max(0.0, row_preds[j]))\n })\n\nsubmission = pd.DataFrame(preds)\nsubmission.to_csv(\"submission.csv\", index=False)\n\nprint(\"πŸŽ‰ Submission saved:\", submission.shape)\nprint(submission.head())\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-12-31T00:02:43.406104Z","iopub.execute_input":"2025-12-31T00:02:43.406489Z","iopub.status.idle":"2025-12-31T00:02:51.973388Z","shell.execute_reply.started":"2025-12-31T00:02:43.406451Z","shell.execute_reply":"2025-12-31T00:02:51.972394Z"}},"outputs":[],"execution_count":null}]}