{"metadata":{"kernelspec":{"name":"python3","display_name":"Python 3","language":"python"},"language_info":{"name":"python","version":"3.11.13","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"},"kaggle":{"accelerator":"gpu","dataSources":[{"sourceId":112509,"databundleVersionId":14254895,"sourceType":"competition"}],"dockerImageVersionId":31192,"isInternetEnabled":true,"language":"python","sourceType":"notebook","isGpuEnabled":true}},"nbformat_minor":4,"nbformat":4,"cells":[{"id":"cc56a12f-d644-4ba7-b4f7-72058cfbc106","cell_type":"markdown","source":"\n\n# 🧭 Introduction
\n\nIn this notebook, we explore the **impact of metadata** on biomass prediction in the [CSIRO – Image2Biomass Prediction](https://www.kaggle.com/competitions/csiro-image2biomass-prediction) competition. \nWhile the primary task focuses on predicting pasture biomass from top-view RGB images, the dataset also includes several **auxiliary metadata fields** such as collection date, state, species, NDVI, and average height. \n\nCollecting such metadata in real-world agricultural monitoring systems is often **expensive and logistically challenging**. Therefore, understanding whether these additional features actually improve model performance is valuable for **assessing cost–benefit trade-offs** in practical deployments.\n\n### Dataset Overview\n\n| Column | Description | Unit | Train | Val/Test |\n| :--- | :--- | :--- | :---: | :---: |\n| **image_path** | Path to the corresponding image | - | ✔ | ✔ |\n| **Sampling_Date** | Date of data collection | YYYY-MM-DD | ✔ | ✖️ |\n| **State** | Australian state | - | ✔ | ✖️ |\n| **Species** | Pasture species (by biomass) | - | ✔ | ✖️ |\n| **Pre_GSHH_NDVI** | Normalized Difference Vegetation Index (from GreenSeeker) | 0–1 | ✔ | ✖️ |\n| **Height_Ave_cm** | Average plant height (falling plate) | cm | ✔ | ✖️ |\n| **Dry_Green_g** | Green vegetation other than clover | grams | ✔ | ✔ |\n| **Dry_Dead_g** | Senescent (dead) material | grams | ✔ | ✔ |\n| **Dry_Clover_g** | Clover component | grams | ✔ | ✔ |\n| **GDM_g** | Green dry matter (Green + Clover) | grams | ✔ | ✔ |\n| **Dry_Total_g** | Total dry biomass (all components) | grams | ✔ | ✔ |\n\n### Experimental Design\n\nWe apply **5-fold cross-validation** across **five experimental settings**, each including a different subset of metadata. \nThis helps identify which metadata fields contribute most to model performance.\n\n| Experiment | Image | Use Date | Use Height | Use NDVI | Use State | Use Species |\n| :--- | :---: | :---: | :---: | :---: | :---: | :---: |\n| **baseline** | ✔ | ✖️ | ✖️ | ✖️ | ✖️ | ✖️ |\n| **temporal_features** | ✔ | ✔ | ✖️ | ✖️ | ✖️ | ✖️ |\n| **measurement_features** | ✔ | ✖️ | ✔ | ✔ | ✖️ | ✖️ |\n| **contextual_features** | ✔ | ✖️ | ✖️ | ✖️ | ✔ | ✔ |\n| **all_features** | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ |\n\n### Model & Training Setup\n\nAll experiments use the same model backbone — **`tf_efficientnetv2_s`** from the [timm](https://github.com/huggingface/pytorch-image-models) library — with identical training configurations across folds. \nNo extensive hyperparameter tuning was performed, as the goal is to **evaluate relative feature importance** rather than achieve leaderboard optimization.\n\nIf you're unfamiliar with the dataset or the metadata fields, you can check out my previous *paper-in-depth* notebook for background context.","metadata":{}},{"id":"27dbb72b","cell_type":"markdown","source":"### Libraries","metadata":{}},{"id":"ff07b07e","cell_type":"code","source":"","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"id":"0296edd8","cell_type":"markdown","source":"### Imports","metadata":{}},{"id":"0854aaf0","cell_type":"code","source":"# system\nimport os\nfrom glob import glob\nfrom tqdm.auto import tqdm\nfrom pathlib import Path\nimport json\nfrom datetime import datetime\n\n# data\nimport cv2\nimport numpy as np\nimport pandas as pd\nimport matplotlib.pyplot as plt\nimport seaborn as sns\nfrom IPython.display import display_html\n\n# deep learning\nimport torch\nimport torch.nn as nn\nfrom torch.utils.data import Dataset, DataLoader\nfrom torch.optim import Adam\nfrom torch.optim.lr_scheduler import CosineAnnealingLR\nimport torchvision.transforms as T\nimport albumentations as A\nimport timm\n\n# sklearn\nfrom sklearn.model_selection import StratifiedKFold\nfrom sklearn.preprocessing import OneHotEncoder, StandardScaler\nfrom sklearn.metrics import r2_score\n\n# other\nimport gc","metadata":{"metadata":{},"trusted":true,"execution":{"iopub.status.busy":"2025-11-13T13:11:39.351707Z","iopub.execute_input":"2025-11-13T13:11:39.352418Z","iopub.status.idle":"2025-11-13T13:11:52.012258Z","shell.execute_reply.started":"2025-11-13T13:11:39.352390Z","shell.execute_reply":"2025-11-13T13:11:52.011441Z"}},"outputs":[],"execution_count":null},{"id":"fc0892ca","cell_type":"markdown","source":"### Config","metadata":{}},{"id":"7c149e0e","cell_type":"code","source":"# System\nPATH = \"/kaggle/input/csiro-biomass\"\nOUTPUT = \"pool\"\nSEED = 35\nDEVICE = torch.device('cuda') if torch.cuda.is_available() else 'cpu'\nUSE_AMP = False\n\n# Hyperparameters\nLR = 1e-3\nEPOCHS = 30\nBATCH_SIZE = 8\nNUM_WORKERS = 4\nWD = 1e-2\nES = 7\nGRAD_ACC = 2\nFOLDS = 5\n\n# Model\nMODEL = 'tf_efficientnetv2_s.in21k_ft_in1k'\nPRETRAINED = True\nIMAGE_SIZE = (512, 256)\nIN_CHANNELS = 3\nNUM_CLASSES = 5\n\nos.makedirs(OUTPUT, exist_ok=True)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-11-13T13:11:52.013606Z","iopub.execute_input":"2025-11-13T13:11:52.013886Z","iopub.status.idle":"2025-11-13T13:11:52.082189Z","shell.execute_reply.started":"2025-11-13T13:11:52.013862Z","shell.execute_reply":"2025-11-13T13:11:52.081375Z"}},"outputs":[],"execution_count":null},{"id":"c1d28816","cell_type":"markdown","source":"### Helpers","metadata":{}},{"id":"8e4017e9","cell_type":"code","source":"def summary(df, desc='Summary'):\n print(f'\\n----- {desc} -----\\n')\n print(f'Shape: {df.shape}')\n print(f'Missing: {df.isna().sum().sum()}')\n print(f'Columns: {df.columns.to_list()}\\n')\n display_html(df.head(3))\n print('\\n')","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-11-13T13:11:52.083036Z","iopub.execute_input":"2025-11-13T13:11:52.083380Z","iopub.status.idle":"2025-11-13T13:11:52.104644Z","shell.execute_reply.started":"2025-11-13T13:11:52.083354Z","shell.execute_reply":"2025-11-13T13:11:52.103898Z"}},"outputs":[],"execution_count":null},{"id":"0a457af9","cell_type":"markdown","source":"# 📂 Load the data
","metadata":{}},{"id":"f92ed3c4","cell_type":"code","source":"train_df = pd.read_csv(os.path.join(PATH, 'train.csv'))","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-11-13T13:11:52.105405Z","iopub.execute_input":"2025-11-13T13:11:52.105644Z","iopub.status.idle":"2025-11-13T13:11:52.136371Z","shell.execute_reply.started":"2025-11-13T13:11:52.105621Z","shell.execute_reply":"2025-11-13T13:11:52.135739Z"}},"outputs":[],"execution_count":null},{"id":"52e86c0f-cf0a-4573-9f61-b3341888917f","cell_type":"code","source":"summary(train_df)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-11-13T13:11:52.138057Z","iopub.execute_input":"2025-11-13T13:11:52.138257Z","iopub.status.idle":"2025-11-13T13:11:52.156937Z","shell.execute_reply.started":"2025-11-13T13:11:52.138242Z","shell.execute_reply":"2025-11-13T13:11:52.156384Z"}},"outputs":[],"execution_count":null},{"id":"ca1a3385-3532-4979-b6f6-6005bee3612c","cell_type":"markdown","source":"# ⚙️ Preprocess data
","metadata":{}},{"id":"8e61aab9-4f04-410d-82ae-86890017184f","cell_type":"code","source":"def preprocess(df, log_target=False):\n\n df = df.copy()\n\n df = (\n df.pivot_table(\n index=[\"image_path\", \"Sampling_Date\", \"State\", \"Species\", \"Pre_GSHH_NDVI\", \"Height_Ave_cm\"],\n columns=\"target_name\",\n values=\"target\"\n )\n .reset_index()\n )\n df.columns.name = None\n\n # Date\n df['Sampling_Date'] = pd.to_datetime(df['Sampling_Date'])\n\n df['dayofyear'] = df['Sampling_Date'].dt.dayofyear\n df['is_year_start'] = df['Sampling_Date'].dt.is_year_start.astype(int)\n df['is_year_end'] = df['Sampling_Date'].dt.is_year_end.astype(int)\n df['quarter'] = df['Sampling_Date'].dt.quarter\n df['is_quarter_start'] = df['Sampling_Date'].dt.is_quarter_start.astype(int)\n df['is_quarter_end'] = df['Sampling_Date'].dt.is_quarter_end.astype(int)\n df['month'] = df['Sampling_Date'].dt.month\n df['day'] = df['Sampling_Date'].dt.day\n\n # One-Hot\n cats = ['State', 'Species']\n oh_enc = OneHotEncoder(handle_unknown='ignore', sparse_output=False)\n X_encoded = oh_enc.fit_transform(df[cats])\n encoded_cols = oh_enc.get_feature_names_out(cats)\n df_encoded = pd.DataFrame(X_encoded, columns=encoded_cols, index=df.index)\n\n df = pd.concat([df.drop(columns=cats), df_encoded], axis=1)\n\n # Target\n target_names = [\n 'Dry_Clover_g', \n 'Dry_Dead_g', \n 'Dry_Green_g', \n 'Dry_Total_g', \n 'GDM_g'\n ]\n if log_target:\n df[target_names] = df[target_names].apply(np.log1p)\n \n return df","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-11-13T13:11:52.157668Z","iopub.execute_input":"2025-11-13T13:11:52.157852Z","iopub.status.idle":"2025-11-13T13:11:52.165307Z","shell.execute_reply.started":"2025-11-13T13:11:52.157837Z","shell.execute_reply":"2025-11-13T13:11:52.164616Z"}},"outputs":[],"execution_count":null},{"id":"815d3331-779a-467a-89e3-2e72f2120e89","cell_type":"code","source":"df = preprocess(train_df, log_target=True)\ndf.head()","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-11-13T13:11:52.166018Z","iopub.execute_input":"2025-11-13T13:11:52.166256Z","iopub.status.idle":"2025-11-13T13:11:52.249023Z","shell.execute_reply.started":"2025-11-13T13:11:52.166236Z","shell.execute_reply":"2025-11-13T13:11:52.248362Z"}},"outputs":[],"execution_count":null},{"id":"90076658","cell_type":"markdown","source":"# 👷♂️ Create dataset
","metadata":{}},{"id":"961f0473-71c3-4256-abe4-9526cb4fe6d8","cell_type":"code","source":"class CSIRODataset(Dataset):\n def __init__(self, df, transform=None, image_size=(512, 256),\n use_date=True, use_height=True, use_ndvi=True,\n use_state=True, use_species=True):\n \n self.df = df\n self.transform = transform\n self.image_size = image_size\n self.all_images = df.image_path.unique()\n \n self.target_names = [\n 'Dry_Clover_g', \n 'Dry_Dead_g', \n 'Dry_Green_g', \n 'Dry_Total_g', \n 'GDM_g'\n ]\n \n self.meta_names = []\n \n if use_date:\n self.meta_names.extend([\n 'dayofyear', 'is_year_start', 'is_year_end',\n 'quarter', 'is_quarter_start', 'is_quarter_end',\n 'month', 'day'\n ])\n \n if use_height:\n self.meta_names.extend(['Height_Ave_cm'])\n \n if use_ndvi:\n self.meta_names.extend(['Pre_GSHH_NDVI'])\n \n if use_state:\n self.meta_names.extend([\n 'State_NSW', 'State_Tas', 'State_Vic', 'State_WA'\n ])\n \n if use_species:\n self.meta_names.extend([\n 'Species_Clover', 'Species_Fescue', \n 'Species_Fescue_CrumbWeed', 'Species_Lucerne',\n 'Species_Mixed', 'Species_Phalaris',\n 'Species_Phalaris_BarleyGrass_SilverGrass_SpearGrass_Clover_Capeweed',\n 'Species_Phalaris_Clover',\n 'Species_Phalaris_Clover_Ryegrass_Barleygrass_Bromegrass',\n 'Species_Phalaris_Ryegrass_Clover', 'Species_Ryegrass',\n 'Species_Ryegrass_Clover', 'Species_SubcloverDalkeith',\n 'Species_SubcloverLosa', 'Species_WhiteClover'\n ])\n \n def __len__(self):\n return len(self.all_images)\n \n def __getitem__(self, idx):\n image_path = self.all_images[idx]\n sample_df = self.df[self.df.image_path == image_path]\n \n image = cv2.imread(os.path.join(PATH, image_path))\n if image is None:\n raise ValueError(f\"Failed to load image: {image_path}\")\n \n image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)\n image = cv2.resize(image, self.image_size)\n \n label = sample_df[self.target_names].iloc[0].values.astype(np.float32)\n \n if self.transform:\n image = self.transform(image=image)[\"image\"]\n \n if not isinstance(image, torch.Tensor):\n image = np.transpose(image, (2, 0, 1))\n image = torch.tensor(image, dtype=torch.float32)\n \n label = torch.tensor(label, dtype=torch.float32)\n \n if self.meta_names:\n metadata = sample_df[self.meta_names].iloc[0].values.astype(np.float32)\n metadata = torch.tensor(metadata, dtype=torch.float32)\n else:\n metadata = torch.tensor([], dtype=torch.float32)\n \n return {\n 'image': image,\n 'label': label,\n 'metadata': metadata\n }","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-11-13T13:11:52.249958Z","iopub.execute_input":"2025-11-13T13:11:52.250402Z","iopub.status.idle":"2025-11-13T13:11:52.259360Z","shell.execute_reply.started":"2025-11-13T13:11:52.250383Z","shell.execute_reply":"2025-11-13T13:11:52.258796Z"}},"outputs":[],"execution_count":null},{"id":"24fb00a7","cell_type":"code","source":"train_transform = A.Compose([\n A.HorizontalFlip(p=0.5),\n A.VerticalFlip(p=0.5),\n A.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225]),\n A.RandomBrightnessContrast(p=0.2),\n])\n\nvalid_transform = A.Compose([\n A.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225])\n])","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-11-13T13:11:52.515032Z","iopub.execute_input":"2025-11-13T13:11:52.515246Z","iopub.status.idle":"2025-11-13T13:11:52.530468Z","shell.execute_reply.started":"2025-11-13T13:11:52.515231Z","shell.execute_reply":"2025-11-13T13:11:52.530023Z"}},"outputs":[],"execution_count":null},{"id":"4afbedb9","cell_type":"markdown","source":"# 🥄 Try dataset
","metadata":{}},{"id":"6bae07c0","cell_type":"code","source":"tmp_ds = CSIRODataset(df, transform=train_transform, image_size=IMAGE_SIZE)\ntmp_dl = DataLoader(tmp_ds, batch_size=BATCH_SIZE, num_workers=NUM_WORKERS, shuffle=True)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-11-13T13:11:52.533316Z","iopub.execute_input":"2025-11-13T13:11:52.533563Z","iopub.status.idle":"2025-11-13T13:11:52.544502Z","shell.execute_reply.started":"2025-11-13T13:11:52.533542Z","shell.execute_reply":"2025-11-13T13:11:52.543778Z"}},"outputs":[],"execution_count":null},{"id":"f616a904-d144-4a4c-b748-ffe29ad86061","cell_type":"code","source":"for idx, batch in enumerate(tmp_dl):\n\n for j in range(5):\n x = batch['image'][j]\n y = batch['label'][j]\n m = batch['metadata'][j]\n print(f'\\nImage: {x.shape}')\n print(f'[Image Stats]: Min: {x.min()}, Max: {x.max()}, Mean: {x.mean()}, Std: {x.std()}')\n plt.imshow(x.permute(1, 2, 0).numpy())\n plt.axis('off')\n plt.show()\n print(f'\\nLabel: {y.shape}')\n print(f'[Label Stats]: Min: {y.min()}, Max: {y.max()}, Mean: {y.mean()}, Std: {y.std()}')\n print(f'\\nMetadata: {m.shape}')\n print(f'[Metadata Stats]: Min: {m.min()}, Max: {m.max()}, Mean: {m.mean()}, Std: {m.std()}')\n print('=====' * 20)\n\n if idx == 0:\n break","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-11-13T13:11:52.545100Z","iopub.execute_input":"2025-11-13T13:11:52.545331Z","iopub.status.idle":"2025-11-13T13:11:55.073732Z","shell.execute_reply.started":"2025-11-13T13:11:52.545298Z","shell.execute_reply":"2025-11-13T13:11:55.072909Z"}},"outputs":[],"execution_count":null},{"id":"6efb8b6e","cell_type":"markdown","source":"# 🧑💻 Create model
","metadata":{}},{"id":"8adece7d-79bd-4090-9e4b-fdcc72077463","cell_type":"code","source":"class CSIROModel(nn.Module):\n def __init__(self, model_name, pretrained=PRETRAINED, in_chans=IN_CHANNELS,\n num_classes=NUM_CLASSES, metadata_dim=0, fusion_dim=256, dropout=0.2):\n\n super(CSIROModel, self).__init__()\n \n self.metadata_dim = metadata_dim\n \n self.image_model = timm.create_model(\n model_name=model_name,\n pretrained=pretrained,\n in_chans=in_chans,\n num_classes=0,\n )\n \n self.image_feature_dim = self.image_model.num_features\n \n if metadata_dim > 0:\n self.metadata_processor = nn.Sequential(\n nn.Linear(metadata_dim, 128),\n nn.ReLU(),\n nn.Dropout(dropout),\n nn.Linear(128, 64),\n nn.ReLU(),\n nn.Dropout(dropout)\n )\n metadata_output_dim = 64\n else:\n self.metadata_processor = None\n metadata_output_dim = 0\n \n combined_dim = self.image_feature_dim + metadata_output_dim\n \n self.fusion = nn.Sequential(\n nn.Linear(combined_dim, fusion_dim),\n nn.ReLU(),\n nn.Dropout(dropout),\n nn.Linear(fusion_dim, fusion_dim // 2),\n nn.ReLU(),\n nn.Dropout(dropout)\n )\n \n self.regressor = nn.Linear(fusion_dim // 2, num_classes)\n \n def forward(self, image, metadata=None):\n\n image_features = self.image_model(image)\n \n if self.metadata_dim > 0 and metadata is not None:\n if metadata.size(1) != self.metadata_dim:\n raise ValueError(f\"Expected metadata_dim={self.metadata_dim}, got {metadata.size(1)}\")\n metadata_features = self.metadata_processor(metadata)\n combined = torch.cat([image_features, metadata_features], dim=1)\n else:\n combined = image_features\n \n fused = self.fusion(combined)\n predictions = self.regressor(fused)\n \n return predictions\n\n\ndef get_model(model_name=MODEL, pretrained=PRETRAINED, in_chans=IN_CHANNELS, \n num_classes=NUM_CLASSES, metadata_dim=0, fusion_dim=256, dropout=0.2):\n\n model = CSIROModel(\n model_name=model_name,\n pretrained=pretrained,\n in_chans=in_chans,\n num_classes=num_classes,\n metadata_dim=metadata_dim,\n fusion_dim=fusion_dim,\n dropout=dropout\n )\n return model","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-11-13T13:11:55.074900Z","iopub.execute_input":"2025-11-13T13:11:55.075186Z","iopub.status.idle":"2025-11-13T13:11:55.086701Z","shell.execute_reply.started":"2025-11-13T13:11:55.075163Z","shell.execute_reply":"2025-11-13T13:11:55.085950Z"}},"outputs":[],"execution_count":null},{"id":"53fd8eef","cell_type":"markdown","source":"# 🥄 Try Model
","metadata":{}},{"id":"80c80203-9268-44df-96d9-58edc51c2d3c","cell_type":"code","source":"tmp_model = get_model(\n model_name=MODEL,\n pretrained=PRETRAINED,\n metadata_dim=0\n)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-11-13T13:11:55.087486Z","iopub.execute_input":"2025-11-13T13:11:55.087808Z","iopub.status.idle":"2025-11-13T13:11:57.317418Z","shell.execute_reply.started":"2025-11-13T13:11:55.087782Z","shell.execute_reply":"2025-11-13T13:11:57.316670Z"}},"outputs":[],"execution_count":null},{"id":"57ad915b","cell_type":"code","source":"tmp_model(next(iter(tmp_dl))['image'])","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-11-13T13:11:57.318254Z","iopub.execute_input":"2025-11-13T13:11:57.318653Z","iopub.status.idle":"2025-11-13T13:12:02.627440Z","shell.execute_reply.started":"2025-11-13T13:11:57.318628Z","shell.execute_reply":"2025-11-13T13:12:02.626520Z"}},"outputs":[],"execution_count":null},{"id":"328199ee","cell_type":"code","source":"del tmp_ds, tmp_dl, tmp_model\ngc.collect()","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-11-13T13:12:02.628748Z","iopub.execute_input":"2025-11-13T13:12:02.629198Z","iopub.status.idle":"2025-11-13T13:12:02.635124Z","shell.execute_reply.started":"2025-11-13T13:12:02.629162Z","shell.execute_reply":"2025-11-13T13:12:02.634501Z"}},"outputs":[],"execution_count":null},{"id":"c1e15afd","cell_type":"markdown","source":"# 🧑🏫 Create trainer
","metadata":{}},{"id":"0b11d4ab-fbb9-49f2-af7b-a1b8cb7509e7","cell_type":"code","source":"class CSIROMetric:\n def __init__(self):\n self.weights = {\n 'Dry_Clover_g': 0.1,\n 'Dry_Dead_g': 0.1,\n 'Dry_Green_g': 0.1,\n 'Dry_Total_g': 0.5,\n 'GDM_g': 0.2,\n }\n self.target_names = list(self.weights.keys())\n self.reset()\n \n def reset(self):\n self.y_pred = []\n self.y_true = []\n self.losses = []\n self.counts = 0\n \n def update(self, preds, labels, count, loss=None):\n assert isinstance(preds, np.ndarray), \"Metric.update: preds must be a numpy array\"\n assert isinstance(labels, np.ndarray), \"Metric.update: labels must be a numpy array\"\n self.y_pred.append(preds)\n self.y_true.append(labels)\n if loss is not None:\n self.losses.append(loss * count)\n self.counts += count\n \n def compute_loss(self):\n if self.counts == 0:\n return 0.0\n return sum(self.losses) / self.counts\n \n def compute_metric(self):\n if len(self.y_pred) == 0:\n return {}\n \n y_pred = np.concatenate(self.y_pred, axis=0)\n y_true = np.concatenate(self.y_true, axis=0)\n \n r2_scores = {}\n for i, target_name in enumerate(self.target_names):\n r2 = r2_score(y_true[:, i], y_pred[:, i])\n r2_scores[f'r2_{target_name}'] = r2\n \n weighted_r2 = sum(\n r2_scores[f'r2_{target_name}'] * self.weights[target_name]\n for target_name in self.target_names\n )\n r2_scores['weighted_r2'] = weighted_r2\n \n return r2_scores\n \n def summary(self):\n metrics = self.compute_metric()\n return {\n \"loss\": self.compute_loss(),\n **metrics\n }","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-11-13T13:12:02.937449Z","iopub.execute_input":"2025-11-13T13:12:02.937699Z","iopub.status.idle":"2025-11-13T13:12:02.952624Z","shell.execute_reply.started":"2025-11-13T13:12:02.937673Z","shell.execute_reply":"2025-11-13T13:12:02.951612Z"}},"outputs":[],"execution_count":null},{"id":"788ec4b3-7372-4961-8fe6-0d6f50cbbb43","cell_type":"code","source":"class CSIROTrainer:\n def __init__(\n self,\n model_name=MODEL,\n pretrained=PRETRAINED,\n in_channels=IN_CHANNELS,\n num_classes=NUM_CLASSES,\n image_size=IMAGE_SIZE,\n batch_size=BATCH_SIZE,\n num_workers=NUM_WORKERS,\n epochs=EPOCHS,\n lr=LR,\n weight_decay=WD,\n grad_accumulation=GRAD_ACC,\n use_amp=USE_AMP,\n early_stopping_patience=ES,\n n_folds=FOLDS,\n seed=SEED,\n device=DEVICE,\n output_dir=OUTPUT,\n experiment_name=None\n ):\n\n self.model_name = model_name\n self.pretrained = pretrained\n self.in_channels = in_channels\n self.num_classes = num_classes\n self.image_size = image_size\n self.batch_size = batch_size\n self.num_workers = num_workers\n self.epochs = epochs\n self.lr = lr\n self.weight_decay = weight_decay\n self.grad_accumulation = grad_accumulation\n self.use_amp = use_amp\n self.early_stopping_patience = early_stopping_patience\n self.n_folds = n_folds\n self.seed = seed\n self.device = torch.device(device if torch.cuda.is_available() else 'cpu')\n \n if experiment_name is None:\n experiment_name = f\"{model_name}_{datetime.now().strftime('%Y%m%d_%H%M%S')}\"\n self.experiment_name = experiment_name\n self.output_dir = Path(output_dir) / experiment_name\n self.output_dir.mkdir(parents=True, exist_ok=True)\n \n self.target_names = ['Dry_Clover_g', 'Dry_Dead_g', 'Dry_Green_g', 'Dry_Total_g', 'GDM_g']\n self.target_weights = {\n 'Dry_Clover_g': 0.1,\n 'Dry_Dead_g': 0.1,\n 'Dry_Green_g': 0.1,\n 'Dry_Total_g': 0.5,\n 'GDM_g': 0.2,\n }\n \n self._save_config()\n \n print(f\"✓ Trainer initialized\")\n print(f\" Device: {self.device}\")\n print(f\" Output: {self.output_dir}\")\n \n def _save_config(self):\n config = {\n 'model_name': self.model_name,\n 'pretrained': self.pretrained,\n 'in_channels': self.in_channels,\n 'num_classes': self.num_classes,\n 'image_size': self.image_size,\n 'batch_size': self.batch_size,\n 'epochs': self.epochs,\n 'lr': self.lr,\n 'weight_decay': self.weight_decay,\n 'grad_accumulation': self.grad_accumulation,\n 'use_amp': self.use_amp,\n 'early_stopping_patience': self.early_stopping_patience,\n 'n_folds': self.n_folds,\n 'seed': self.seed,\n 'experiment_name': self.experiment_name,\n }\n \n with open(self.output_dir / 'config.json', 'w') as f:\n json.dump(config, f, indent=2)\n \n def _one_epoch(self, model, data_loader, criterion, optimizer, scheduler, \n epoch, phase, scaler_amp=None):\n model.train() if phase == 'train' else model.eval()\n\n metric_tracker = CSIROMetric()\n total_steps = len(data_loader)\n log_every = max(1, int(total_steps * 0.1))\n \n for i, batch in enumerate(data_loader):\n x = batch['image'].to(self.device)\n y = batch['label'].to(self.device)\n m = batch['metadata'].to(self.device)\n \n if phase == 'train':\n if self.use_amp and scaler_amp is not None:\n with torch.amp.autocast(device_type=self.device.type, dtype=torch.bfloat16):\n preds = model(x, metadata=m)\n loss = criterion(preds, y)\n \n scaler_amp.scale(loss).backward()\n \n if (i + 1) % self.grad_accumulation == 0:\n scaler_amp.step(optimizer)\n scaler_amp.update()\n optimizer.zero_grad()\n if scheduler is not None:\n scheduler.step()\n else:\n preds = model(x, metadata=m)\n loss = criterion(preds, y)\n loss.backward()\n \n if (i + 1) % self.grad_accumulation == 0:\n optimizer.step()\n optimizer.zero_grad()\n if scheduler is not None:\n scheduler.step()\n else:\n with torch.no_grad():\n if self.use_amp:\n with torch.amp.autocast(device_type=self.device.type, dtype=torch.bfloat16):\n preds = model(x, metadata=m)\n loss = criterion(preds, y)\n else:\n preds = model(x, metadata=m)\n loss = criterion(preds, y)\n \n metric_tracker.update(\n preds.detach().cpu().numpy(), \n y.detach().cpu().numpy(), \n x.size(0), \n loss.item()\n )\n \n if (i + 1) % log_every == 0 or (i + 1) == total_steps:\n progress_pct = 100 * (i + 1) / total_steps\n current_summary = metric_tracker.summary()\n print(f\" [{phase}] {progress_pct:5.1f}% | Loss: {current_summary['loss']:.4f} | R²: {current_summary['weighted_r2']:.4f}\")\n \n return metric_tracker.summary()\n \n def _train_fold(self, fold, train_dl, val_dl, model, fold_dir):\n criterion = nn.MSELoss()\n optimizer = torch.optim.Adam(model.parameters(), lr=self.lr, weight_decay=self.weight_decay)\n \n total_steps = len(train_dl) * self.epochs // self.grad_accumulation\n scheduler = CosineAnnealingLR(optimizer, T_max=total_steps)\n \n scaler_amp = torch.amp.GradScaler('cuda') if self.use_amp else None\n \n history = {\n 'train_loss': [], 'val_loss': [],\n 'train_weighted_r2': [], 'val_weighted_r2': [],\n }\n for target in self.target_names:\n history[f'train_r2_{target}'] = []\n history[f'val_r2_{target}'] = []\n \n best_val_loss = float('inf')\n best_val_r2 = -float('inf')\n patience_counter = 0\n \n for epoch in range(self.epochs):\n print(f\"\\n[Epoch {epoch+1}/{self.epochs}]\")\n train_metrics = self._one_epoch(\n model, train_dl, criterion, optimizer, scheduler, \n epoch, 'train', scaler_amp\n )\n \n val_metrics = self._one_epoch(\n model, val_dl, criterion, optimizer, None, \n epoch, 'val', None\n )\n \n history['train_loss'].append(train_metrics['loss'])\n history['val_loss'].append(val_metrics['loss'])\n history['train_weighted_r2'].append(train_metrics['weighted_r2'])\n history['val_weighted_r2'].append(val_metrics['weighted_r2'])\n \n for target in self.target_names:\n history[f'train_r2_{target}'].append(train_metrics[f'r2_{target}'])\n history[f'val_r2_{target}'].append(val_metrics[f'r2_{target}'])\n \n print(f\" Summary: Train Loss={train_metrics['loss']:.4f} R²={train_metrics['weighted_r2']:.4f} | \"\n f\"Val Loss={val_metrics['loss']:.4f} R²={val_metrics['weighted_r2']:.4f}\")\n \n if val_metrics['weighted_r2'] > best_val_r2:\n best_val_r2 = val_metrics['weighted_r2']\n best_val_loss = val_metrics['loss']\n torch.save({\n 'epoch': epoch,\n 'model_state_dict': model.state_dict(),\n 'optimizer_state_dict': optimizer.state_dict(),\n 'scheduler_state_dict': scheduler.state_dict(),\n 'val_loss': val_metrics['loss'],\n 'val_weighted_r2': val_metrics['weighted_r2'],\n 'scaler': scaler_amp\n }, fold_dir / 'best_model.pth')\n print(f\" ✓ Best model saved (R²: {best_val_r2:.4f})\")\n patience_counter = 0\n else:\n patience_counter += 1\n print(f\" ⏳ No improvement (Patience: {patience_counter}/{self.early_stopping_patience})\")\n \n if patience_counter >= self.early_stopping_patience:\n print(f\"\\n⚠ Early stopping at epoch {epoch+1}\")\n break\n \n self._plot_training_history(history, fold, fold_dir)\n \n pd.DataFrame(history).to_csv(fold_dir / 'history.csv', index=False)\n \n print(f\"\\n{'='*60}\")\n print(f\"FOLD {fold+1} COMPLETE - Best Val Loss: {best_val_loss:.4f} | Best Val R²: {best_val_r2:.4f}\")\n print(f\"{'='*60}\")\n \n return history, best_val_r2, best_val_loss\n \n def train(self, df, use_date=True, use_height=True, use_ndvi=True, use_state=True, use_species=True):\n\n print(f\"\\n{'='*70}\")\n print(f\"Starting Cross-Validation Training: {self.experiment_name}\")\n print(f\"{'='*70}\")\n \n df = df.copy()\n df['strat_bin'] = pd.qcut(df['Dry_Total_g'], q=5, labels=False, duplicates='drop')\n \n skf = StratifiedKFold(n_splits=self.n_folds, shuffle=True, random_state=self.seed)\n \n fold_results = []\n \n scl_cols = ['Pre_GSHH_NDVI', 'Height_Ave_cm', 'dayofyear', 'quarter', 'month', 'day']\n \n for fold, (train_idx, val_idx) in enumerate(skf.split(df, df['strat_bin'])):\n print(f\"\\n{'='*70}\")\n print(f\"FOLD {fold+1}/{self.n_folds}\")\n print(f\"{'='*70}\")\n \n fold_dir = self.output_dir / f'fold_{fold+1}'\n fold_dir.mkdir(exist_ok=True)\n \n train_df = df.iloc[train_idx].reset_index(drop=True).copy()\n valid_df = df.iloc[val_idx].reset_index(drop=True).copy()\n \n scaler = StandardScaler()\n train_df[scl_cols] = scaler.fit_transform(train_df[scl_cols])\n valid_df[scl_cols] = scaler.transform(valid_df[scl_cols])\n \n train_ds = CSIRODataset(\n train_df, \n transform=train_transform, \n image_size=self.image_size,\n use_date=use_date, \n use_height=use_height, \n use_ndvi=use_ndvi, \n use_state=use_state, \n use_species=use_species\n )\n val_ds = CSIRODataset(\n valid_df, \n transform=valid_transform, \n image_size=self.image_size,\n use_date=use_date, \n use_height=use_height, \n use_ndvi=use_ndvi, \n use_state=use_state, \n use_species=use_species\n )\n \n train_dl = DataLoader(\n train_ds, \n batch_size=self.batch_size, \n num_workers=self.num_workers, \n shuffle=True,\n pin_memory=True\n )\n val_dl = DataLoader(\n val_ds, \n batch_size=self.batch_size, \n num_workers=self.num_workers, \n shuffle=False,\n pin_memory=True\n )\n \n metadata_dim = len(train_ds[0]['metadata'])\n\n model = get_model(metadata_dim=metadata_dim)\n model = model.to(self.device)\n \n print(f\"[Fold {fold+1}] Train size: {len(train_df)} | Valid size: {len(valid_df)}\")\n print(f\"[Fold {fold+1}] Metadata dim: {metadata_dim}\")\n \n history, best_r2, best_loss = self._train_fold(fold, train_dl, val_dl, model, fold_dir)\n \n fold_results.append({\n 'fold': fold + 1,\n 'best_val_r2': best_r2,\n 'best_val_loss': best_loss,\n 'history': history,\n 'metadata_dim': metadata_dim\n })\n \n summary_df = pd.DataFrame([\n {\n 'fold': r['fold'],\n 'best_val_r2': r['best_val_r2'],\n 'best_val_loss': r['best_val_loss'],\n 'metadata_dim': r['metadata_dim']\n }\n for r in fold_results\n ])\n summary_df.to_csv(self.output_dir / 'cv_summary.csv', index=False)\n \n self._print_summary(fold_results)\n self._plot_fold_comparison(fold_results)\n \n return fold_results\n \n def _print_summary(self, fold_results):\n print(f\"\\n{'='*70}\")\n print(\"CROSS-VALIDATION SUMMARY\")\n print(f\"{'='*70}\")\n \n avg_r2 = np.mean([r['best_val_r2'] for r in fold_results])\n std_r2 = np.std([r['best_val_r2'] for r in fold_results])\n avg_loss = np.mean([r['best_val_loss'] for r in fold_results])\n std_loss = np.std([r['best_val_loss'] for r in fold_results])\n \n for result in fold_results:\n print(f\"[Fold {result['fold']}] Val R² = {result['best_val_r2']:.4f}, Val Loss = {result['best_val_loss']:.4f}\")\n \n print(f\"\\n{'─'*70}\")\n print(f\"Average Val R²: {avg_r2:.4f} ± {std_r2:.4f}\")\n print(f\"Average Val Loss: {avg_loss:.4f} ± {std_loss:.4f}\")\n print(f\"{'='*70}\\n\")\n \n def _plot_training_history(self, history, fold, fold_dir):\n fig, axes = plt.subplots(2, 4, figsize=(24, 10))\n fig.suptitle(f'Fold {fold+1} Training History - {self.experiment_name}', fontsize=16)\n \n epochs = range(1, len(history['train_loss']) + 1)\n \n axes[0, 0].plot(epochs, history['train_loss'], 'b-', label='Train Loss', linewidth=2)\n axes[0, 0].plot(epochs, history['val_loss'], 'r-', label='Val Loss', linewidth=2)\n axes[0, 0].set_xlabel('Epoch', fontsize=10)\n axes[0, 0].set_ylabel('Loss', fontsize=10)\n axes[0, 0].set_title('Loss History', fontsize=12, fontweight='bold')\n axes[0, 0].legend()\n axes[0, 0].grid(True, alpha=0.3)\n \n axes[0, 1].plot(epochs, history['train_weighted_r2'], 'b-', label='Train R²', linewidth=2)\n axes[0, 1].plot(epochs, history['val_weighted_r2'], 'r-', label='Val R²', linewidth=2)\n axes[0, 1].set_xlabel('Epoch', fontsize=10)\n axes[0, 1].set_ylabel('Weighted R²', fontsize=10)\n axes[0, 1].set_title('Weighted R² History', fontsize=12, fontweight='bold')\n axes[0, 1].legend()\n axes[0, 1].grid(True, alpha=0.3)\n \n plot_positions = [(0, 2), (0, 3), (1, 0), (1, 1), (1, 2)]\n \n for idx, target in enumerate(self.target_names):\n row, col = plot_positions[idx]\n train_key = f'train_r2_{target}'\n val_key = f'val_r2_{target}'\n \n axes[row, col].plot(epochs, history[train_key], 'b-', label='Train', linewidth=2)\n axes[row, col].plot(epochs, history[val_key], 'r-', label='Val', linewidth=2)\n axes[row, col].set_xlabel('Epoch', fontsize=10)\n axes[row, col].set_ylabel('R²', fontsize=10)\n axes[row, col].set_ylim(-3.0, 1.0)\n axes[row, col].set_title(f'{target} R²', fontsize=12, fontweight='bold')\n axes[row, col].legend()\n axes[row, col].grid(True, alpha=0.3)\n \n ax_waterfall = axes[1, 3]\n \n final_r2_values = [history[f'val_r2_{target}'][-1] for target in self.target_names]\n contributions = [final_r2_values[i] * self.target_weights[self.target_names[i]] \n for i in range(len(self.target_names))]\n final_weighted_r2 = sum(contributions)\n \n cumulative = [0]\n for contrib in contributions:\n cumulative.append(cumulative[-1] + contrib)\n \n colors = ['#fdf0d5', '#c1121f', '#02c39a', '#669bbc', '#9467bd', '#003049']\n \n for i in range(len(self.target_names)):\n ax_waterfall.bar(i, contributions[i], bottom=cumulative[i], \n color=colors[i], edgecolor='black', linewidth=1.5,\n label=f'{self.target_names[i]} (w={self.target_weights[self.target_names[i]]})')\n \n ax_waterfall.text(i, cumulative[i] + contributions[i]/2, \n f'{contributions[i]:.3f}', \n ha='center', va='center', fontsize=8, fontweight='bold')\n \n ax_waterfall.bar(len(self.target_names), final_weighted_r2, \n color='green', edgecolor='black', linewidth=2, \n label=f'Total', alpha=0.7)\n ax_waterfall.text(len(self.target_names), final_weighted_r2/2, \n f'{final_weighted_r2:.3f}', \n ha='center', va='center', fontsize=10, fontweight='bold')\n \n for i in range(len(self.target_names)):\n ax_waterfall.plot([i+0.4, i+1-0.4], [cumulative[i+1], cumulative[i+1]], \n 'k--', linewidth=1, alpha=0.5)\n \n categories = self.target_names + ['Final\\nWeighted R²']\n ax_waterfall.set_xticks(range(len(categories)))\n ax_waterfall.set_xticklabels(categories, rotation=45, ha='right', fontsize=9)\n ax_waterfall.set_ylabel('Weighted R² Contribution', fontsize=10)\n ax_waterfall.set_title('Final Weighted R² Waterfall', fontsize=12, fontweight='bold')\n ax_waterfall.axhline(y=0, color='black', linestyle='-', linewidth=0.8)\n ax_waterfall.grid(True, alpha=0.3, axis='y')\n ax_waterfall.legend(fontsize=7, loc='upper left')\n \n plt.tight_layout()\n plt.savefig(fold_dir / 'training_history.png', dpi=150, bbox_inches='tight')\n plt.close()\n \n def _plot_fold_comparison(self, fold_results):\n fig, axes = plt.subplots(1, 2, figsize=(14, 5))\n fig.suptitle(f'Cross-Validation Results - {self.experiment_name}', fontsize=14, fontweight='bold')\n \n folds = [r['fold'] for r in fold_results]\n r2_scores = [r['best_val_r2'] for r in fold_results]\n losses = [r['best_val_loss'] for r in fold_results]\n \n axes[0].bar(folds, r2_scores, color='steelblue', edgecolor='black', linewidth=1.5)\n axes[0].axhline(y=np.mean(r2_scores), color='red', linestyle='--', \n linewidth=2, label=f'Mean: {np.mean(r2_scores):.4f}')\n axes[0].set_xlabel('Fold', fontsize=12)\n axes[0].set_ylabel('Best Validation R²', fontsize=12)\n axes[0].set_title('Best Validation R² per Fold', fontsize=13, fontweight='bold')\n axes[0].legend()\n axes[0].grid(True, alpha=0.3, axis='y')\n \n for i, (fold, r2) in enumerate(zip(folds, r2_scores)):\n axes[0].text(fold, r2 + 0.01, f'{r2:.4f}', ha='center', va='bottom', fontsize=9)\n \n axes[1].bar(folds, losses, color='coral', edgecolor='black', linewidth=1.5)\n axes[1].axhline(y=np.mean(losses), color='red', linestyle='--', \n linewidth=2, label=f'Mean: {np.mean(losses):.4f}')\n axes[1].set_xlabel('Fold', fontsize=12)\n axes[1].set_ylabel('Best Validation Loss', fontsize=12)\n axes[1].set_title('Best Validation Loss per Fold', fontsize=13, fontweight='bold')\n axes[1].legend()\n axes[1].grid(True, alpha=0.3, axis='y')\n \n for i, (fold, loss) in enumerate(zip(folds, losses)):\n axes[1].text(fold, loss + max(losses)*0.01, f'{loss:.4f}', \n ha='center', va='bottom', fontsize=9)\n \n plt.tight_layout()\n plt.savefig(self.output_dir / 'fold_comparison.png', dpi=150, bbox_inches='tight')\n plt.close()\n \n def summary(self, fold_results):\n n_folds = len(fold_results)\n \n fig = plt.figure(figsize=(18, 4 * n_folds + 5))\n gs = fig.add_gridspec(n_folds + 1, 3, hspace=0.4, wspace=0.3)\n \n fig.suptitle(f'Complete Cross-Validation Summary - {self.experiment_name}', \n fontsize=16, fontweight='bold', y=0.995)\n \n for fold_idx, result in enumerate(fold_results):\n fold = result['fold']\n history = result['history']\n epochs = range(1, len(history['train_loss']) + 1)\n \n ax_loss = fig.add_subplot(gs[fold_idx, 0])\n ax_loss.plot(epochs, history['train_loss'], 'b-', label='Train', linewidth=2, alpha=0.7)\n ax_loss.plot(epochs, history['val_loss'], 'r-', label='Val', linewidth=2, alpha=0.7)\n ax_loss.set_xlabel('Epoch', fontsize=9)\n ax_loss.set_ylabel('Loss', fontsize=9)\n ax_loss.set_title(f'Fold {fold} - Loss (Best: {result[\"best_val_loss\"]:.4f})', \n fontsize=11, fontweight='bold')\n ax_loss.legend(fontsize=8)\n ax_loss.grid(True, alpha=0.3)\n \n ax_r2 = fig.add_subplot(gs[fold_idx, 1])\n ax_r2.plot(epochs, history['train_weighted_r2'], 'b-', label='Train', linewidth=2, alpha=0.7)\n ax_r2.plot(epochs, history['val_weighted_r2'], 'r-', label='Val', linewidth=2, alpha=0.7)\n ax_r2.set_xlabel('Epoch', fontsize=9)\n ax_r2.set_ylabel('Weighted R²', fontsize=9)\n ax_r2.set_title(f'Fold {fold} - Weighted R² (Best: {result[\"best_val_r2\"]:.4f})', \n fontsize=11, fontweight='bold')\n ax_r2.legend(fontsize=8)\n ax_r2.grid(True, alpha=0.3)\n \n ax_waterfall = fig.add_subplot(gs[fold_idx, 2])\n \n final_r2_values = [history[f'val_r2_{target}'][-1] for target in self.target_names]\n contributions = [final_r2_values[i] * self.target_weights[self.target_names[i]] \n for i in range(len(self.target_names))]\n final_weighted_r2 = sum(contributions)\n \n cumulative = [0]\n for contrib in contributions:\n cumulative.append(cumulative[-1] + contrib)\n \n colors = ['#fdf0d5', '#c1121f', '#02c39a', '#669bbc', '#9467bd']\n \n for i in range(len(self.target_names)):\n ax_waterfall.bar(i, contributions[i], bottom=cumulative[i], \n color=colors[i], edgecolor='black', linewidth=1,\n label=f'{self.target_names[i][:10]}')\n \n ax_waterfall.text(i, cumulative[i] + contributions[i]/2, \n f'{contributions[i]:.3f}', \n ha='center', va='center', fontsize=7, fontweight='bold')\n \n ax_waterfall.bar(len(self.target_names), final_weighted_r2, \n color='green', edgecolor='black', linewidth=1.5, \n label='Total', alpha=0.7)\n ax_waterfall.text(len(self.target_names), final_weighted_r2/2, \n f'{final_weighted_r2:.3f}', \n ha='center', va='center', fontsize=8, fontweight='bold')\n \n for i in range(len(self.target_names)):\n ax_waterfall.plot([i+0.4, i+1-0.4], [cumulative[i+1], cumulative[i+1]], \n 'k--', linewidth=0.8, alpha=0.5)\n \n categories = [t[:8] for t in self.target_names] + ['Final']\n ax_waterfall.set_xticks(range(len(categories)))\n ax_waterfall.set_xticklabels(categories, rotation=45, ha='right', fontsize=7)\n ax_waterfall.set_ylabel('Weighted R²', fontsize=9)\n ax_waterfall.set_title(f'Fold {fold} - Final Waterfall', fontsize=11, fontweight='bold')\n ax_waterfall.axhline(y=0, color='black', linestyle='-', linewidth=0.8)\n ax_waterfall.grid(True, alpha=0.3, axis='y')\n ax_waterfall.legend(fontsize=6, loc='upper left')\n \n folds = [r['fold'] for r in fold_results]\n r2_scores = [r['best_val_r2'] for r in fold_results]\n losses = [r['best_val_loss'] for r in fold_results]\n \n ax_loss_comp = fig.add_subplot(gs[n_folds, 0])\n ax_loss_comp.bar(folds, losses, color='coral', edgecolor='black', linewidth=1.5)\n ax_loss_comp.axhline(y=np.mean(losses), color='red', linestyle='--', \n linewidth=2, label=f'Mean: {np.mean(losses):.4f}')\n ax_loss_comp.set_xlabel('Fold', fontsize=10)\n ax_loss_comp.set_ylabel('Best Validation Loss', fontsize=10)\n ax_loss_comp.set_title('Loss Comparison Across Folds', fontsize=11, fontweight='bold')\n ax_loss_comp.legend(fontsize=8)\n ax_loss_comp.grid(True, alpha=0.3, axis='y')\n \n for fold, loss in zip(folds, losses):\n ax_loss_comp.text(fold, loss + max(losses)*0.01, f'{loss:.4f}', \n ha='center', va='bottom', fontsize=8)\n \n ax_r2_comp = fig.add_subplot(gs[n_folds, 1])\n ax_r2_comp.bar(folds, r2_scores, color='steelblue', edgecolor='black', linewidth=1.5)\n ax_r2_comp.axhline(y=np.mean(r2_scores), color='red', linestyle='--', \n linewidth=2, label=f'Mean: {np.mean(r2_scores):.4f}')\n ax_r2_comp.set_xlabel('Fold', fontsize=10)\n ax_r2_comp.set_ylabel('Best Validation R²', fontsize=10)\n ax_r2_comp.set_title('R² Comparison Across Folds', fontsize=11, fontweight='bold')\n ax_r2_comp.legend(fontsize=8)\n ax_r2_comp.grid(True, alpha=0.3, axis='y')\n \n for fold, r2 in zip(folds, r2_scores):\n ax_r2_comp.text(fold, r2 + 0.01, f'{r2:.4f}', \n ha='center', va='bottom', fontsize=8)\n \n ax_table = fig.add_subplot(gs[n_folds, 2])\n ax_table.axis('off')\n \n avg_r2 = np.mean(r2_scores)\n std_r2 = np.std(r2_scores)\n avg_loss = np.mean(losses)\n std_loss = np.std(losses)\n min_r2 = np.min(r2_scores)\n max_r2 = np.max(r2_scores)\n min_loss = np.min(losses)\n max_loss = np.max(losses)\n \n table_data = [\n ['Metric', 'Value'],\n ['─' * 20, '─' * 20],\n ['Mean Val R²', f'{avg_r2:.4f} ± {std_r2:.4f}'],\n ['Min Val R²', f'{min_r2:.4f}'],\n ['Max Val R²', f'{max_r2:.4f}'],\n ['', ''],\n ['Mean Val Loss', f'{avg_loss:.4f} ± {std_loss:.4f}'],\n ['Min Val Loss', f'{min_loss:.4f}'],\n ['Max Val Loss', f'{max_loss:.4f}'],\n ['', ''],\n ['# Folds', f'{n_folds}'],\n ['# Epochs', f'{len(fold_results[0][\"history\"][\"train_loss\"])}'],\n ]\n \n table = ax_table.table(cellText=table_data, cellLoc='left', loc='center',\n colWidths=[0.5, 0.5])\n table.auto_set_font_size(False)\n table.set_fontsize(9)\n table.scale(1, 2)\n \n for i in range(len(table_data)):\n if i == 0:\n table[(i, 0)].set_facecolor('#4472C4')\n table[(i, 1)].set_facecolor('#4472C4')\n table[(i, 0)].set_text_props(weight='bold', color='white')\n table[(i, 1)].set_text_props(weight='bold', color='white')\n elif i % 2 == 0:\n table[(i, 0)].set_facecolor('#E7E6E6')\n table[(i, 1)].set_facecolor('#E7E6E6')\n \n ax_table.set_title('Summary Statistics', fontsize=11, fontweight='bold', pad=20)\n\n plt.show()\n plt.savefig(self.output_dir / 'complete_cv_summary.png', dpi=150, bbox_inches='tight')\n plt.close()\n \n print(f\"\\n{'='*70}\")\n print(\"FINAL CROSS-VALIDATION SUMMARY\")\n print(f\"{'='*70}\")\n print(f\"\\nExperiment: {self.experiment_name}\")\n print(f\"Number of Folds: {n_folds}\")\n print(f\"\\nPer-Fold Results:\")\n print(f\"{'─'*70}\")\n for result in fold_results:\n print(f\"Fold {result['fold']:2d} | Val R²: {result['best_val_r2']:.4f} | Val Loss: {result['best_val_loss']:.4f}\")\n \n print(f\"\\n{'─'*70}\")\n print(f\"Average Validation R²: {avg_r2:.4f} ± {std_r2:.4f}\")\n print(f\"Range: [{min_r2:.4f}, {max_r2:.4f}]\")\n print(f\"\\nAverage Validation Loss: {avg_loss:.4f} ± {std_loss:.4f}\")\n print(f\"Range: [{min_loss:.4f}, {max_loss:.4f}]\")\n print(f\"{'='*70}\\n\")\n print(f\"✓ Complete summary saved to: {self.output_dir / 'complete_cv_summary.png'}\")","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-11-13T13:12:02.953719Z","iopub.execute_input":"2025-11-13T13:12:02.953969Z","iopub.status.idle":"2025-11-13T13:12:03.019889Z","shell.execute_reply.started":"2025-11-13T13:12:02.953953Z","shell.execute_reply":"2025-11-13T13:12:03.019058Z"}},"outputs":[],"execution_count":null},{"id":"d0df3aaa-6cb2-488f-8749-06d43095ace8","cell_type":"markdown","source":"# 🧪 Experiment
","metadata":{}},{"id":"a480c07c-fd86-483c-a4c1-e4354e934fa8","cell_type":"code","source":"experiments = {\n 'baseline': {\n 'use_date': False,\n 'use_height': False,\n 'use_ndvi': False,\n 'use_state': False,\n 'use_species': False\n },\n 'temporal_features': {\n 'use_date': True,\n 'use_height': False,\n 'use_ndvi': False,\n 'use_state': False,\n 'use_species': False\n },\n 'measurement_features': {\n 'use_date': False,\n 'use_height': True,\n 'use_ndvi': True,\n 'use_state': False,\n 'use_species': False\n },\n 'contextual_features': {\n 'use_date': False,\n 'use_height': False,\n 'use_ndvi': False,\n 'use_state': True,\n 'use_species': True\n },\n 'all_features': {\n 'use_date': True,\n 'use_height': True,\n 'use_ndvi': True,\n 'use_state': True,\n 'use_species': True\n },\n}","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-11-13T13:12:03.020622Z","iopub.execute_input":"2025-11-13T13:12:03.020872Z","iopub.status.idle":"2025-11-13T13:12:03.037840Z","shell.execute_reply.started":"2025-11-13T13:12:03.020848Z","shell.execute_reply":"2025-11-13T13:12:03.036928Z"}},"outputs":[],"execution_count":null},{"id":"6552c19b-97cf-4dea-b2cf-0cd489629779","cell_type":"markdown","source":"### Baseline
","metadata":{}},{"id":"902adc5d-531f-4eea-90db-a23f6f5ffc69","cell_type":"code","source":"experiment_name = 'baseline'\ntrainer = CSIROTrainer(experiment_name=experiment_name)\nresults = trainer.train(df, **experiments[experiment_name])","metadata":{"trusted":true,"_kg_hide-output":true,"execution":{"iopub.status.busy":"2025-11-13T13:12:03.038629Z","iopub.execute_input":"2025-11-13T13:12:03.038892Z","iopub.status.idle":"2025-11-13T13:14:50.481473Z","shell.execute_reply.started":"2025-11-13T13:12:03.038866Z","shell.execute_reply":"2025-11-13T13:14:50.480751Z"}},"outputs":[],"execution_count":null},{"id":"428fc66a-3010-4582-a6b4-8fb7eddab852","cell_type":"code","source":"trainer.summary(results)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-11-13T13:14:50.482664Z","iopub.execute_input":"2025-11-13T13:14:50.482862Z","iopub.status.idle":"2025-11-13T13:14:53.151681Z","shell.execute_reply.started":"2025-11-13T13:14:50.482841Z","shell.execute_reply":"2025-11-13T13:14:53.151044Z"}},"outputs":[],"execution_count":null},{"id":"7070862f-62bb-4428-bab4-c2ca462f7e28","cell_type":"markdown","source":"### Temporal Features
","metadata":{}},{"id":"f7389faa-9889-453f-a45d-0ac036e806af","cell_type":"code","source":"experiment_name = 'temporal_features'\ntrainer = CSIROTrainer(experiment_name=experiment_name)\nresults = trainer.train(df, **experiments[experiment_name])","metadata":{"trusted":true,"_kg_hide-output":true,"execution":{"iopub.status.busy":"2025-11-13T13:14:53.152515Z","iopub.execute_input":"2025-11-13T13:14:53.152793Z","iopub.status.idle":"2025-11-13T13:17:39.109463Z","shell.execute_reply.started":"2025-11-13T13:14:53.152769Z","shell.execute_reply":"2025-11-13T13:17:39.108775Z"}},"outputs":[],"execution_count":null},{"id":"dc67371e-68b3-42b8-967f-e05c39d43d6c","cell_type":"code","source":"trainer.summary(results)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-11-13T13:17:39.110840Z","iopub.execute_input":"2025-11-13T13:17:39.111229Z","iopub.status.idle":"2025-11-13T13:17:41.801319Z","shell.execute_reply.started":"2025-11-13T13:17:39.111202Z","shell.execute_reply":"2025-11-13T13:17:41.800700Z"}},"outputs":[],"execution_count":null},{"id":"08adf12e-4826-41cf-ae14-b5f013e0898a","cell_type":"markdown","source":"### Measurement Features
","metadata":{}},{"id":"172bb0f7-197d-49bd-9793-a44a5cf24199","cell_type":"code","source":"experiment_name = 'measurement_features'\ntrainer = CSIROTrainer(experiment_name=experiment_name)\nresults = trainer.train(df, **experiments[experiment_name])","metadata":{"trusted":true,"_kg_hide-output":true,"execution":{"iopub.status.busy":"2025-11-13T13:17:41.802183Z","iopub.execute_input":"2025-11-13T13:17:41.802935Z","iopub.status.idle":"2025-11-13T13:20:28.977009Z","shell.execute_reply.started":"2025-11-13T13:17:41.802914Z","shell.execute_reply":"2025-11-13T13:20:28.976045Z"}},"outputs":[],"execution_count":null},{"id":"51c5a4a2-9dc9-49dc-a26d-bd21fff2c45a","cell_type":"code","source":"trainer.summary(results)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-11-13T13:20:28.978715Z","iopub.execute_input":"2025-11-13T13:20:28.979114Z","iopub.status.idle":"2025-11-13T13:20:31.728491Z","shell.execute_reply.started":"2025-11-13T13:20:28.979088Z","shell.execute_reply":"2025-11-13T13:20:31.727676Z"}},"outputs":[],"execution_count":null},{"id":"0774a3e6-4254-43f4-9d9c-77fffa178056","cell_type":"markdown","source":"### Contextual Features
","metadata":{}},{"id":"2b271e07-98ba-46c4-98b0-edf3633521ef","cell_type":"code","source":"experiment_name = 'contextual_features'\ntrainer = CSIROTrainer(experiment_name=experiment_name)\nresults = trainer.train(df, **experiments[experiment_name])","metadata":{"trusted":true,"_kg_hide-output":true,"execution":{"iopub.status.busy":"2025-11-13T13:20:31.729449Z","iopub.execute_input":"2025-11-13T13:20:31.730131Z","iopub.status.idle":"2025-11-13T13:23:19.307746Z","shell.execute_reply.started":"2025-11-13T13:20:31.730110Z","shell.execute_reply":"2025-11-13T13:23:19.307125Z"}},"outputs":[],"execution_count":null},{"id":"afa18863-dd6d-48f9-80a4-d60cc77c0db1","cell_type":"code","source":"trainer.summary(results)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-11-13T13:23:19.308970Z","iopub.execute_input":"2025-11-13T13:23:19.309202Z","iopub.status.idle":"2025-11-13T13:23:22.033176Z","shell.execute_reply.started":"2025-11-13T13:23:19.309182Z","shell.execute_reply":"2025-11-13T13:23:22.032330Z"}},"outputs":[],"execution_count":null},{"id":"182bfb60-aa04-4c35-b611-629fb2afab84","cell_type":"markdown","source":"### All Features
","metadata":{}},{"id":"8a51efc0-e80f-4f34-baac-af2e13ea6840","cell_type":"code","source":"experiment_name = 'all_features'\ntrainer = CSIROTrainer(experiment_name=experiment_name)\nresults = trainer.train(df, **experiments[experiment_name])","metadata":{"trusted":true,"_kg_hide-output":true,"execution":{"iopub.status.busy":"2025-11-13T13:23:22.034079Z","iopub.execute_input":"2025-11-13T13:23:22.034880Z","iopub.status.idle":"2025-11-13T13:26:12.364199Z","shell.execute_reply.started":"2025-11-13T13:23:22.034857Z","shell.execute_reply":"2025-11-13T13:26:12.363261Z"}},"outputs":[],"execution_count":null},{"id":"1968f83b-ca1c-47ec-94a2-59b57faf3a0a","cell_type":"code","source":"trainer.summary(results)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-11-13T13:26:12.365916Z","iopub.execute_input":"2025-11-13T13:26:12.366239Z","iopub.status.idle":"2025-11-13T13:26:15.158970Z","shell.execute_reply.started":"2025-11-13T13:26:12.366212Z","shell.execute_reply":"2025-11-13T13:26:15.158124Z"}},"outputs":[],"execution_count":null},{"id":"826712ed","cell_type":"markdown","source":"# 📋 Summary
","metadata":{}},{"id":"c534263d-5439-4011-8fea-40d25e993cb7","cell_type":"code","source":"comparison = {\n 'experiment': [],\n 'r2_mean': [],\n 'r2_std': [],\n 'loss_mean': [],\n 'loss_std': [],\n 'use_date': [],\n 'use_height': [],\n 'use_ndvi': [],\n 'use_state': [],\n 'use_species': []\n}\n\nfor experiment, features in experiments.items():\n cv_summary = pd.read_csv(os.path.join(OUTPUT, experiment, 'cv_summary.csv'))\n \n comparison['experiment'].append(experiment)\n \n r2_mean = cv_summary['best_val_r2'].mean()\n r2_std = cv_summary['best_val_r2'].std()\n \n comparison['r2_mean'].append(r2_mean)\n comparison['r2_std'].append(r2_std)\n\n loss_mean = cv_summary['best_val_loss'].mean()\n loss_std = cv_summary['best_val_loss'].std()\n comparison['loss_mean'].append(loss_mean)\n comparison['loss_std'].append(loss_std)\n \n for feature, value in features.items():\n comparison[feature].append(value)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-11-13T13:26:15.159965Z","iopub.execute_input":"2025-11-13T13:26:15.160269Z","iopub.status.idle":"2025-11-13T13:26:15.176263Z","shell.execute_reply.started":"2025-11-13T13:26:15.160248Z","shell.execute_reply":"2025-11-13T13:26:15.175340Z"}},"outputs":[],"execution_count":null},{"id":"e2d95f90-98b7-4605-bffd-9b01540449e8","cell_type":"code","source":"summary_df = pd.DataFrame(comparison).set_index('experiment')\n\nstyled = summary_df.style.background_gradient(\n subset=['r2_mean'],\n cmap='Greens'\n).format(precision=3)\n\ndisplay(styled)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-11-13T13:26:15.177186Z","iopub.execute_input":"2025-11-13T13:26:15.177389Z","iopub.status.idle":"2025-11-13T13:26:15.282058Z","shell.execute_reply.started":"2025-11-13T13:26:15.177372Z","shell.execute_reply":"2025-11-13T13:26:15.281310Z"}},"outputs":[],"execution_count":null},{"id":"8be836f0-521c-46fe-b73b-457994e3f412","cell_type":"markdown","source":"### ⬆️ Upvote if you find it helpful 🤗 ⬆️
","metadata":{}}]}