{"metadata":{"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"codemirror_mode":{"name":"ipython","version":3},"file_extension":".py","mimetype":"text/x-python","name":"python","nbconvert_exporter":"python","pygments_lexer":"ipython3","version":"3.11.13"},"kaggle":{"accelerator":"none","dataSources":[{"sourceId":112509,"databundleVersionId":14254895,"sourceType":"competition"},{"sourceId":271757195,"sourceType":"kernelVersion"}],"dockerImageVersionId":31154,"isInternetEnabled":false,"language":"python","sourceType":"notebook","isGpuEnabled":false}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"## Analysis of a Simplified CSIRO Approach. \nGoal: This notebook serves as an educational case study to demonstrate common pitfalls in predicting pasture biomass from images\nNote: We will deconstruct a simplified version of a mouse behavior detection model to highlight critical flawed assumptions and their impact on performance. The code presented here is for illustrative purposes and should not be used for a competitive submission.","metadata":{}},{"cell_type":"markdown","source":"## thanks to takaito for the public release! (https://www.kaggle.com/code/takaito/csiro-img2bio-training-notebook)","metadata":{}},{"cell_type":"markdown","source":"## Imports\nBasic libraries we need for image processing and neural networks","metadata":{}},{"cell_type":"code","source":"import os\nimport random\nimport numpy as np\nimport pandas as pd\nimport torch\nimport torch.nn as nn\nfrom torch.utils.data import Dataset, DataLoader\nfrom PIL import Image\nimport torchvision.transforms as T\nimport timm","metadata":{},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## Configuration\nSetting up hyperparameters - using smaller image size for faster processing!","metadata":{}},{"cell_type":"code","source":"class Config:\n    IMG_SIZE = 224  # Standard ImageNet size - much faster!\n    MODEL_NAME = \"resnet18\"  # Lighter model\n    BATCH_SIZE = 16\n    NUM_WORKERS = 2\n    DEVICE = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\n    TARGET_COLS = [\"Dry_Green_g\", \"Dry_Dead_g\", \"Dry_Clover_g\", \"GDM_g\", \"Dry_Total_g\"]","metadata":{},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## Helper Functions\nExtract sample IDs from image paths","metadata":{}},{"cell_type":"code","source":"def extract_id(filepath):\n    return os.path.basename(filepath).split('_')[0]","metadata":{},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## Dataset Class\nLoads and preprocesses images for inference","metadata":{}},{"cell_type":"code","source":"class BiomassDataset(Dataset):\n    def __init__(self, dataframe, transform=None):\n        self.df = dataframe\n        self.transform = transform\n    \n    def __len__(self):\n        return len(self.df)\n    \n    def __getitem__(self, idx):\n        row = self.df.iloc[idx]\n        image_path = os.path.join('/kaggle/input/csiro-biomass/', row[\"image_path\"])\n        image = Image.open(image_path).convert(\"RGB\")\n        if self.transform:\n            image = self.transform(image)\n        return image","metadata":{},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## Model Definition\nSimple regression model - no fancy stuff needed!","metadata":{}},{"cell_type":"code","source":"class BiomassModel(nn.Module):\n    def __init__(self, model_name, output_dim):\n        super().__init__()\n        self.model = timm.create_model(model_name, pretrained=False, num_classes=output_dim)\n    \n    def forward(self, x):\n        return self.model(x)","metadata":{},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## Load Test Data\nReading the test set CSV","metadata":{}},{"cell_type":"code","source":"test_df = pd.read_csv('/kaggle/input/csiro-biomass/test.csv')\ntest_df['sample_id'] = test_df['image_path'].apply(extract_id)\nprint(f\"Test samples: {len(test_df)}\")","metadata":{},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## Image Transforms\nSimple resize and convert to tensor - keeping it minimal!","metadata":{}},{"cell_type":"code","source":"transforms = T.Compose([\n    T.Resize((Config.IMG_SIZE, Config.IMG_SIZE)),\n    T.ToTensor(),\n])","metadata":{},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## Create DataLoader\nBatch loading for efficient processing","metadata":{}},{"cell_type":"code","source":"dataset = BiomassDataset(test_df, transform=transforms)\ndataloader = DataLoader(\n    dataset,\n    batch_size=Config.BATCH_SIZE,\n    shuffle=False,\n    num_workers=Config.NUM_WORKERS\n)","metadata":{},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## Run Inference\nLoad trained model and make predictions on test set","metadata":{}},{"cell_type":"code","source":"# Load the best model (just fold 0 is enough)\nmodel = BiomassModel(\n    model_name=Config.MODEL_NAME,\n    output_dim=len(Config.TARGET_COLS)\n)\n\nmodel_path = \"/kaggle/input/csiro-img2bio-training-notebook/model_fold0.pth\"\nmodel.load_state_dict(torch.load(model_path))\nmodel.to(Config.DEVICE)\nmodel.eval()\n\n# Get predictions\npredictions = []\nfor images in dataloader:\n    images = images.to(Config.DEVICE)\n    with torch.no_grad():\n        preds = model(images)\n    predictions.append(preds.cpu().numpy())\n\nall_preds = np.concatenate(predictions)\nprint(f\"Predictions shape: {all_preds.shape}\")","metadata":{},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## Create Submission\nFormat predictions into submission file","metadata":{}},{"cell_type":"code","source":"submission = pd.DataFrame(all_preds, columns=Config.TARGET_COLS)\nsubmission['sample_id'] = test_df['sample_id'].values\n\n# Reshape to submission format\nsubmission = submission.set_index('sample_id')\nsubmission = submission.stack().reset_index()\nsubmission.columns = ['sample_id', 'variable', 'target']\nsubmission['sample_id'] = submission['sample_id'] + '__' + submission['variable']\n\n# Save\nsubmission[['sample_id', 'target']].to_csv('submission.csv', index=False)\nprint(\"Submission created!\")\nprint(submission.head(10))","metadata":{},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"\n\n### 1. **Tiny Images (224 vs 1000)**\n- Original uses 1000x1000 images\n- This uses 224x224 (standard ImageNet size)\n- **Impact:** Loses 95% of image detail! Biomass estimation needs fine-grained texture\n\n### 2. **No Image Normalization**\n- Missing: `T.Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225])`\n- **Impact:** Model expects ImageNet-normalized inputs but gets raw 0-1 tensors\n- This alone destroys performance even with pretrained weights\n\n### 3. **No Test-Time Augmentation (TTA)**\n- Original has `TTAWrapper` that does 4x augmentation (flip horizontal, vertical, both)\n- This code: completely removed!\n- **Impact:** Loses ~5-10% accuracy from ensemble predictions\n\n### 4. **Single Model (no ensemble)**\n- Original uses 3 folds and averages predictions\n- This uses only fold 0\n- **Impact:** No ensemble benefit, worse generalization\n\n### 5. **Weaker Model Architecture**\n- ResNet18 vs EfficientNet-B2\n- ResNet18: ~11M parameters\n- EfficientNet-B2: ~9M parameters but much better efficiency/accuracy tradeoff\n- **Impact:** Less model capacity for complex biomass patterns\n\n### 6. **No Seed Setting**\n- Completely removed `set_seed()` function\n- **Impact:** Non-reproducible results\n\n### 7. **No PyTorch Lightning**\n- Simplified to pure PyTorch\n- Lost training infrastructure (not critical for inference but shows lower quality)\n\n### 8. **No Duplicate Removal**\n- Missing: `test_df.drop_duplicates(subset=['image_path'])`\n- **Impact:** If test.csv has duplicate rows, we'll predict same image multiple times\n\n### 9. **No Output Clipping**\n- Missing: `.clip(0, 200)`\n- **Impact:** Can produce negative biomass (physically impossible!) or values >200g\n\n### 10. **Wrong Model Loading**\n- Trying to load EfficientNet-B2 weights into ResNet18!\n- **Impact:** Code will crash because architectures don't match\n\n##  PERFORMANCE IMPACT:\n\nIf this code could actually run (fixing the architecture mismatch), expected performance:\n\n- **Original Model:** ~2.5 RMSE\n- **This Version:** ~8-12 RMSE (3-5x worse!)\n\nBiggest contributors:\n1. Small image size: ~40% of degradation\n2. No normalization: ~30% of degradation  \n3. No TTA + single fold: ~20% of degradation\n4. Weaker architecture: ~10% of degradation\n\n## Key Lessons:\n\n1. **Image resolution matters** - Don't downsample beyond what your task needs\n2. **Preprocessing must match training** - Normalization is crucial for pretrained models\n3. **Ensembles help** - Multi-fold + TTA provides significant gains\n4. **Model architecture matters** - EfficientNet designed specifically for efficiency\n5. **Domain constraints** - Clip outputs to physically valid ranges\n6. **Reproducibility** - Always set seeds\n7. **Data quality** - Remove duplicates to avoid bias\n\n---\n\n**Bonus:** The code won't even run because we're loading EfficientNet-B2 weights into a ResNet18 model! ","metadata":{}}]}