{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"name":"python","version":"3.12.12","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"},"kaggle":{"accelerator":"none","dataSources":[],"dockerImageVersionId":31234,"isInternetEnabled":false,"language":"python","sourceType":"notebook","isGpuEnabled":false}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# 📊 CSIRO Biomass Competition Metric\n\n**Pooled Weighted R² Score** matching competition evaluation\n\n---\n\n## 🔌 Import Instructions\n\n**In your main notebook, use this import:**\n\n```python\nimport sys\nsys.path.insert(0, '/kaggle/usr/lib/weighted_r_score')\nfrom weighted_r_score import calc_competition_metric, validate_predictions\n```\n\n---\n\n## Overview\n\nCompetition uses a **single pooled R²** across all predictions, not a weighted average of per-target R² scores.\n\n**Mathematical formulation:**\n- Treats (N, 5) predictions as **5N rows** with target-specific weights\n- Single weighted mean: ȳ_w = Σ(w_j × y_j) / Σ(w_j)\n- R²_weighted = 1 − SS_res / SS_tot\n\n## Target Weights\n\n| Target | Weight | Description |\n|--------|--------|-------------|\n| **Dry_Green_g** | 0.1 | Green grass biomass |\n| **Dry_Clover_g** | 0.1 | Clover biomass |\n| **Dry_Dead_g** | 0.1 | Dead material biomass |\n| **GDM_g** | 0.2 | Green Dry Matter (Green + Clover) |\n| **Dry_Total_g** | 0.5 | Total biomass (all components) |\n\n**Mathematical relationships enforced:**\n- GDM = Dry_Green + Dry_Clover\n- Dry_Total = Dry_Green + Dry_Clover + Dry_Dead\n\n## Key Functions\n\n### `calc_competition_metric(outputs, targets, ...)`\n**Main scoring function**\n- **Inputs:** \n  - `outputs`: (N, 3) predictions [Green, Clover, Dead]\n  - `targets`: (N, 3) or (N, 5) ground truth\n- **Returns:** Pooled weighted R² score\n- **Auto-derives** GDM and Total from 3 base components\n- **Options:**\n  - `clamp_negatives=True`: Clamp biomass to [0, ∞)\n  - `verify_gt=True`: Check relationship validity\n  - `force_relationships=True`: Override provided GDM/Total\n\n### `validate_predictions(outputs, targets, ...)`\n**Full diagnostic report**\n- Competition R² (pooled)\n- Per-target R² (unweighted, for debugging)\n- Ground truth relationship verification\n- Negative prediction warnings\n\n## Usage Examples\n\n```python\n# Import\nfrom weighted_r_score.weighted_r_score import calc_competition_metric, validate_predictions\n\n# Training validation (verbose warnings)\nval_r2 = calc_competition_metric(\n    val_preds,      # (N, 3) \n    val_targets,    # (N, 3) or (N, 5)\n    clamp_negatives=False,  # Warn about negatives\n    verify_gt=False         # Skip redundant checks\n)\n\n# Final evaluation (full report)\nresults = validate_predictions(\n    test_preds, \n    test_targets,\n    clamp_negatives=True,  # Ensure valid biomass\n    verbose=True           # Print breakdown\n)\nprint(f\"Competition R²: {results['competition_r2']:.4f}\")\n```","metadata":{}},{"cell_type":"code","source":"# CSIRO Biomass Competition Metric - Pooled Weighted R²\n\n\nimport numpy as np\nimport torch\nimport warnings\n\n# Constants\nTARGET_ORDER = ['Dry_Green_g', 'Dry_Clover_g', 'Dry_Dead_g', 'GDM_g', 'Dry_Total_g']\nTARGET_WEIGHTS = np.array([0.1, 0.1, 0.1, 0.2, 0.5])\n\n# Numerical thresholds\nEPSILON_NUM = 1e-12   # Numerical stability (zero variance detection)\nEPSILON_REL = 1e-6    # Relationship validation tolerance (grams)\n\n\ndef weighted_r2_score(y_true, y_pred, target_weights=None, epsilon=EPSILON_NUM):\n    \"\"\"\n    Pooled weighted R² as per competition evaluation.\n    \n    Treats (N, 5) arrays as 5N rows with per-target weights:\n    - R²_w = 1 - SS_res / SS_tot\n    - Single weighted mean across all targets\n    \n    Args:\n        y_true: (N, 5) ground truth [Green, Clover, Dead, GDM, Total]\n        y_pred: (N, 5) predictions in same order\n        target_weights: (5,) weights, default [0.1, 0.1, 0.1, 0.2, 0.5]\n        epsilon: Threshold for zero-variance detection\n        \n    Returns:\n        float: Weighted R² score\n    \"\"\"\n    if target_weights is None:\n        target_weights = TARGET_WEIGHTS\n    \n    # Input validation\n    target_weights = np.asarray(target_weights)\n    assert y_true.ndim == 2 and y_pred.ndim == 2, \"Inputs must be 2D arrays\"\n    assert y_true.shape == y_pred.shape, f\"Shape mismatch: {y_true.shape} vs {y_pred.shape}\"\n    assert y_true.shape[1] == 5, f\"Expected 5 targets, got {y_true.shape[1]}\"\n    assert target_weights.shape == (5,), f\"Expected 5 weights, got {target_weights.shape}\"\n    \n    # Flatten to 5N rows\n    N = y_true.shape[0]\n    y_true_flat = y_true.reshape(-1)\n    y_pred_flat = y_pred.reshape(-1)\n    weights_flat = np.tile(target_weights, N)\n    \n    # Weighted mean (single scalar across all targets)\n    w_sum = np.sum(weights_flat * y_true_flat)\n    w_total = np.sum(weights_flat)\n    weighted_mean = w_sum / w_total\n    \n    # Weighted sums of squares\n    residuals = y_true_flat - y_pred_flat\n    deviations = y_true_flat - weighted_mean\n    \n    ss_res = np.sum(weights_flat * residuals ** 2)\n    ss_tot = np.sum(weights_flat * deviations ** 2)\n    \n    # Handle edge cases\n    if ss_tot < epsilon:\n        if ss_res < epsilon:\n            return 1.0  # Perfect predictions on constant data\n        else:\n            warnings.warn(\"Zero variance in targets but non-zero residuals\")\n            return -np.inf\n    \n    return 1.0 - (ss_res / ss_tot)\n\n\ndef _verify_relationships(y_true, tolerance=EPSILON_REL, verbose=False):\n    \"\"\"\n    Check mathematical relationships: GDM = Green+Clover, Total = Green+Clover+Dead\n    \n    Args:\n        y_true: (N, 5) array [Green, Clover, Dead, GDM, Total]\n        tolerance: Maximum allowed error in grams\n        verbose: If True, print success message\n        \n    Raises:\n        Warning if relationships violated\n    \"\"\"\n    green, clover, dead, gdm, total = y_true.T\n    \n    gdm_expected = green + clover\n    total_expected = green + clover + dead\n    \n    gdm_error = np.abs(gdm - gdm_expected).max()\n    total_error = np.abs(total - total_expected).max()\n    \n    if gdm_error > tolerance or total_error > tolerance:\n        warnings.warn(\n            f\"Ground truth relationships violated! \"\n            f\"GDM error: {gdm_error:.6f}g, Total error: {total_error:.6f}g. \"\n            f\"Consider force_relationships=True\"\n        )\n    elif verbose:\n        print(f\"✓ Ground truth relationships verified (max error: {max(gdm_error, total_error):.2e}g)\")\n\n\ndef _check_negative_predictions(y_pred, clamp=False):\n    \"\"\"\n    Check for physically impossible negative biomass predictions.\n    \n    Args:\n        y_pred: (N, 5) predictions\n        clamp: If True, clamp to 0 (modifies in-place)\n        \n    Returns:\n        y_pred: Clamped array if clamp=True, otherwise unchanged\n        \n    Warns if negatives found\n    \"\"\"\n    neg_mask = y_pred < 0\n    n_neg = neg_mask.sum()\n    \n    if n_neg > 0:\n        min_val = y_pred[neg_mask].min()\n        n_samples = neg_mask.any(axis=1).sum()\n        \n        warnings.warn(\n            f\"Found {n_neg} negative predictions across {n_samples} samples \"\n            f\"(min: {min_val:.3f}g). Biomass must be ≥ 0. \"\n            f\"{'Clamping to 0.' if clamp else 'Consider clamping or model constraints.'}\"\n        )\n        \n        if clamp:\n            y_pred = np.maximum(0, y_pred)\n    \n    return y_pred\n\n\ndef calc_competition_metric(outputs, targets, \n                           force_relationships=False, \n                           verify_gt=True,\n                           clamp_negatives=False,\n                           verbose=False):\n    \"\"\"\n    Convert 3-output predictions to 5-target format and calculate pooled R²\n    \n    Args:\n        outputs: (N, 3) predictions [Green, Clover, Dead]\n        targets: (N, 3) or (N, 5) ground truth\n                 If (N, 3): GDM and Total derived automatically\n                 If (N, 5): Uses provided values unless force_relationships=True\n        force_relationships: Always derive GDM/Total from components\n        verify_gt: Check mathematical relationships in ground truth\n        clamp_negatives: Clamp predictions to [0, ∞) before scoring\n        verbose: Print verification messages\n        \n    Returns:\n        float: Pooled weighted R² score\n    \"\"\"\n    # Convert to numpy\n    if isinstance(outputs, torch.Tensor):\n        outputs = outputs.detach().cpu().numpy()\n    if isinstance(targets, torch.Tensor):\n        targets = targets.detach().cpu().numpy()\n    \n    assert outputs.shape[1] == 3, f\"Outputs must be (N, 3), got {outputs.shape}\"\n    \n    # Derive 5-target predictions: [Green, Clover, Dead, GDM, Total]\n    # Clamp FIRST if requested\n    green, clover, dead = outputs.T\n    if clamp_negatives:\n        green = np.maximum(0, green)\n        clover = np.maximum(0, clover)\n        dead = np.maximum(0, dead)\n\n    # THEN derive relationships from clamped values\n    y_pred = np.column_stack([\n        green,\n        clover,\n        dead,\n        green + clover,              # Now uses clamped values\n        green + clover + dead        # Now uses clamped values\n    ])\n    \n    # Build ground truth\n    if targets.shape[1] == 3:\n        # Derive GDM and Total\n        y_true = np.column_stack([\n            targets[:, 0],\n            targets[:, 1],\n            targets[:, 2],\n            targets[:, 0] + targets[:, 1],\n            targets.sum(axis=1)\n        ])\n    elif targets.shape[1] == 5:\n        if force_relationships:\n            # Override provided GDM/Total with derived values\n            y_true = np.column_stack([\n                targets[:, 0],\n                targets[:, 1],\n                targets[:, 2],\n                targets[:, 0] + targets[:, 1],\n                targets[:, 0] + targets[:, 1] + targets[:, 2]\n            ])\n            if verbose:\n                print(\"⚠️  force_relationships=True: using derived GDM/Total\")\n        else:\n            y_true = targets.copy()\n            if verify_gt:\n                _verify_relationships(y_true, verbose=verbose)\n    else:\n        raise ValueError(f\"targets must be (N, 3) or (N, 5), got {targets.shape}\")\n    \n    return weighted_r2_score(y_true, y_pred)\n\n\ndef validate_predictions(outputs, targets, \n                        clamp_negatives=False,\n                        verbose=True):\n    \"\"\"\n    Full validation with per-target R² breakdown for debugging\n    \n    Args:\n        outputs: (N, 3) predictions [Green, Clover, Dead]\n        targets: (N, 3) or (N, 5) ground truth\n        clamp_negatives: Clamp predictions to [0, ∞)\n        verbose: Print detailed breakdown\n    \n    Returns:\n        dict: {\n            'competition_r2': pooled weighted R²,\n            'per_target_r2': individual R² scores (unweighted)\n        }\n    \"\"\"\n    # Competition metric\n    comp_r2 = calc_competition_metric(\n        outputs, targets, \n        clamp_negatives=clamp_negatives,\n        verify_gt=verbose,\n        verbose=verbose\n    )\n    \n    # Build 5-target arrays\n    if isinstance(outputs, torch.Tensor):\n        outputs = outputs.detach().cpu().numpy()\n    if isinstance(targets, torch.Tensor):\n        targets = targets.detach().cpu().numpy()\n    \n    # Clamp FIRST if requested\n    green, clover, dead = outputs.T\n    if clamp_negatives:\n        green = np.maximum(0, green)\n        clover = np.maximum(0, clover)\n        dead = np.maximum(0, dead)\n    \n    # THEN derive relationships from clamped values\n    y_pred = np.column_stack([\n        green,\n        clover,\n        dead,\n        green + clover,              # Now uses clamped values\n        green + clover + dead        # Now uses clamped values\n    ])\n    \n    if targets.shape[1] == 3:\n        y_true = np.column_stack([\n            targets[:, 0], targets[:, 1], targets[:, 2],\n            targets[:, 0] + targets[:, 1],\n            targets.sum(axis=1)\n        ])\n    else:\n        y_true = targets\n    \n    # Per-target R² (for debugging)\n    per_target = {}\n    for i, name in enumerate(TARGET_ORDER):\n        ss_res = np.sum((y_true[:, i] - y_pred[:, i]) ** 2)\n        ss_tot = np.sum((y_true[:, i] - y_true[:, i].mean()) ** 2)\n        r2 = 1 - (ss_res / ss_tot) if ss_tot > EPSILON_NUM else (1.0 if ss_res < EPSILON_NUM else -np.inf)\n        per_target[name] = r2\n    \n    if verbose:\n        print(f\"\\n{'='*60}\")\n        print(f\"VALIDATION METRICS\")\n        print(f\"{'='*60}\")\n        print(f\"Competition R² (pooled): {comp_r2:.4f}\")\n        print(f\"\\nPer-target R² (unweighted, for debugging):\")\n        for name in TARGET_ORDER:\n            weight = TARGET_WEIGHTS[list(TARGET_ORDER).index(name)]\n            print(f\"  {name:15s}: {per_target[name]:7.4f} (weight: {weight:.1f})\")\n        print(f\"{'='*60}\\n\")\n    \n    return {\n        'competition_r2': comp_r2,\n        'per_target_r2': per_target\n    }","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-01-02T13:31:22.152358Z","iopub.execute_input":"2026-01-02T13:31:22.152578Z","iopub.status.idle":"2026-01-02T13:31:26.867002Z","shell.execute_reply.started":"2026-01-02T13:31:22.152555Z","shell.execute_reply":"2026-01-02T13:31:26.865738Z"}},"outputs":[],"execution_count":null}]}