{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"},"kaggle":{"accelerator":"none","dataSources":[{"sourceId":112509,"databundleVersionId":14254895,"sourceType":"competition"}],"dockerImageVersionId":31234,"isInternetEnabled":true,"language":"python","sourceType":"notebook","isGpuEnabled":false}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# 🌱 CSIRO Image2Biomass — Stable & High-Score Lightweight Pipeline - Method & Rationale\n\nA stability-first, submission-driven machine learning notebook for the  \n**CSIRO – Image2Biomass Prediction (Kaggle Competition)**.\n\n🚀 **Final public score: ~0.41**  \n🧱 **Zero failed submissions**  \n🪶 **No heavy CNNs — lightweight models only**\n","metadata":{}},{"cell_type":"markdown","source":"> ### ⚠️ **Competition Status Notice**\n> **This competition is still ongoing. Therefore, this notebook intentionally avoids sharing any rule-violating content. The focus is on methodology and engineering insights; as such, this is a discussion-only template and is NOT designed to be executed as a standalone submission.**\n","metadata":{}},{"cell_type":"markdown","source":"## 📌 Project Philosophy\n\nThis project intentionally prioritizes **robustness and submission reliability** over model complexity.\n\nInstead of starting with deep CNN architectures, I focused on building a pipeline that:\n\n- never breaks on Kaggle Batch (`Save & Run All`)\n- survives hidden test variations\n- guarantees a valid submission every time\n- still pushes the score as high as possible using lightweight models\n\nOnly after the submission logic was proven stable did I optimize for performance.\n","metadata":{}},{"cell_type":"markdown","source":"## 🚀 Key Strengths\n\n### 🧱 Stability-first engineering\n- Submission built strictly from `sample_submission.csv`\n- No dependency on execution order or hidden state\n- Explicit handling of:\n  - missing columns\n  - NaN / inf predictions\n  - target-name mismatches\n  - per-row fallback logic\n\n### 🪶 Lightweight but competitive modeling\n- One model per `target_name`\n- RandomForest / ExtraTrees regressors\n- Features restricted to **train–test intersection**\n- Group-aware logic by `image_id`\n\n### 📈 Iterative score improvement (real attempts)\n- Median-only baseline (~0.23)\n- CNN experiments → instability & regressions\n- Final lightweight ensemble → **~0.41**\n","metadata":{}},{"cell_type":"markdown","source":"## 🔬 What I Tried (and What I Learned)\n\n1. **Median-only baseline**\n   - Guaranteed submission success\n   - Served as a diagnostic and fallback layer\n\n2. **CNN-based experiments**\n   - Encountered multiple hard failure modes:\n     - 3 vs 5 target mismatch\n     - blocked pretrained weights\n     - checkpoint shape incompatibilities\n     - inf / NaN predictions\n   - Result: unstable submissions and score drops\n\n3. **Final strategy**\n   - Lightweight tabular models\n   - Strict feature symmetry\n   - Submission-first logic\n   - Robust fallback on every row\n\nThis shift is what unlocked both **stability and score gains**.\n","metadata":{}},{"cell_type":"code","source":"# =========================================================\n# Environment & Imports\n# =========================================================\n\nimport os\nimport numpy as np\nimport pandas as pd\n\nfrom sklearn.ensemble import RandomForestRegressor, ExtraTreesRegressor\nfrom sklearn.model_selection import GroupKFold\n\nDATA_DIR = \"/kaggle/input/csiro-biomass\"\nWORK_DIR = \"/kaggle/working\"\n\nprint(\"KAGGLE_KERNEL_RUN_TYPE:\", os.environ.get(\"KAGGLE_KERNEL_RUN_TYPE\"))\n\nassert os.path.exists(DATA_DIR)\n","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# =========================================================\n# Load data\n# =========================================================\n\ntrain_df = pd.read_csv(f\"{DATA_DIR}/train.csv\")\ntest_df  = pd.read_csv(f\"{DATA_DIR}/test.csv\")\n\nprint(\"Train shape:\", train_df.shape)\nprint(\"Test shape :\", test_df.shape)\n","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## 🧩 Target-wise Modeling Strategy\n\nEach biomass variable behaves differently.\n\nInstead of forcing a single multi-output model,  \nI train **one regressor per `target_name`**, which:\n\n- improves stability\n- simplifies debugging\n- avoids cross-target interference\n","metadata":{}},{"cell_type":"code","source":"# =========================================================\n# Safe feature selection (train-test symmetric)\n# =========================================================\n\nSAFE_FEATURES = [\n    \"Pre_GSHH_NDVI\",\n    \"Height_Ave_cm\"\n]\n\n# Only keep features that actually exist in BOTH datasets\nSAFE_FEATURES = [f for f in SAFE_FEATURES if f in train_df.columns and f in test_df.columns]\n\nprint(\"Using features:\", SAFE_FEATURES)\n","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# =========================================================\n# Train one model per target_name\n# =========================================================\n\nmodels = {}\nmedian_map = train_df.groupby(\"target_name\")[\"target\"].median().to_dict()\n\nfor tgt in train_df[\"target_name\"].unique():\n    sub = train_df[train_df[\"target_name\"] == tgt].copy()\n\n    X = sub[SAFE_FEATURES].fillna(sub[SAFE_FEATURES].median())\n    y = sub[\"target\"]\n    groups = sub[\"sample_id\"].str.split(\"__\").str[0]\n\n    model = ExtraTreesRegressor(\n        n_estimators=300,\n        max_depth=None,\n        min_samples_leaf=2,\n        random_state=42,\n        n_jobs=-1\n    )\n\n    model.fit(X, y)\n    models[tgt] = model\n\n    print(f\"Trained model for {tgt} | samples={len(sub)}\")\n","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## 📤 Submission Construction (Critical Cell)\n\nThis submission logic strictly follows the **community-proven median-merge pattern**,\nwith CNN-style predictions replaced by lightweight model outputs.\n\nDesign guarantees:\n- correct row count\n- correct column names\n- numeric-only predictions\n- no NaN / inf on hidden test\n","metadata":{}},{"cell_type":"code","source":"# =========================================================\n# FINAL SUBMISSION — Batch-safe & robust\n# =========================================================\n\nsample_sub = pd.read_csv(f\"{DATA_DIR}/sample_submission.csv\")\nsubmission = sample_sub.copy()\n\npreds = []\n\nfor _, row in submission.iterrows():\n    sid = row[\"sample_id\"]\n    tgt = sid.split(\"__\")[1]\n\n    # fallback default\n    pred = median_map.get(tgt, 0.0)\n\n    if tgt in models:\n        row_test = test_df[test_df[\"sample_id\"] == sid]\n\n        if not row_test.empty:\n            X = row_test[SAFE_FEATURES].copy()\n            X = X.fillna(X.median())\n\n            try:\n                pred_model = models[tgt].predict(X)[0]\n                if np.isfinite(pred_model):\n                    pred = float(pred_model)\n            except Exception:\n                pass\n\n    preds.append(pred)\n\nsubmission[\"target\"] = preds\n\n# final sanitation\nsubmission[\"target\"] = submission[\"target\"].astype(float)\nsubmission[\"target\"] = submission[\"target\"].replace([np.inf, -np.inf], np.nan)\nsubmission[\"target\"] = submission[\"target\"].fillna(submission[\"target\"].median())\nsubmission[\"target\"] = submission[\"target\"].clip(0, 200)\n\n# safety checks\nassert submission.shape[0] == sample_sub.shape[0]\nassert submission[\"target\"].notna().all()\n\nsubmission.to_csv(\"submission.csv\", index=False)\n\nprint(\"✅ submission.csv saved\")\nsubmission.head()\n","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## 🧠 Final Notes\n\n- This notebook prioritizes **engineering reliability** over raw model complexity\n- The final solution achieved a **~0.41 public score**\n- The competition is ongoing, and future work may explore:\n  - CNN-based feature extraction\n  - hybrid image + tabular models\n  - ensembling strategies\n\nBut any future improvements will follow the same principle:\n**submission stability first, performance second.**\n","metadata":{}}]}