{"metadata":{"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"name":"python","version":"3.12.12","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"},"kaggle":{"accelerator":"none","dataSources":[{"sourceType":"competition","sourceId":10038,"databundleVersionId":180088}],"dockerImageVersionId":31328,"isInternetEnabled":true,"language":"python","sourceType":"notebook","isGpuEnabled":false}},"nbformat_minor":4,"nbformat":4,"cells":[{"id":"6af656f8","cell_type":"markdown","source":"# XGBoost + LightGBM Ensemble Forecasting for E-Commerce KPIs\n## Predicting Weekly Visits, Orders, Revenue & CVR 13 Weeks Ahead \n### Google Analytics Customer Revenue Prediction\n\n---\n\n### Overview\n\nThis notebook builds a **portfolio case study** for a short-horizon forecasting approach for four key e-commerce KPIs sourced from Google Analytics:\n\n- **Visits** — total weekly sessions\n- **Orders** — total weekly transactions\n- **Revenue** — total weekly transaction value (USD)\n- **CVR** — conversion rate (orders ÷ visits)\n\nThe pipeline uses direct multi-step forecasting with gradient-boosted trees, plus walk-forward backtesting to simulate how the model would behave in real use.\n\n### What This Notebook Demonstrates\n\n- Building a forecasting dataset from raw Google Analytics export data\n- Engineering lag, rolling, and calendar/holiday features for weekly series\n- Validating forecasts with a leakage-aware walk-forward backtest\n- Comparing **XGBoost** and **LightGBM** on identical data and features\n- Stress-testing performance on both a **stable** period and a **spike-heavy** period\n- Producing a 13-week forward forecast that a business team could review using a ensemble forecast with weights applied to XGBoost and LightGBM results\n\n---\n\n### Methodology\n\n| Design choice | Decision | Reason |\n|---|---|---|\n| Forecasting strategy | Direct multi-step | No recursive error accumulation |\n| Models compared | XGBoost vs LightGBM | Both gradient boosted trees, similar architecture |\n| Validation | Walk-forward, 8 folds | Honest out-of-sample, no look-ahead |\n| Test window | Last 13 weeks held out | Never seen during any training fold |\n| Horizon | 13 weeks | Operationally useful planning window |\n| Min lag | 8 weeks | Safe for all horizon steps, no leakage |\n\n---\n\n### Key Results — Holdout Test Window\n\nAll results below are from the **held-out test window only** — the last 13 weeks never seen during any training fold.\n\n| | | Jan–Apr 2018 | | Sep–Dec 2017 | |\n|---|---|---|---|---|---|\n| **KPI** | **Model** | **MAE** | **MAPE** | **MAE** | **MAPE** |\n| Visits | XGBoost | 4.31 | 0.02% | 14.21 | 0.06% |\n| | LightGBM | 25.18 | 0.12% | 72.75 | 0.27% |\n| Orders | XGBoost | 0.13 | 0.06% | 0.14 | 0.08% |\n| | LightGBM | 0.55 | 0.25% | 0.10 | 0.07% |\n| Revenue (USD) | XGBoost | 13.31 | 0.07% | 29.54 | 0.19% |\n| | LightGBM | 138.64 | 0.70% | 5.83 | 0.04% |\n| CVR | XGBoost | — | 5.78% | — | 5.56% |\n| | LightGBM | — | 0.54% | — | 0.07% |\n\n> XGBoost wins consistently on visits. Orders and revenue split by period — XGBoost in Jan-Apr, LightGBM in Sep-Dec. LightGBM wins CVR in both periods. Neither model dominates cleanly across all conditions, suggesting an ensemble would outperform either alone.\n\n---\n\n### Baseline Comparison\n\nXGBoost outperformed last-value, 4-week mean, and seasonal naive baselines across all targets in both windows — confirming the model learns genuine temporal structure.\n\n### Feature Ablation\n\nRemoving `lag_52` produced the largest degradation in both windows. Rolling statistics contributed meaningfully for all targets. Holiday flags had smaller and less consistent effects.\n\n---\n\n### Data Source\n\n[Google Analytics Customer Revenue Prediction](https://www.kaggle.com/c/ga-customer-revenue-prediction)\nTraining data: `train_v2.csv` — ~1.7M session-level rows, Aug 2016 – May 2018.\n","metadata":{}},{"id":"bc7d7857","cell_type":"markdown","source":"---\n## 1 · Data Loading & Aggregation\n\nThe raw CSV stores session metrics as **nested JSON strings**. We parse only the `totals` column containing visits, transactions, and transactionRevenue.\n\n- **Memory efficiency** — `usecols=[\"date\",\"totals\"]` loads 2 of ~60 columns, cutting memory ~95%\n- **Revenue conversion** — stored in micros (÷ 1,000,000 → USD)\n- **Partial week removal** — final week dropped if visits < 50% of recent median\n- **Weekly aggregation** — session rows summed to ISO week grain\n- **CVR** — computed as `orders / visits` after aggregation\n","metadata":{}},{"id":"37cacfd5","cell_type":"code","source":"import os, json, warnings\nimport numpy as np\nimport pandas as pd\nimport matplotlib.pyplot as plt\nimport matplotlib.ticker as mticker\nfrom sklearn.metrics import mean_absolute_error, mean_squared_error\nfrom xgboost import XGBRegressor\nwarnings.filterwarnings(\"ignore\")\n\nINPUT = \"/kaggle/input/competitions/ga-customer-revenue-prediction\"\n\ndef load_weekly(path):\n    df = pd.read_csv(path, dtype=str, low_memory=False, usecols=[\"date\", \"totals\"])\n    def parse_totals(x):\n        try:\n            t = json.loads(x)\n            return (float(t.get(\"visits\") or 0),\n                    float(t.get(\"transactions\") or 0),\n                    float(t.get(\"transactionRevenue\") or 0))\n        except:\n            return (0.0, 0.0, 0.0)\n    parsed        = df[\"totals\"].map(parse_totals)\n    df[\"visits\"]  = [x[0] for x in parsed]\n    df[\"orders\"]  = [x[1] for x in parsed]\n    df[\"revenue\"] = [x[2] for x in parsed]\n    df.drop(columns=[\"totals\"], inplace=True)\n    df[\"date\"] = pd.to_datetime(df[\"date\"], format=\"%Y%m%d\")\n    df[\"week\"] = df[\"date\"].dt.to_period(\"W\").dt.start_time\n    df.drop(columns=[\"date\"], inplace=True)\n    weekly = df.groupby(\"week\").agg(\n        visits  = (\"visits\",  \"sum\"),\n        orders  = (\"orders\",  \"sum\"),\n        revenue = (\"revenue\", \"sum\"),\n    ).reset_index()\n    weekly[\"revenue\"] = weekly[\"revenue\"] / 1e6\n    weekly[\"cvr\"]     = weekly[\"orders\"] / weekly[\"visits\"].replace(0, np.nan)\n    return weekly.sort_values(\"week\").reset_index(drop=True)\n\nweekly_raw = load_weekly(f\"{INPUT}/train_v2.csv\")\n\n# drop last week if partial (visits < 50% of recent median)\nrecent_median = weekly_raw[\"visits\"].iloc[-8:-1].median()\nif weekly_raw[\"visits\"].iloc[-1] < 0.5 * recent_median:\n    print(f\"Dropping partial week: {weekly_raw['week'].iloc[-1].date()} \"\n          f\"({weekly_raw['visits'].iloc[-1]:.0f} visits vs median {recent_median:.0f})\")\n    weekly = weekly_raw.iloc[:-1].reset_index(drop=True)\nelse:\n    weekly = weekly_raw.copy()\n\nprint(f\"Weeks:      {len(weekly)}\")\nprint(f\"Date range: {weekly.week.min().date()}  ->  {weekly.week.max().date()}\")\nweekly.tail(4)\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-04-29T18:44:05.974014Z","iopub.execute_input":"2026-04-29T18:44:05.975065Z","iopub.status.idle":"2026-04-29T18:48:15.291255Z","shell.execute_reply.started":"2026-04-29T18:44:05.975023Z","shell.execute_reply":"2026-04-29T18:48:15.289820Z"}},"outputs":[],"execution_count":null},{"id":"79661e40","cell_type":"markdown","source":"---\n## 2 · Exploratory Data Analysis\n\n91 weeks spanning Aug 2016 – Apr 2018. Seasonal spikes in Nov–Dec 2016 and Nov–Dec 2017 (Black Friday / Christmas)\n","metadata":{}},{"id":"d2301844","cell_type":"code","source":"# ── Cell 3 · EDA — Weekly by Week-of-Year, 3 years ────────────────────────\nTARGETS = [\"visits\", \"orders\", \"revenue\", \"cvr\"]\nLABELS  = {\"visits\": \"Visits\", \"orders\": \"Orders\",\n           \"revenue\": \"Revenue (USD)\", \"cvr\": \"CVR\"}\n\nC = {\n    \"visits\": \"#4C9BE8\", \"orders\": \"#F4A347\",\n    \"revenue\": \"#5EC97A\", \"cvr\": \"#C97BE8\",\n    \"actual\": \"#CCCCCC\", \"bg\": \"#0F0F0F\",\n    \"panel\": \"#161616\", \"text\": \"#F0F0F0\",\n    \"grid\": \"#2A2A2A\", \"subtext\": \"#888888\",\n}\n\nplt.rcParams.update({\n    \"figure.facecolor\": C[\"bg\"],   \"axes.facecolor\":  C[\"panel\"],\n    \"axes.edgecolor\":   C[\"grid\"], \"axes.labelcolor\": C[\"text\"],\n    \"xtick.color\":      C[\"subtext\"], \"ytick.color\":  C[\"subtext\"],\n    \"text.color\":       C[\"text\"], \"grid.color\":      C[\"grid\"],\n    \"grid.linewidth\":   0.5,       \"font.family\":     \"monospace\",\n})\n\ndef fmt_yaxis(ax, target):\n    if target == \"revenue\":\n        ax.yaxis.set_major_formatter(mticker.FuncFormatter(\n            lambda x, _: f\"${x/1e3:.0f}k\" if x >= 1000 else f\"${x:.0f}\"))\n    elif target == \"cvr\":\n        ax.yaxis.set_major_formatter(mticker.FuncFormatter(\n            lambda x, _: f\"{x*100:.2f}%\"))\n    else:\n        ax.yaxis.set_major_formatter(mticker.FuncFormatter(\n            lambda x, _: f\"{x/1e3:.0f}k\" if x >= 1000 else f\"{x:.0f}\"))\n\nYEAR_COLORS = {\n    \"visits\":  {\"2016\": \"#9FE1CB\", \"2017\": \"#1D9E75\", \"2018\": \"#085041\"},\n    \"orders\":  {\"2016\": \"#FAC775\", \"2017\": \"#BA7517\", \"2018\": \"#633806\"},\n    \"revenue\": {\"2016\": \"#85B7EB\", \"2017\": \"#378ADD\", \"2018\": \"#0C447C\"},\n    \"cvr\":     {\"2016\": \"#AFA9EC\", \"2017\": \"#7F77DD\", \"2018\": \"#3C3489\"},\n}\n\nyear_styles = {\n    \"2016\": {\"lw\": 1.5, \"ls\": \"-\",  \"label\": \"2016 (Aug–Dec)\"},\n    \"2017\": {\"lw\": 1.5, \"ls\": \"-\",  \"label\": \"2017 (full year)\"},\n    \"2018\": {\"lw\": 1.5, \"ls\": \"--\", \"label\": \"2018 (Jan–Apr)\"},\n}\n\nMONTH_WEEK = [1, 5, 9, 13, 18, 22, 26, 31, 35, 40, 44, 48]\nMONTHS     = [\"Jan\",\"Feb\",\"Mar\",\"Apr\",\"May\",\"Jun\",\n              \"Jul\",\"Aug\",\"Sep\",\"Oct\",\"Nov\",\"Dec\"]\n\nfig, axes = plt.subplots(2, 2, figsize=(16, 8))\nfig.suptitle(\"Weekly KPIs by Week-of-Year — 2016 vs 2017 vs 2018\",\n             fontsize=14, color=C[\"text\"], fontfamily=\"monospace\", y=1.01)\n\nfor ax, target in zip(axes.flat, TARGETS):\n    for year, style in year_styles.items():\n        sub = weekly[weekly[\"week\"].dt.year == int(year)].copy()\n        sub[\"woy\"] = sub[\"week\"].dt.isocalendar().week.astype(int)\n        ax.plot(\n            sub[\"woy\"], sub[target],\n            color=YEAR_COLORS[target][year],\n            lw=style[\"lw\"], ls=style[\"ls\"],\n            marker=\"o\", markersize=3,\n            label=style[\"label\"],\n        )\n    ax.set_title(LABELS[target], color=C[\"text\"], fontsize=11, loc=\"left\")\n    ax.set_xticks(MONTH_WEEK)\n    ax.set_xticklabels(MONTHS, fontsize=9)\n    ax.set_xlim(1, 53)\n    ax.grid(True, axis=\"y\")\n    ax.legend(fontsize=8, framealpha=0.2)\n    fmt_yaxis(ax, target)\n\nfig.tight_layout()\nplt.show()","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-04-29T18:48:21.853213Z","iopub.execute_input":"2026-04-29T18:48:21.854373Z","iopub.status.idle":"2026-04-29T18:48:22.783730Z","shell.execute_reply.started":"2026-04-29T18:48:21.854319Z","shell.execute_reply":"2026-04-29T18:48:22.782369Z"}},"outputs":[],"execution_count":null},{"id":"5ab02036-b915-4a10-bc68-eca33899f971","cell_type":"markdown","source":"-Visits and Orders are more volatile from Oct onward. Revenue and CVR are volatile throughout the year.","metadata":{}},{"id":"3d0a6281","cell_type":"markdown","source":"---\n## 3 · Config & Feature Engineering\n\n### Direct Multi-Step Forecasting\n\n4 targets × 13 horizons = **52 independent models**. Each predicts directly from real historical data — no model sees another model's output.\n\nMinimum lag = **8 weeks** — safe for all 13 horizon steps with no look-ahead leakage.\n\n| Feature group | Features |\n|---|---|\n| Lag values | lag_8, lag_12, lag_16, lag_20, lag_52 |\n| Rolling stats | 4w & 8w trailing mean + std (anchored at lag_8) |\n| Calendar | week_of_year, month, quarter, year, linear trend |\n| Holiday flags | BFCM (wk 47–48), Christmas (wk 51–52), New Year (wk 1–2) |\n","metadata":{}},{"id":"59811e0f","cell_type":"code","source":"# ── Cell 4 · Config & Feature Engineering ─────────────────────────────────\nBACKTEST_WEEKS = 13\nFORECAST_WEEKS = 13\nLAGS  = [8, 12, 16, 20, 52]   # lag_52 added for same-week-last-year signal\nROLLS = [4, 8]\nFMT_STR = {\"visits\": \"{:.0f}\", \"orders\": \"{:.0f}\",\n           \"revenue\": \"${:,.0f}\", \"cvr\": \"{:.3%}\"}\n\nFEATURE_COLS = (\n    [f\"lag_{l}\" for l in LAGS]\n    + [f\"roll_mean_{w}\" for w in ROLLS]\n    + [f\"roll_std_{w}\"  for w in ROLLS]\n    + [\"week_of_year\", \"month\", \"quarter\", \"year\", \"t\",\n       \"is_bfcm\", \"is_xmas\", \"is_newyear\"]\n)\n\nXGB_PARAMS = dict(\n    objective        = \"reg:squarederror\",\n    n_estimators     = 300,\n    learning_rate    = 0.05,\n    max_depth        = 4,\n    subsample        = 0.8,\n    colsample_bytree = 0.8,\n    verbosity        = 0,\n)\n\ndef add_features(df, target):\n    df = df.copy()\n    s  = df[target]\n    for lag in LAGS:\n        df[f\"lag_{lag}\"] = s.shift(lag)\n    for w in ROLLS:\n        df[f\"roll_mean_{w}\"] = s.shift(LAGS[0]).rolling(w).mean()\n        df[f\"roll_std_{w}\"]  = s.shift(LAGS[0]).rolling(w).std()\n    df[\"week_of_year\"] = df[\"week\"].dt.isocalendar().week.astype(int)\n    df[\"month\"]        = df[\"week\"].dt.month\n    df[\"quarter\"]      = df[\"week\"].dt.quarter\n    df[\"year\"]         = df[\"week\"].dt.year\n    df[\"t\"]            = np.arange(len(df))\n    df[\"is_bfcm\"]      = df[\"week_of_year\"].isin([47, 48]).astype(int)\n    df[\"is_xmas\"]      = df[\"week_of_year\"].isin([51, 52]).astype(int)\n    df[\"is_newyear\"]   = df[\"week_of_year\"].isin([1,  2 ]).astype(int)\n    return df\n\nn          = len(weekly)\ntest_start = n - BACKTEST_WEEKS\nprint(f\"Total weeks : {n}\")\nprint(f\"Train       : week 0 – {test_start-1} \"\n      f\"({weekly['week'].iloc[0].date()} – {weekly['week'].iloc[test_start-1].date()})\")\nprint(f\"Test        : week {test_start} – {n-1} \"\n      f\"({weekly['week'].iloc[test_start].date()} – {weekly['week'].iloc[-1].date()})\")\nprint(f\"\\n✓ features ready\")","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-04-29T18:48:33.699493Z","iopub.execute_input":"2026-04-29T18:48:33.699935Z","iopub.status.idle":"2026-04-29T18:48:33.714819Z","shell.execute_reply.started":"2026-04-29T18:48:33.699902Z","shell.execute_reply":"2026-04-29T18:48:33.713385Z"}},"outputs":[],"execution_count":null},{"id":"33ff0ed3","cell_type":"code","source":"# ── summary table: average actuals per week for both test windows ──────────\n\nn          = len(weekly)\ntest_start = n - BACKTEST_WEEKS   # Jan 2018 window start index\n\n# Jan–Apr 2018 window\njan_2018 = weekly.iloc[test_start: test_start + BACKTEST_WEEKS]\n\n# Sep–Dec 2017 window\nsept_start_date = pd.Timestamp(\"2017-09-25\")\nsept_start_idx  = weekly[weekly[\"week\"] == sept_start_date].index[0]\nsept_2017 = weekly.iloc[sept_start_idx: sept_start_idx + 13]\n\n# build summary\nrows = []\nfor target in TARGETS:\n    rows.append({\n        \"Metric\":                LABELS[target],\n        \"Sep–Dec 2017 avg/wk\":  sept_2017[target].mean(),\n        \"Sep–Dec 2017 min\":     sept_2017[target].min(),\n        \"Sep–Dec 2017 max\":     sept_2017[target].max(),\n        \"Jan–Apr 2018 avg/wk\":  jan_2018[target].mean(),\n        \"Jan–Apr 2018 min\":     jan_2018[target].min(),\n        \"Jan–Apr 2018 max\":     jan_2018[target].max(),\n    })\n\nsummary_df = pd.DataFrame(rows).set_index(\"Metric\")\n\n# format each row based on metric type\ndef fmt_val(val, target):\n    if target == \"revenue\":\n        return f\"${val:,.0f}\"\n    elif target == \"cvr\":\n        return f\"{val*100:.3f}%\"\n    else:\n        return f\"{val:,.0f}\"\n\nformatted_rows = []\nfor target, label in zip(TARGETS, summary_df.index):\n    r = summary_df.loc[label]\n    formatted_rows.append({\n        \"Metric\":                label,\n        \"Sep–Dec 2017 avg/wk\":  fmt_val(r[\"Sep–Dec 2017 avg/wk\"], target),\n        \"Sep–Dec 2017 min\":     fmt_val(r[\"Sep–Dec 2017 min\"],     target),\n        \"Sep–Dec 2017 max\":     fmt_val(r[\"Sep–Dec 2017 max\"],     target),\n        \"Jan–Apr 2018 avg/wk\":  fmt_val(r[\"Jan–Apr 2018 avg/wk\"], target),\n        \"Jan–Apr 2018 min\":     fmt_val(r[\"Jan–Apr 2018 min\"],     target),\n        \"Jan–Apr 2018 max\":     fmt_val(r[\"Jan–Apr 2018 max\"],     target),\n    })\n\nfmt_df = pd.DataFrame(formatted_rows).set_index(\"Metric\")\n\nprint(\"Average Actuals per Week — Both Test Windows\")\nprint(\"=\" * 75)\nprint(fmt_df.to_string())\nprint()\n\nfmt_df.style \\\n    .set_caption(\"Average Weekly Actuals — Test Window Comparison\") \\\n    .set_properties(**{\"text-align\": \"center\"}) \\\n    .set_table_styles([\n        {\"selector\": \"caption\",\n         \"props\": [(\"font-size\",\"13px\"),(\"font-weight\",\"bold\"),(\"padding\",\"10px\")]},\n        {\"selector\": \"th\",\n         \"props\": [(\"background-color\",\"#222\"),(\"color\",\"#eee\"),(\"padding\",\"8px 14px\")]},\n        {\"selector\": \"td\",\n         \"props\": [(\"padding\",\"6px 14px\")]},\n        {\"selector\": \"th.col_heading\",\n         \"props\": [(\"text-align\",\"center\")]},\n    ])","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-04-29T18:48:38.491658Z","iopub.execute_input":"2026-04-29T18:48:38.492403Z","iopub.status.idle":"2026-04-29T18:48:38.659061Z","shell.execute_reply.started":"2026-04-29T18:48:38.492363Z","shell.execute_reply":"2026-04-29T18:48:38.657763Z"}},"outputs":[],"execution_count":null},{"id":"d3dd6f11","cell_type":"markdown","source":"---\n## 4 · Walk-Forward Backtest — XGBoost Jan-Apr 2018\n\n8 folds slide across the training portion. The **last 13 weeks are held out entirely**.\n\nTwo metrics reported separately:\n- **WF MAPE** — averaged across all 8 folds\n- **Holdout MAPE** — final held-out 13 weeks only (Jan–Apr 2018)\n","metadata":{}},{"id":"671d0007","cell_type":"code","source":"# ── Cell 5a · Walk-Forward Backtest — XGBoost ─────────────────────────────\ndef run_backtest(weekly, target):\n    n          = len(weekly)\n    test_start = n - BACKTEST_WEEKS\n    min_train  = 60          # bumped from 40 to clear lag_52 warmup\n    n_folds    = 8\n\n    df_feat    = add_features(weekly, target).reset_index(drop=True)\n    X_all      = df_feat[FEATURE_COLS]\n    feat_valid = X_all.notna().all(axis=1)\n\n    step    = max(1, (test_start - min_train) // n_folds)\n    origins = list(range(min_train, test_start, step))[:n_folds]\n    if test_start not in origins:\n        origins.append(test_start)\n\n    fold_results = []\n    test_result  = None\n\n    for origin in origins:\n        preds = []\n        for h in range(1, BACKTEST_WEEKS + 1):\n            y    = pd.Series(weekly[target].shift(-h).values, index=df_feat.index)\n            mask = (df_feat.index < origin) & feat_valid & y.notna()\n            if mask.sum() < 10:\n                preds.append(np.nan)\n                continue\n            m = XGBRegressor(**XGB_PARAMS)\n            m.fit(X_all[mask], y[mask])\n            preds.append(m.predict(X_all.iloc[[origin - 1]])[0])\n\n        actual = weekly[target].iloc[origin: origin + BACKTEST_WEEKS].values\n        weeks  = weekly[\"week\"].iloc[origin: origin + BACKTEST_WEEKS].values\n        result = {\"origin\": origin, \"pred\": np.array(preds[:len(actual)]),\n                  \"actual\": actual, \"weeks\": weeks}\n        fold_results.append(result)\n        if origin == test_start:\n            test_result = result\n\n    return fold_results, test_result\n\n\ndef compute_wf_metrics(fold_results):\n    \"\"\"Average MAE/RMSE/MAPE across all walk-forward folds.\"\"\"\n    all_pred   = np.concatenate([r[\"pred\"]   for r in fold_results])\n    all_actual = np.concatenate([r[\"actual\"] for r in fold_results])\n    valid      = ~np.isnan(all_pred) & ~np.isnan(all_actual)\n    mae  = mean_absolute_error(all_actual[valid], all_pred[valid])\n    rmse = np.sqrt(mean_squared_error(all_actual[valid], all_pred[valid]))\n    mape = np.mean(np.abs(all_actual[valid] - all_pred[valid]) /\n                   (np.abs(all_actual[valid]) + 1e-8)) * 100\n    h_mae = []\n    for h in range(BACKTEST_WEEKS):\n        a = np.array([r[\"actual\"][h] for r in fold_results if len(r[\"actual\"]) > h])\n        p = np.array([r[\"pred\"][h]   for r in fold_results if len(r[\"pred\"])   > h])\n        v = ~np.isnan(p) & ~np.isnan(a)\n        h_mae.append(mean_absolute_error(a[v], p[v]) if v.sum() > 0 else np.nan)\n    return {\"mae\": mae, \"rmse\": rmse, \"mape\": mape, \"h_mae\": h_mae}\n\n\ndef compute_holdout_metrics(test_result):\n    \"\"\"MAE/RMSE/MAPE on the final held-out 13-week window only.\"\"\"\n    pred   = test_result[\"pred\"]\n    actual = test_result[\"actual\"]\n    valid  = ~np.isnan(pred) & ~np.isnan(actual)\n    mae  = mean_absolute_error(actual[valid], pred[valid])\n    rmse = np.sqrt(mean_squared_error(actual[valid], pred[valid]))\n    mape = np.mean(np.abs(actual[valid] - pred[valid]) /\n                   (np.abs(actual[valid]) + 1e-8)) * 100\n    return {\"mae\": mae, \"rmse\": rmse, \"mape\": mape}\n\n\n# ── run ────────────────────────────────────────────────────────────────────\nprint(\"Running XGBoost backtests — pooled walk-forward + final 13-week holdout ...\\n\")\n\nall_folds       = {}\nwf_metrics      = {}\ntest_results    = {}\nholdout_metrics = {}\n\nfor target in TARGETS:\n    print(f\"  {target} ...\", end=\" \")\n    folds, test_r           = run_backtest(weekly, target)\n    all_folds[target]       = folds\n    test_results[target]    = test_r\n    wf_metrics[target]      = compute_wf_metrics(folds)\n    holdout_metrics[target] = compute_holdout_metrics(test_r)\n    print(\n        f\"WF MAE={wf_metrics[target]['mae']:.4f}  \"\n        f\"WF MAPE={wf_metrics[target]['mape']:.4f}%  \"\n        f\"HOLDOUT MAE={holdout_metrics[target]['mae']:.4f}  \"\n        f\"HOLDOUT MAPE={holdout_metrics[target]['mape']:.4f}%\"\n    )\n\nprint(\"\\n✓ XGBoost backtest complete\")","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-04-29T18:48:45.508003Z","iopub.execute_input":"2026-04-29T18:48:45.509014Z","iopub.status.idle":"2026-04-29T18:49:30.527365Z","shell.execute_reply.started":"2026-04-29T18:48:45.508962Z","shell.execute_reply":"2026-04-29T18:49:30.526275Z"}},"outputs":[],"execution_count":null},{"id":"43dc7599","cell_type":"markdown","source":"---\n## 5 · Walk-Forward Backtest — LightGBM Jan-Apr 2018\n\nIdentical structure to XGBoost — same origins, features, held-out test window.\n\nKey parameter differences for small dataset:\n- `num_leaves=15` (not `max_depth`) — correct LightGBM parameter\n- `min_child_samples=3` — reduced from default 20 to handle ~60 training rows\n- `mask.sum() < 5` threshold — reduced from 10 to allow smaller training windows\n","metadata":{}},{"id":"f1697389","cell_type":"code","source":"# ── LightGBM backtest — identical structure to XGBoost ────────────────────\nimport lightgbm as lgb\n\nLGB_PARAMS_FINAL = dict(\n    objective         = \"regression\",\n    n_estimators      = 300,\n    learning_rate     = 0.05,\n    num_leaves        = 15,\n    min_child_samples = 3,\n    subsample         = 0.8,\n    colsample_bytree  = 0.8,\n    verbose           = -1,\n)\n\ndef run_backtest_lgb(weekly, target):\n    n          = len(weekly)\n    test_start = n - BACKTEST_WEEKS\n    min_train  = 60\n    n_folds    = 8\n\n    df_feat    = add_features(weekly, target).reset_index(drop=True)\n    X_all      = df_feat[FEATURE_COLS]\n    feat_valid = X_all.notna().all(axis=1)\n\n    step    = max(1, (test_start - min_train) // n_folds)\n    origins = list(range(min_train, test_start, step))[:n_folds]\n    if test_start not in origins:\n        origins.append(test_start)\n\n    fold_results = []\n    test_result  = None\n\n    for origin in origins:\n        preds = []\n        for h in range(1, BACKTEST_WEEKS + 1):\n            y    = pd.Series(weekly[target].shift(-h).values, index=df_feat.index)\n            mask = (df_feat.index < origin) & feat_valid & y.notna()\n            if mask.sum() < 5:\n                preds.append(np.nan)\n                continue\n            m = lgb.LGBMRegressor(**LGB_PARAMS_FINAL)\n            m.fit(X_all[mask], y[mask])\n            preds.append(m.predict(X_all.iloc[[origin - 1]])[0])\n\n        actual = weekly[target].iloc[origin: origin + BACKTEST_WEEKS].values\n        weeks  = weekly[\"week\"].iloc[origin: origin + BACKTEST_WEEKS].values\n        result = {\"origin\": origin,\n                  \"pred\":   np.array(preds[:len(actual)]),\n                  \"actual\": actual,\n                  \"weeks\":  weeks}\n        fold_results.append(result)\n        if origin == test_start:\n            test_result = result\n\n    return fold_results, test_result\n\nprint(\"✓ run_backtest_lgb defined\")\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-04-29T18:49:30.530087Z","iopub.execute_input":"2026-04-29T18:49:30.530465Z","iopub.status.idle":"2026-04-29T18:49:35.321327Z","shell.execute_reply.started":"2026-04-29T18:49:30.530428Z","shell.execute_reply":"2026-04-29T18:49:35.319580Z"}},"outputs":[],"execution_count":null},{"id":"ba158b22","cell_type":"code","source":"print(\"Running LightGBM backtests — pooled walk-forward + final 13-week holdout ...\\n\")\nall_folds_lgb       = {}\nwf_metrics_lgb      = {}\ntest_results_lgb    = {}\nholdout_metrics_lgb = {}\n\nfor target in TARGETS:\n    print(f\"  {target} ...\", end=\" \")\n    folds, test_r                  = run_backtest_lgb(weekly, target)\n    all_folds_lgb[target]          = folds\n    test_results_lgb[target]       = test_r\n    wf_metrics_lgb[target]         = compute_wf_metrics(folds)\n    holdout_metrics_lgb[target]    = compute_holdout_metrics(test_r)\n    print(\n        f\"WF MAE={wf_metrics_lgb[target]['mae']:.4f}  \"\n        f\"WF MAPE={wf_metrics_lgb[target]['mape']:.4f}%  \"\n        f\"HOLDOUT MAE={holdout_metrics_lgb[target]['mae']:.4f}  \"\n        f\"HOLDOUT MAPE={holdout_metrics_lgb[target]['mape']:.4f}%\"\n    )\n\nprint(\"\\n✓ LightGBM backtest complete\")\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-04-29T18:53:05.413243Z","iopub.execute_input":"2026-04-29T18:53:05.413886Z","iopub.status.idle":"2026-04-29T18:53:23.382947Z","shell.execute_reply.started":"2026-04-29T18:53:05.413840Z","shell.execute_reply":"2026-04-29T18:53:23.381891Z"}},"outputs":[],"execution_count":null},{"id":"03ad4372","cell_type":"markdown","source":"---\n## 6 · Model Comparison — Jan–Apr 2018 Holdout\n\nHeld-out test window: **Jan 29 – Apr 23 2018** (13 weeks, stable period).\nResults are holdout only — the 13 weeks the model never saw during training.\n","metadata":{}},{"id":"19550f71","cell_type":"code","source":"# ── Cell · Backtest Comparison Table — XGBoost vs LightGBM (Jan 29 – Apr 23) ──\nimport pandas as pd\nrows = []\nfor target in TARGETS:\n    xgb_r  = test_results[target]\n    lgbm_r = test_results_lgb[target]          # ← fixed variable name\n    actual = xgb_r[\"actual\"]\n    xgb_mae  = mean_absolute_error(actual, xgb_r[\"pred\"])\n    xgb_rmse = np.sqrt(mean_squared_error(actual, xgb_r[\"pred\"]))\n    xgb_mape = np.mean(np.abs(actual - xgb_r[\"pred\"]) / (np.abs(actual) + 1e-8)) * 100\n    lgbm_mae  = mean_absolute_error(actual, lgbm_r[\"pred\"])\n    lgbm_rmse = np.sqrt(mean_squared_error(actual, lgbm_r[\"pred\"]))\n    lgbm_mape = np.mean(np.abs(actual - lgbm_r[\"pred\"]) / (np.abs(actual) + 1e-8)) * 100\n    winner_mae  = \"XGB\" if xgb_mae  < lgbm_mae  else \"LGBM\"\n    winner_rmse = \"XGB\" if xgb_rmse < lgbm_rmse else \"LGBM\"\n    winner_mape = \"XGB\" if xgb_mape < lgbm_mape else \"LGBM\"\n    rows.append({\n        \"Target\":       LABELS[target],\n        \"XGB MAE\":      f\"{xgb_mae:.4f}\",\n        \"LGBM MAE\":     f\"{lgbm_mae:.4f}\",\n        \"Best MAE\":     winner_mae,\n        \"XGB RMSE\":     f\"{xgb_rmse:.4f}\",\n        \"LGBM RMSE\":    f\"{lgbm_rmse:.4f}\",\n        \"Best RMSE\":    winner_rmse,\n        \"XGB MAPE\":     f\"{xgb_mape:.4f}%\",\n        \"LGBM MAPE\":    f\"{lgbm_mape:.4f}%\",\n        \"Best MAPE\":    winner_mape,\n    })\ncmp_df = pd.DataFrame(rows).set_index(\"Target\")\ndef highlight_best(val):\n    if val == \"XGB\":\n        return \"background-color: #1D9E75; color: white\"\n    elif val == \"LGBM\":\n        return \"background-color: #378ADD; color: white\"\n    return \"\"\ncmp_df.style \\\n    .set_caption(\"Backtest Comparison — XGBoost vs LightGBM (2018-01-29 to 2018-04-23)\") \\\n    .applymap(highlight_best, subset=[\"Best MAE\", \"Best RMSE\", \"Best MAPE\"]) \\\n    .set_properties(**{\"text-align\": \"center\"}) \\\n    .set_table_styles([\n        {\"selector\": \"caption\",\n         \"props\": [(\"font-size\",\"14px\"),(\"font-weight\",\"bold\"),(\"padding\",\"10px\")]},\n        {\"selector\": \"th\",\n         \"props\": [(\"background-color\",\"#222\"),(\"color\",\"#eee\"),(\"padding\",\"8px 14px\")]},\n        {\"selector\": \"td\",\n         \"props\": [(\"padding\",\"6px 14px\")]},\n    ])","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-04-29T18:53:39.472293Z","iopub.execute_input":"2026-04-29T18:53:39.472982Z","iopub.status.idle":"2026-04-29T18:53:39.506901Z","shell.execute_reply.started":"2026-04-29T18:53:39.472945Z","shell.execute_reply":"2026-04-29T18:53:39.505675Z"}},"outputs":[],"execution_count":null},{"id":"d2d0876c-e4df-487c-955b-9752fd4cac2d","cell_type":"markdown","source":"XGBoost outperformed on visits orders, revenue and LightGBM outperformed on CVR.","metadata":{}},{"id":"d1c02daf","cell_type":"markdown","source":"---\n## 7 · Backtest Chart — Jan–Apr 2018\n\nGrey line = training history · Coloured = XGBoost · Dashed = LightGBM · Red crosses = actuals.\n","metadata":{}},{"id":"3bd8b64d","cell_type":"code","source":"# ── Cell · Backtest Chart — XGBoost vs LightGBM (test window only) ────────\nimport matplotlib.pyplot as plt\nimport matplotlib.ticker as mticker\nimport matplotlib.dates as mdates\nimport numpy as np\n\nfig, axes = plt.subplots(2, 2, figsize=(16, 10))\nfig.suptitle(\"Backtest — XGBoost vs LightGBM (2018-01-29 to 2018-04-23)\",\n             fontsize=13, color=C[\"text\"], fontfamily=\"monospace\", y=1.01)\n\nfor ax, target in zip(axes.flat, TARGETS):\n    xgb_r  = test_results[target]\n    lgbm_r = test_results_lgb[target]\n\n    test_weeks  = weekly[\"week\"].iloc[n - BACKTEST_WEEKS: n]\n    test_actual = weekly[target].iloc[n - BACKTEST_WEEKS: n]\n\n    # xgboost — line + dots\n    ax.plot(test_weeks, xgb_r[\"pred\"],\n            color=\"#00E5A0\", lw=2, marker=\"o\", markersize=6,\n            markeredgecolor=\"white\", markeredgewidth=1,\n            label=\"XGBoost forecast\", zorder=3)\n\n    # lightgbm — dashed line, no markers\n    ax.plot(test_weeks, lgbm_r[\"pred\"],\n            color=\"#60CFFF\", lw=2, marker=None,\n            ls=\"--\", label=\"LightGBM forecast\", zorder=2)\n\n    # actuals — x markers only, no line\n    ax.scatter(test_weeks, test_actual,\n               color=\"#FF6B6B\", marker=\"X\", s=100,\n               linewidths=1.5, edgecolors=\"white\",\n               label=\"Actuals\", zorder=5)\n\n    ax.set_title(LABELS[target], color=C[\"text\"], fontsize=11, loc=\"left\")\n    ax.xaxis.set_major_locator(mdates.WeekdayLocator(byweekday=mdates.MO))\n    ax.xaxis.set_major_formatter(mdates.DateFormatter(\"%b %d\"))\n    plt.setp(ax.get_xticklabels(), rotation=45, ha=\"right\", fontsize=8)\n    ax.set_xlim(test_weeks.iloc[0], test_weeks.iloc[-1])\n    ax.grid(True, axis=\"y\")\n    ax.legend(fontsize=8, framealpha=0.2)\n    fmt_yaxis(ax, target)\n\nfig.tight_layout()\nplt.show()","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-04-29T18:53:43.113646Z","iopub.execute_input":"2026-04-29T18:53:43.115187Z","iopub.status.idle":"2026-04-29T18:53:44.658157Z","shell.execute_reply.started":"2026-04-29T18:53:43.115142Z","shell.execute_reply":"2026-04-29T18:53:44.656923Z"}},"outputs":[],"execution_count":null},{"id":"cab51efe-9bec-4320-b8ec-c8c9b1fd506b","cell_type":"markdown","source":"Visits, orders, revenue coming in close with some variance in CVR","metadata":{}},{"id":"b157a07e","cell_type":"markdown","source":"---\n## 8 · Backtest — XGBoost, Sep–Dec 2017\n\n","metadata":{}},{"id":"bddef5df","cell_type":"code","source":"# ── define the Sept–Dec 2017 test window by date ──────────────────────────\nSEPT_WEEKS = 13\n\nsept_start_date = pd.Timestamp(\"2017-09-25\")\nsept_start_idx  = weekly[weekly[\"week\"] == sept_start_date].index[0]\n\nprint(f\"Start index: {sept_start_idx}\")\nprint(f\"Start date:  {weekly['week'].iloc[sept_start_idx].date()}\")\nprint(f\"End date:    {weekly['week'].iloc[sept_start_idx + SEPT_WEEKS - 1].date()}\")","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-04-29T18:54:00.861465Z","iopub.execute_input":"2026-04-29T18:54:00.861997Z","iopub.status.idle":"2026-04-29T18:54:00.872422Z","shell.execute_reply.started":"2026-04-29T18:54:00.861962Z","shell.execute_reply":"2026-04-29T18:54:00.870803Z"}},"outputs":[],"execution_count":null},{"id":"10cbdd6a","cell_type":"code","source":"# ── Backtest — XGBoost Sept–Dec 2017 — WF + Holdout ──────────────────────\n# lag_52 dropped — not enough history before Sept 2017\nLAGS_SEPT = [8, 12, 16, 20]\n\nFEATURE_COLS_SEPT = (\n    [f\"lag_{l}\"       for l in LAGS_SEPT]\n  + [f\"roll_mean_{w}\" for w in ROLLS]\n  + [f\"roll_std_{w}\"  for w in ROLLS]\n  + [\"week_of_year\", \"month\", \"quarter\", \"year\", \"t\",\n     \"is_bfcm\", \"is_xmas\", \"is_newyear\"]\n)\n\ndef add_features_sept(df, target):\n    df = df.copy()\n    s  = df[target]\n    for lag in LAGS_SEPT:\n        df[f\"lag_{lag}\"] = s.shift(lag)\n    for w in ROLLS:\n        df[f\"roll_mean_{w}\"] = s.shift(LAGS_SEPT[0]).rolling(w).mean()\n        df[f\"roll_std_{w}\"]  = s.shift(LAGS_SEPT[0]).rolling(w).std()\n    df[\"week_of_year\"] = df[\"week\"].dt.isocalendar().week.astype(int)\n    df[\"month\"]        = df[\"week\"].dt.month\n    df[\"quarter\"]      = df[\"week\"].dt.quarter\n    df[\"year\"]         = df[\"week\"].dt.year\n    df[\"t\"]            = np.arange(len(df))\n    df[\"is_bfcm\"]      = df[\"week_of_year\"].isin([47, 48]).astype(int)\n    df[\"is_xmas\"]      = df[\"week_of_year\"].isin([51, 52]).astype(int)\n    df[\"is_newyear\"]   = df[\"week_of_year\"].isin([1,  2 ]).astype(int)\n    return df\n\n\ndef run_backtest_sept(weekly, target):\n    \"\"\"\n    Walk-forward backtest anchored at Sept 2017.\n    Test window = last 13 weeks of the Sept–Dec 2017 period.\n    Folds slide within the data available before the test window.\n    Uses LAGS_SEPT (no lag_52) — not enough history before Sept 2017.\n    \"\"\"\n    sept_start_date = pd.Timestamp(\"2017-09-25\")\n    sept_start_idx  = weekly[weekly[\"week\"] == sept_start_date].index[0]\n\n    # held-out test = the 13 weeks starting at sept_start_idx\n    test_start = sept_start_idx\n    min_train  = 40       # need at least 40 weeks to train\n    n_folds    = 6        # fewer folds — less room before Sept 2017\n\n    df_feat    = add_features_sept(weekly, target).reset_index(drop=True)\n    X_all      = df_feat[FEATURE_COLS_SEPT]\n    feat_valid = X_all.notna().all(axis=1)\n\n    step    = max(1, (test_start - min_train) // n_folds)\n    origins = list(range(min_train, test_start, step))[:n_folds]\n    if test_start not in origins:\n        origins.append(test_start)\n\n    fold_results = []\n    test_result  = None\n\n    for origin in origins:\n        preds = []\n        for h in range(1, SEPT_WEEKS + 1):\n            y    = pd.Series(weekly[target].shift(-h).values, index=df_feat.index)\n            mask = (df_feat.index < origin) & feat_valid & y.notna()\n            if mask.sum() < 5:\n                preds.append(np.nan)\n                continue\n            m = XGBRegressor(**XGB_PARAMS)\n            m.fit(X_all[mask], y[mask])\n            preds.append(m.predict(X_all.iloc[[origin - 1]])[0])\n\n        actual = weekly[target].iloc[origin: origin + SEPT_WEEKS].values\n        weeks  = weekly[\"week\"].iloc[origin: origin + SEPT_WEEKS].values\n        result = {\"origin\": origin,\n                  \"pred\":   np.array(preds[:len(actual)]),\n                  \"actual\": actual,\n                  \"weeks\":  weeks}\n        fold_results.append(result)\n        if origin == test_start:\n            test_result = result\n\n    return fold_results, test_result\n\n\n# ── run ────────────────────────────────────────────────────────────────────\nSEPT_WEEKS = 13\n\nprint(\"Running XGBoost Sept–Dec 2017 backtest — WF + Holdout ...\\n\")\n\nall_folds_sept       = {}\nwf_metrics_sept      = {}\ntest_results_xgb_sept = {}\nholdout_metrics_sept = {}\n\nfor target in TARGETS:\n    print(f\"  {target} ...\", end=\"  \")\n    folds, test_r                  = run_backtest_sept(weekly, target)\n    all_folds_sept[target]         = folds\n    test_results_xgb_sept[target]  = test_r\n    wf_metrics_sept[target]        = compute_wf_metrics(folds)\n    holdout_metrics_sept[target]   = compute_holdout_metrics(test_r)\n    print(\n        f\"WF MAE={wf_metrics_sept[target]['mae']:.4f}  \"\n        f\"WF MAPE={wf_metrics_sept[target]['mape']:.4f}%  \"\n        f\"HOLDOUT MAE={holdout_metrics_sept[target]['mae']:.4f}  \"\n        f\"HOLDOUT MAPE={holdout_metrics_sept[target]['mape']:.4f}%\"\n    )\n\nprint(\"\\n✓ XGBoost Sept–Dec 2017 backtest complete\")","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-04-29T18:54:03.315501Z","iopub.execute_input":"2026-04-29T18:54:03.316016Z","iopub.status.idle":"2026-04-29T18:54:47.498546Z","shell.execute_reply.started":"2026-04-29T18:54:03.315969Z","shell.execute_reply":"2026-04-29T18:54:47.497658Z"}},"outputs":[],"execution_count":null},{"id":"a4e99af3","cell_type":"markdown","source":"---\n## 9 · Backtest — LightGBM, Sep–Dec 2017\n\nWith ~60 training rows, LightGBM's leaf-wise growth overfits — errors are substantially larger than XGBoost across all targets.\n","metadata":{}},{"id":"bd3cf2a7","cell_type":"code","source":"# ── Backtest — LightGBM Sept–Dec 2017 — WF + Holdout ─────────────────────\n\ndef run_backtest_sept_lgb(weekly, target):\n    \"\"\"\n    Identical structure to run_backtest_sept but uses LightGBM.\n    Uses FEATURE_COLS_SEPT (no lag_52) — not enough history before Sept 2017.\n    \"\"\"\n    sept_start_date = pd.Timestamp(\"2017-09-25\")\n    sept_start_idx  = weekly[weekly[\"week\"] == sept_start_date].index[0]\n\n    test_start = sept_start_idx\n    min_train  = 40\n    n_folds    = 6\n\n    df_feat    = add_features_sept(weekly, target).reset_index(drop=True)\n    X_all      = df_feat[FEATURE_COLS_SEPT]\n    feat_valid = X_all.notna().all(axis=1)\n\n    step    = max(1, (test_start - min_train) // n_folds)\n    origins = list(range(min_train, test_start, step))[:n_folds]\n    if test_start not in origins:\n        origins.append(test_start)\n\n    fold_results = []\n    test_result  = None\n\n    for origin in origins:\n        preds = []\n        for h in range(1, SEPT_WEEKS + 1):\n            y    = pd.Series(weekly[target].shift(-h).values, index=df_feat.index)\n            mask = (df_feat.index < origin) & feat_valid & y.notna()\n            if mask.sum() < 5:\n                preds.append(np.nan)\n                continue\n            m = lgb.LGBMRegressor(**LGB_PARAMS_FINAL)\n            m.fit(X_all[mask], y[mask])\n            preds.append(m.predict(X_all.iloc[[origin - 1]])[0])\n\n        actual = weekly[target].iloc[origin: origin + SEPT_WEEKS].values\n        weeks  = weekly[\"week\"].iloc[origin: origin + SEPT_WEEKS].values\n        result = {\"origin\": origin,\n                  \"pred\":   np.array(preds[:len(actual)]),\n                  \"actual\": actual,\n                  \"weeks\":  weeks}\n        fold_results.append(result)\n        if origin == test_start:\n            test_result = result\n\n    return fold_results, test_result\n\n\n# ── run ────────────────────────────────────────────────────────────────────\nprint(\"Running LightGBM Sept–Dec 2017 backtest — WF + Holdout ...\\n\")\n\nall_folds_sept_lgb       = {}\nwf_metrics_sept_lgb      = {}\ntest_results_lgb_sept    = {}\nholdout_metrics_sept_lgb = {}\n\nfor target in TARGETS:\n    print(f\"  {target} ...\", end=\"  \")\n    folds, test_r                      = run_backtest_sept_lgb(weekly, target)\n    all_folds_sept_lgb[target]         = folds\n    test_results_lgb_sept[target]      = test_r\n    wf_metrics_sept_lgb[target]        = compute_wf_metrics(folds)\n    holdout_metrics_sept_lgb[target]   = compute_holdout_metrics(test_r)\n    print(\n        f\"WF MAE={wf_metrics_sept_lgb[target]['mae']:.4f}  \"\n        f\"WF MAPE={wf_metrics_sept_lgb[target]['mape']:.4f}%  \"\n        f\"HOLDOUT MAE={holdout_metrics_sept_lgb[target]['mae']:.4f}  \"\n        f\"HOLDOUT MAPE={holdout_metrics_sept_lgb[target]['mape']:.4f}%\"\n    )\n\nprint(\"\\n✓ LightGBM Sept–Dec 2017 backtest complete\")","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-04-29T18:54:50.464696Z","iopub.execute_input":"2026-04-29T18:54:50.465284Z","iopub.status.idle":"2026-04-29T18:55:11.925328Z","shell.execute_reply.started":"2026-04-29T18:54:50.465247Z","shell.execute_reply":"2026-04-29T18:55:11.924237Z"}},"outputs":[],"execution_count":null},{"id":"19d13aeb","cell_type":"markdown","source":"---\n## 10 · Model Comparison — Sep–Dec 2017","metadata":{}},{"id":"463e92ff","cell_type":"code","source":"# ── Comparison Table — XGBoost vs LightGBM (Sept–Dec 2017) ───────────────\nrows = []\nfor target in TARGETS:\n    xgb_r  = test_results_xgb_sept[target]\n    lgb_r  = test_results_lgb_sept[target]\n    actual = xgb_r[\"actual\"]\n\n    xgb_mae  = mean_absolute_error(actual, xgb_r[\"pred\"])\n    xgb_rmse = np.sqrt(mean_squared_error(actual, xgb_r[\"pred\"]))\n    xgb_mape = np.mean(np.abs(actual - xgb_r[\"pred\"]) / (np.abs(actual) + 1e-8)) * 100\n\n    lgb_mae  = mean_absolute_error(actual, lgb_r[\"pred\"])\n    lgb_rmse = np.sqrt(mean_squared_error(actual, lgb_r[\"pred\"]))\n    lgb_mape = np.mean(np.abs(actual - lgb_r[\"pred\"]) / (np.abs(actual) + 1e-8)) * 100\n\n    rows.append({\n        \"Target\":    LABELS[target],\n        \"XGB MAE\":   f\"{xgb_mae:.4f}\",\n        \"LGB MAE\":   f\"{lgb_mae:.4f}\",\n        \"Best MAE\":  \"XGB\" if xgb_mae  < lgb_mae  else \"LGB\",\n        \"XGB RMSE\":  f\"{xgb_rmse:.4f}\",\n        \"LGB RMSE\":  f\"{lgb_rmse:.4f}\",\n        \"Best RMSE\": \"XGB\" if xgb_rmse < lgb_rmse else \"LGB\",\n        \"XGB MAPE\":  f\"{xgb_mape:.4f}%\",\n        \"LGB MAPE\":  f\"{lgb_mape:.4f}%\",\n        \"Best MAPE\": \"XGB\" if xgb_mape < lgb_mape else \"LGB\",\n    })\n\ncmp_df = pd.DataFrame(rows).set_index(\"Target\")\n\ndef highlight_best(val):\n    if val == \"XGB\":\n        return \"background-color: #1D9E75; color: white\"\n    elif val == \"LGB\":\n        return \"background-color: #378ADD; color: white\"\n    return \"\"\n\ncmp_df.style \\\n    .set_caption(\"Backtest Comparison — XGBoost vs LightGBM (Sep–Dec 2017)\") \\\n    .applymap(highlight_best, subset=[\"Best MAE\", \"Best RMSE\", \"Best MAPE\"]) \\\n    .set_properties(**{\"text-align\": \"center\"}) \\\n    .set_table_styles([\n        {\"selector\": \"caption\",\n         \"props\": [(\"font-size\",\"14px\"),(\"font-weight\",\"bold\"),(\"padding\",\"10px\")]},\n        {\"selector\": \"th\",\n         \"props\": [(\"background-color\",\"#222\"),(\"color\",\"#eee\"),(\"padding\",\"8px 14px\")]},\n        {\"selector\": \"td\",\n         \"props\": [(\"padding\",\"6px 14px\")]},\n    ])\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-04-29T18:55:19.866081Z","iopub.execute_input":"2026-04-29T18:55:19.866723Z","iopub.status.idle":"2026-04-29T18:55:19.902954Z","shell.execute_reply.started":"2026-04-29T18:55:19.866686Z","shell.execute_reply":"2026-04-29T18:55:19.902124Z"}},"outputs":[],"execution_count":null},{"id":"55d738cb-d36c-4e2a-bc44-51f5059fb2c1","cell_type":"markdown","source":"XGBoost wins on visits. Orders on average is better on LightGBM (low MAE) but could have siginificant outliers compared to XGBoost (higher RMSE). LightGBM wins on Revenue and CVR","metadata":{}},{"id":"fd4eefa7","cell_type":"markdown","source":"---\n## 11 · Backtest Chart — Sep–Dec 2017\n\n","metadata":{}},{"id":"d5553dd5","cell_type":"code","source":"# ── Cell · Chart — XGBoost vs LightGBM (Sept–Dec 2017 test window) ────────\nsept_weeks_dt = pd.to_datetime(\n    weekly[\"week\"].iloc[sept_start_idx: sept_start_idx + SEPT_WEEKS].values\n)\n\nfig, axes = plt.subplots(2, 2, figsize=(16, 10))\nfig.suptitle(\"Backtest — XGBoost vs LightGBM (2017-09-25 to 2017-12-18)\",\n             fontsize=13, color=C[\"text\"], fontfamily=\"monospace\", y=1.01)\n\nfor ax, target in zip(axes.flat, TARGETS):\n    xgb_r  = test_results_xgb_sept[target]\n    lgbm_r = test_results_lgb_sept[target]\n\n    # xgboost — line + dots\n    ax.plot(sept_weeks_dt, xgb_r[\"pred\"],\n            color=\"#00E5A0\", lw=2, marker=\"o\", markersize=6,\n            markeredgecolor=\"white\", markeredgewidth=1,\n            label=\"XGBoost forecast\", zorder=3)\n\n    # lightgbm — dashed line\n    ax.plot(sept_weeks_dt, lgbm_r[\"pred\"],\n            color=\"#60CFFF\", lw=2, marker=None,\n            ls=\"--\", label=\"LightGBM forecast\", zorder=2)\n\n    # actuals — x markers only, no line\n    ax.scatter(sept_weeks_dt, xgb_r[\"actual\"],\n               color=\"#FF6B6B\", marker=\"X\", s=100,\n               linewidths=1.5, edgecolors=\"white\",\n               label=\"Actuals\", zorder=5)\n\n    ax.set_title(LABELS[target], color=C[\"text\"], fontsize=11, loc=\"left\")\n    ax.xaxis.set_major_locator(mdates.WeekdayLocator(byweekday=mdates.MO))\n    ax.xaxis.set_major_formatter(mdates.DateFormatter(\"%b %d\"))\n    plt.setp(ax.get_xticklabels(), rotation=45, ha=\"right\", fontsize=8)\n    ax.set_xlim(sept_weeks_dt[0], sept_weeks_dt[-1])\n    ax.grid(True, axis=\"y\")\n    ax.legend(fontsize=8, framealpha=0.2)\n    fmt_yaxis(ax, target)\n\nfig.tight_layout()\nplt.show()","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-04-29T18:55:26.915001Z","iopub.execute_input":"2026-04-29T18:55:26.915958Z","iopub.status.idle":"2026-04-29T18:55:28.180800Z","shell.execute_reply.started":"2026-04-29T18:55:26.915921Z","shell.execute_reply":"2026-04-29T18:55:28.179384Z"}},"outputs":[],"execution_count":null},{"id":"a0c80ed7","cell_type":"markdown","source":"---\n## 12 · MAPE Comparison — XGBoost vs LightGBM\n\nSide-by-side MAPE across both test windows. Solid bars = XGBoost · Hatched bars = LightGBM.\n","metadata":{}},{"id":"81da59d1","cell_type":"code","source":"# ── MAPE Comparison Chart — XGBoost vs LightGBM ──────────────────────────\ndef get_metrics(r):\n    valid = ~np.isnan(r[\"pred\"]) & ~np.isnan(r[\"actual\"])\n    a, p  = r[\"actual\"][valid], r[\"pred\"][valid]\n    return {\n        \"mae\":  np.mean(np.abs(a - p)),\n        \"rmse\": np.sqrt(np.mean((a - p)**2)),\n        \"mape\": np.mean(np.abs(a - p) / (np.abs(a) + 1e-8)) * 100,\n    }\n\nxgb_jan  = {t: get_metrics(test_results[t])          for t in TARGETS}\nxgb_sept = {t: get_metrics(test_results_xgb_sept[t]) for t in TARGETS}\nlgb_jan  = {t: get_metrics(test_results_lgb[t])      for t in TARGETS}\nlgb_sept = {t: get_metrics(test_results_lgb_sept[t]) for t in TARGETS}\n\nfig, axes = plt.subplots(1, 2, figsize=(18, 7))\nfig.suptitle(\"XGBoost vs LightGBM — MAPE by KPI\\n\"\n             \"Direct Multi-Step Forecasting · 13-Week Horizon\",\n             fontsize=14, color=C[\"text\"], fontfamily=\"monospace\", y=1.02)\n\nx       = np.arange(len(TARGETS))\nwidth   = 0.35\nxlabels = [LABELS[t].replace(\" (USD)\", \"\") for t in TARGETS]\n\nperiods = [\n    (\"Jan–Apr 2018  —  Stable Period\\n(low volatility, tight visit range)\",\n     xgb_jan, lgb_jan),\n    (\"Sep–Dec 2017  —  Volatile Period\\n(includes BFCM spike weeks)\",\n     xgb_sept, lgb_sept),\n]\n\nfor ax, (title, xgb_d, lgb_d) in zip(axes, periods):\n    xgb_mapes = [xgb_d[t][\"mape\"] for t in TARGETS]\n    lgb_mapes = [lgb_d[t][\"mape\"] for t in TARGETS]\n\n    bars_x = ax.bar(x - width/2, xgb_mapes, width,\n                    color=[C[t] for t in TARGETS], alpha=0.9, label=\"XGBoost\")\n    bars_l = ax.bar(x + width/2, lgb_mapes, width,\n                    color=[C[t] for t in TARGETS], alpha=0.45,\n                    label=\"LightGBM\", hatch=\"//\")\n\n    max_val = max(max(xgb_mapes), max(lgb_mapes))\n    ax.set_ylim(0, max_val * 1.2)\n\n    for bar in list(bars_x) + list(bars_l):\n        h = bar.get_height()\n        ax.text(bar.get_x() + bar.get_width()/2, h + max_val * 0.02,\n                f\"{h:.1f}%\", ha=\"center\", va=\"bottom\", fontsize=8, color=C[\"text\"])\n\n    ax.set_title(title, color=C[\"text\"], fontsize=11, loc=\"left\", pad=10)\n    ax.set_xticks(x)\n    ax.set_xticklabels(xlabels, fontsize=10)\n    ax.set_ylabel(\"MAPE %\", fontsize=9)\n    ax.grid(True, axis=\"y\")\n    ax.legend(fontsize=9, framealpha=0.2)\n\nfig.tight_layout()\nplt.show()\n\n# ── winner summary ─────────────────────────────────────────────────────────\nprint(f\"\\n{'='*52}\")\nprint(f\"  Winner by MAPE — Holdout Results\")\nprint(f\"{'='*52}\")\nprint(f\"  {'KPI':<14} {'Jan–Apr 2018':>18}  {'Sep–Dec 2017':>16}\")\nprint(f\"  {'-'*50}\")\nfor target in TARGETS:\n    jan_w  = \"XGBoost\" if xgb_jan[target][\"mape\"]  <= lgb_jan[target][\"mape\"]  else \"LightGBM\"\n    sept_w = \"XGBoost\" if xgb_sept[target][\"mape\"] <= lgb_sept[target][\"mape\"] else \"LightGBM\"\n    label  = LABELS[target].replace(\" (USD)\", \"\")\n    print(f\"  {label:<14} {jan_w:>18}  {sept_w:>16}\")\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-04-29T18:55:53.529484Z","iopub.execute_input":"2026-04-29T18:55:53.530152Z","iopub.status.idle":"2026-04-29T18:55:54.165229Z","shell.execute_reply.started":"2026-04-29T18:55:53.530106Z","shell.execute_reply":"2026-04-29T18:55:54.163043Z"}},"outputs":[],"execution_count":null},{"id":"0e25b7a4","cell_type":"markdown","source":"---\n## 13 · Forward Forecast — Next 13 Weeks\n\nRetrain on **all 91 weeks**, predict 13 weeks beyond Apr 23 2018.\nConfidence bands = ±1 MAE (inner) and ±1.5 MAE (outer), derived from per-horizon backtest errors.\n","metadata":{}},{"id":"3658146b","cell_type":"code","source":"# ── Cell 8 · Forward Forecast — Next 13 Weeks (XGBoost) ──────────────────\ndef forecast_forward(weekly, target, n_steps):\n    df_feat    = add_features(weekly, target).reset_index(drop=True)\n    X_all      = df_feat[FEATURE_COLS]\n    feat_valid = X_all.notna().all(axis=1)\n\n    preds = []\n    for h in range(1, n_steps + 1):\n        y    = pd.Series(weekly[target].shift(-h).values, index=df_feat.index)\n        mask = feat_valid & y.notna()\n        m    = XGBRegressor(**XGB_PARAMS)\n        m.fit(X_all[mask], y[mask])\n        preds.append(m.predict(X_all.iloc[[-1]])[0])\n\n    future = pd.date_range(\n        weekly[\"week\"].iloc[-1] + pd.Timedelta(weeks=1),\n        periods=n_steps, freq=\"W-SUN\"\n    )\n    return np.array(preds), future\n\nprint(f\"Forecasting {FORECAST_WEEKS} weeks from {weekly['week'].iloc[-1].date()} ...\\n\")\nall_forecasts = {}\nfor target in TARGETS:\n    pred, fwks            = forecast_forward(weekly, target, FORECAST_WEEKS)\n    all_forecasts[target] = (pred, fwks)\n    vals = \"  \".join(FMT_STR[target].format(v) for v in pred[:4])\n    print(f\"  {target:<10}: {vals} ...\")\n\nprint(\"\\n✓ XGB forward forecast complete\")","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-04-29T19:11:25.480477Z","iopub.execute_input":"2026-04-29T19:11:25.480836Z","iopub.status.idle":"2026-04-29T19:11:32.089239Z","shell.execute_reply.started":"2026-04-29T19:11:25.480808Z","shell.execute_reply":"2026-04-29T19:11:32.088492Z"}},"outputs":[],"execution_count":null},{"id":"83b49b3f-c9a8-4227-abdd-557cce2640e2","cell_type":"code","source":"# ── Cell · Build LGB Forward Forecasts (mirrors forecast_forward with LGB) ──\ndef forecast_forward_lgb(weekly, target, n_steps):\n    df_feat    = add_features(weekly, target).reset_index(drop=True)\n    X_all      = df_feat[FEATURE_COLS]\n    feat_valid = X_all.notna().all(axis=1)\n\n    preds = []\n    for h in range(1, n_steps + 1):\n        y    = pd.Series(weekly[target].shift(-h).values, index=df_feat.index)\n        mask = feat_valid & y.notna()\n        m    = lgb.LGBMRegressor(**LGB_PARAMS_FINAL)\n        m.fit(X_all[mask], y[mask])\n        preds.append(m.predict(X_all.iloc[[-1]])[0])\n\n    future = pd.date_range(\n        weekly[\"week\"].iloc[-1] + pd.Timedelta(weeks=1),\n        periods=n_steps, freq=\"W-SUN\"\n    )\n    return np.array(preds), future\n\nall_forecasts_lgb = {}\nfor target in TARGETS:\n    pred, fwks              = forecast_forward_lgb(weekly, target, FORECAST_WEEKS)\n    all_forecasts_lgb[target] = (pred, fwks)\n    vals = \"  \".join(FMT_STR[target].format(v) for v in pred[:4])\n    print(f\"  {target:<10}: {vals} ...\")\n\nprint(\"\\n✓ LGB forward forecast complete\")","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-04-29T19:09:54.794758Z","iopub.execute_input":"2026-04-29T19:09:54.796429Z","iopub.status.idle":"2026-04-29T19:09:58.142900Z","shell.execute_reply.started":"2026-04-29T19:09:54.796368Z","shell.execute_reply":"2026-04-29T19:09:58.141518Z"}},"outputs":[],"execution_count":null},{"id":"58a16d8a","cell_type":"markdown","source":"---\n## 14 · Forecast Chart — All Four KPIs\n\nLast 30 weeks of history for context. XGBoost forecast in bright green and LightGBM forecast in dotted orange.\n","metadata":{}},{"id":"9438191e-d218-4245-a368-f38af08e7b2f","cell_type":"code","source":"all_metrics = {}\nfor target in TARGETS:\n    h_mae = []\n    for h in range(1, FORECAST_WEEKS + 1):\n        fold_errors = []\n        for fold in all_folds[target]:\n            if h <= len(fold[\"pred\"]):\n                err = abs(fold[\"actual\"][h-1] - fold[\"pred\"][h-1])\n                fold_errors.append(err)\n        h_mae.append(np.mean(fold_errors) if fold_errors else 0.0)\n    all_metrics[target] = {\"h_mae\": h_mae}\n\nprint(\"✓ all_metrics ready\")","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-04-29T19:10:47.955993Z","iopub.execute_input":"2026-04-29T19:10:47.956352Z","iopub.status.idle":"2026-04-29T19:10:47.965989Z","shell.execute_reply.started":"2026-04-29T19:10:47.956324Z","shell.execute_reply":"2026-04-29T19:10:47.964746Z"}},"outputs":[],"execution_count":null},{"id":"35e5c507","cell_type":"code","source":"# ── Cell · Forward Forecast Chart — Per Year Lines + XGB Next 13 Weeks ────\nimport matplotlib.pyplot as plt\nimport matplotlib.ticker as mticker\nimport numpy as np\n\nMONTH_WEEK = [1, 5, 9, 13, 18, 22, 26, 31, 35, 40, 44, 48]\nMONTHS     = [\"Jan\",\"Feb\",\"Mar\",\"Apr\",\"May\",\"Jun\",\n              \"Jul\",\"Aug\",\"Sep\",\"Oct\",\"Nov\",\"Dec\"]\n\nYEAR_COLORS = {\n    \"visits\":  {\"2016\": \"#9FE1CB\", \"2017\": \"#1D9E75\", \"2018\": \"#085041\"},\n    \"orders\":  {\"2016\": \"#FAC775\", \"2017\": \"#BA7517\", \"2018\": \"#633806\"},\n    \"revenue\": {\"2016\": \"#85B7EB\", \"2017\": \"#378ADD\", \"2018\": \"#0C447C\"},\n    \"cvr\":     {\"2016\": \"#AFA9EC\", \"2017\": \"#7F77DD\", \"2018\": \"#3C3489\"},\n}\n\nfig, axes = plt.subplots(4, 1, figsize=(18, 20))\nfig.suptitle(\"Weekly KPIs + XGBoost & LightGBM 13-Week Forecast — by Year\",\n             fontsize=13, color=C[\"text\"], fontfamily=\"monospace\", y=1.01)\n\nfor ax, target in zip(axes, TARGETS):\n    pred, fwks     = all_forecasts[target]\n    lgb_pred, _    = all_forecasts_lgb[target]          # ← LGB forecast\n    h_mae          = np.array(all_metrics[target][\"h_mae\"])\n    if len(h_mae) < FORECAST_WEEKS:\n        h_mae  = np.append(h_mae, np.full(FORECAST_WEEKS - len(h_mae), h_mae[-1]))\n\n    # ── actuals per year as lines ──────────────────────────────────────────\n    for year, ls in [(\"2016\", \"-\"), (\"2017\", \"-\"), (\"2018\", \"--\")]:\n        sub = weekly[weekly[\"week\"].dt.year == int(year)].copy()\n        sub[\"woy\"] = sub[\"week\"].dt.isocalendar().week.astype(int)\n        ax.plot(\n            sub[\"woy\"], sub[target],\n            color=YEAR_COLORS[target][year],\n            lw=1.5, ls=ls,\n            marker=\"o\", markersize=3,\n            label=f\"Actuals {year}\",\n            zorder=2,\n        )\n\n    # ── xgb forecast — next 13 weeks ──────────────────────────────────────\n    fwks_woy = pd.DatetimeIndex(fwks).isocalendar().week.astype(int).values\n\n    # confidence bands\n    ax.fill_between(fwks_woy, pred - 1.5*h_mae, pred + 1.5*h_mae,\n                    color=C[target], alpha=0.08, label=\"±1.5 MAE band\")\n    ax.fill_between(fwks_woy, pred - h_mae, pred + h_mae,\n                    color=C[target], alpha=0.16, label=\"±1 MAE band\")\n\n    # XGB forecast line\n    ax.plot(fwks_woy, pred,\n            color=\"#00E5A0\", lw=2.5, marker=\"o\", markersize=5,\n            markeredgecolor=\"white\", markeredgewidth=1,\n            label=\"XGB forecast\", zorder=5)\n\n    # ── LGB forecast line ─────────────────────────────────────────────────\n    ax.plot(fwks_woy, lgb_pred,\n            color=\"#FF9F43\", lw=2, marker=\"s\", markersize=4,\n            markeredgecolor=\"white\", markeredgewidth=1,\n            ls=\"-.\", label=\"LGB forecast\", zorder=4)   # ← added\n\n    # connector from last 2018 actual to first forecast point\n    last_2018 = weekly[weekly[\"week\"].dt.year == 2018].copy()\n    last_2018[\"woy\"] = last_2018[\"week\"].dt.isocalendar().week.astype(int)\n    ax.plot(\n        [last_2018[\"woy\"].iloc[-1], fwks_woy[0]],\n        [last_2018[target].iloc[-1], pred[0]],\n        color=\"#00E5A0\", lw=1.5, ls=\"--\", alpha=0.5, zorder=4,\n    )\n\n    # ── LGB connector ─────────────────────────────────────────────────────\n    ax.plot(\n        [last_2018[\"woy\"].iloc[-1], fwks_woy[0]],\n        [last_2018[target].iloc[-1], lgb_pred[0]],\n        color=\"#FF9F43\", lw=1.5, ls=\"--\", alpha=0.5, zorder=3,\n    )                                                        # ← added\n\n    # vertical divider at forecast start\n    ax.axvline(x=last_2018[\"woy\"].iloc[-1],\n               color=C[\"subtext\"], lw=1, ls=\"--\", alpha=0.5)\n\n    ax.set_title(LABELS[target], color=C[\"text\"], fontsize=11, loc=\"left\")\n    ax.set_xticks(MONTH_WEEK)\n    ax.set_xticklabels(MONTHS, fontsize=9)\n    ax.set_xlim(1, 53)\n    ax.grid(True, axis=\"y\")\n    ax.legend(fontsize=8, framealpha=0.2, ncol=3, loc=\"upper left\")\n    fmt_yaxis(ax, target)\n\nfig.tight_layout()\nplt.show()","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-04-29T19:10:52.297744Z","iopub.execute_input":"2026-04-29T19:10:52.298079Z","iopub.status.idle":"2026-04-29T19:10:53.791668Z","shell.execute_reply.started":"2026-04-29T19:10:52.298051Z","shell.execute_reply":"2026-04-29T19:10:53.789906Z"}},"outputs":[],"execution_count":null},{"id":"0aff7529","cell_type":"markdown","source":"---\n## 15 · 13-Week Forecast Table\n","metadata":{}},{"id":"2de785cb","cell_type":"code","source":"# ── Cell 11 · 13-Week Forecast Table ──────────────────────────────────────\nrows = []\nfor i, wk in enumerate(all_forecasts[\"visits\"][1]):\n    row = [wk.strftime(\"%Y-%m-%d\")]\n    for t in TARGETS:\n        row.append(FMT_STR[t].format(all_forecasts[t][0][i]))\n        row.append(FMT_STR[t].format(all_forecasts_lgb[t][0][i]))  # ← added\n    rows.append(row)\n\nfc_df = pd.DataFrame(rows, columns=[\n    \"Week\",\n    \"Visits XGB\",  \"Visits LGB\",\n    \"Orders XGB\",  \"Orders LGB\",\n    \"Revenue XGB\", \"Revenue LGB\",\n    \"CVR XGB\",     \"CVR LGB\",\n])\nfc_df.index = fc_df.index + 1\n\nfc_df.style \\\n    .set_caption(f\"13-Week Forward Forecast — XGBoost vs LightGBM — from {weekly['week'].iloc[-1].date()}\") \\\n    .set_properties(**{\"text-align\": \"center\", \"color\": \"#000000\", \"background-color\": \"#ffffff\"}) \\\n    .set_table_styles([\n        {\"selector\": \"caption\",\n         \"props\": [(\"font-size\",\"14px\"),(\"font-weight\",\"bold\"),(\"padding\",\"10px\"),(\"color\",\"#000000\")]},\n        {\"selector\": \"th\",\n         \"props\": [(\"background-color\",\"#f0f0f0\"),(\"color\",\"#000000\"),(\"padding\",\"8px 14px\"),(\"border\",\"1px solid #cccccc\")]},\n        {\"selector\": \"td\",\n         \"props\": [(\"padding\",\"6px 14px\"),(\"color\",\"#000000\"),(\"background-color\",\"#ffffff\"),(\"border\",\"1px solid #eeeeee\")]},\n        {\"selector\": \"tr:nth-child(even) td\",\n         \"props\": [(\"background-color\",\"#f9f9f9\")]},\n        {\"selector\": \"tr:hover td\",\n         \"props\": [(\"background-color\",\"#e8f4ff\")]},\n    ])","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-04-29T19:12:43.822648Z","iopub.execute_input":"2026-04-29T19:12:43.823085Z","iopub.status.idle":"2026-04-29T19:12:43.846431Z","shell.execute_reply.started":"2026-04-29T19:12:43.823053Z","shell.execute_reply":"2026-04-29T19:12:43.845112Z"}},"outputs":[],"execution_count":null},{"id":"383ca0bd-ff9b-4391-986f-2bf2b35acea1","cell_type":"code","source":"# ── Ensemble forecast — weighted blend per KPI ────────────────────────────\nWEIGHTS = {\n    \"visits\":  (0.8, 0.2),   # (xgb_weight, lgb_weight)\n    \"orders\":  (0.5, 0.5),\n    \"revenue\": (0.5, 0.5),\n    \"cvr\":     (0.2, 0.8),\n}\n\nall_forecasts_ensemble = {}\nfor target in TARGETS:\n    xgb_w, lgb_w = WEIGHTS[target]\n    xgb_pred     = all_forecasts[target][0]\n    lgb_pred     = all_forecasts_lgb[target][0]\n    fwks         = all_forecasts[target][1]\n    ensemble     = xgb_w * xgb_pred + lgb_w * lgb_pred\n    all_forecasts_ensemble[target] = (ensemble, fwks)\n    print(f\"{target:<10}  XGB weight={xgb_w}  LGB weight={lgb_w}\")\n\nprint(\"\\n✓ ensemble forecasts ready\")","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-04-29T19:12:35.824879Z","iopub.execute_input":"2026-04-29T19:12:35.825364Z","iopub.status.idle":"2026-04-29T19:12:35.838315Z","shell.execute_reply.started":"2026-04-29T19:12:35.825309Z","shell.execute_reply":"2026-04-29T19:12:35.835318Z"}},"outputs":[],"execution_count":null},{"id":"77b0a897-9f4b-4eeb-bd87-594dbc0b44b5","cell_type":"code","source":"# ── Ensemble 13-Week Forecast Table ───────────────────────────────────────\nrows = []\nfor i, wk in enumerate(all_forecasts_ensemble[\"visits\"][1]):\n    row = [wk.strftime(\"%Y-%m-%d\")]\n    for t in TARGETS:\n        row.append(FMT_STR[t].format(all_forecasts_ensemble[t][0][i]))\n    rows.append(row)\n\nfc_df = pd.DataFrame(rows, columns=[\"Week\", \"Visits\", \"Orders\", \"Revenue\", \"CVR\"])\nfc_df.index = fc_df.index + 1\n\nfc_df.style \\\n    .set_caption(f\"13-Week Ensemble Forecast — from {weekly['week'].iloc[-1].date()}\") \\\n    .set_properties(**{\"text-align\": \"center\", \"color\": \"#000000\",\n                       \"background-color\": \"#ffffff\"}) \\\n    .set_table_styles([\n        {\"selector\": \"caption\",\n         \"props\": [(\"font-size\",\"14px\"),(\"font-weight\",\"bold\"),\n                   (\"padding\",\"10px\"),(\"color\",\"#000000\")]},\n        {\"selector\": \"th\",\n         \"props\": [(\"background-color\",\"#f0f0f0\"),(\"color\",\"#000000\"),\n                   (\"padding\",\"8px 14px\"),(\"border\",\"1px solid #cccccc\")]},\n        {\"selector\": \"td\",\n         \"props\": [(\"padding\",\"6px 14px\"),(\"color\",\"#000000\"),\n                   (\"background-color\",\"#ffffff\"),(\"border\",\"1px solid #eeeeee\")]},\n        {\"selector\": \"tr:nth-child(even) td\",\n         \"props\": [(\"background-color\",\"#f9f9f9\")]},\n        {\"selector\": \"tr:hover td\",\n         \"props\": [(\"background-color\",\"#e8f4ff\")]},\n    ])","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-04-29T19:12:53.851678Z","iopub.execute_input":"2026-04-29T19:12:53.852158Z","iopub.status.idle":"2026-04-29T19:12:53.882075Z","shell.execute_reply.started":"2026-04-29T19:12:53.852119Z","shell.execute_reply":"2026-04-29T19:12:53.878743Z"}},"outputs":[],"execution_count":null},{"id":"f094f103-b614-4049-b474-7e924fdc407f","cell_type":"markdown","source":"used weights noted above","metadata":{}},{"id":"50aedc5b","cell_type":"markdown","source":"---\n## 16 · Baseline Comparison\n\nXGBoost vs three simple baselines: last value, 4-week mean, seasonal naive.  \nIf XGBoost cannot beat these, the model adds no value over trivial heuristics.\n","metadata":{}},{"id":"1463c432","cell_type":"code","source":"# ── Cell · Simple Baselines — Jan-Apr 2018 Holdout ─────────────────────\ndef simple_baselines_for_test(weekly, target, test_start, horizon):\n    actual = weekly[target].iloc[test_start:test_start + horizon].values\n\n    last_val = weekly[target].iloc[test_start - 1]\n    pred_last = np.repeat(last_val, len(actual))\n\n    pred_roll4 = np.repeat(\n        weekly[target].iloc[max(0, test_start - 4):test_start].mean(),\n        len(actual)\n    )\n\n    if test_start - 52 >= 0:\n        pred_snaive = weekly[target].iloc[\n            test_start - 52:test_start - 52 + len(actual)\n        ].values\n    else:\n        pred_snaive = np.full(len(actual), np.nan)\n\n    return {\n        \"actual\": actual,\n        \"Last value\": pred_last,\n        \"4-week mean\": pred_roll4,\n        \"Seasonal naive (lag_52)\": pred_snaive,\n    }\n\n\nbaseline_rows = []\ntest_start = len(weekly) - BACKTEST_WEEKS\n\nfor target in TARGETS:\n    xgb_r = test_results[target]\n    base = simple_baselines_for_test(weekly, target, test_start, BACKTEST_WEEKS)\n    actual = base[\"actual\"]\n\n    candidates = {\n        \"XGBoost\": xgb_r[\"pred\"],\n        \"Last value\": base[\"Last value\"],\n        \"4-week mean\": base[\"4-week mean\"],\n        \"Seasonal naive (lag_52)\": base[\"Seasonal naive (lag_52)\"],\n    }\n\n    for model_name, pred in candidates.items():\n        valid = ~np.isnan(pred)\n        baseline_rows.append({\n            \"Target\": LABELS[target],\n            \"Model\": model_name,\n            \"MAE\": mean_absolute_error(actual[valid], pred[valid]),\n            \"RMSE\": np.sqrt(mean_squared_error(actual[valid], pred[valid])),\n            \"MAPE %\": np.mean(\n                np.abs(actual[valid] - pred[valid]) / (np.abs(actual[valid]) + 1e-8)\n            ) * 100,\n        })\n\nbaseline_df = pd.DataFrame(baseline_rows)\nbaseline_df = baseline_df.sort_values([\"Target\", \"MAPE %\", \"MAE\"]).reset_index(drop=True)\nbaseline_df\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-04-29T19:12:59.918469Z","iopub.execute_input":"2026-04-29T19:12:59.919040Z","iopub.status.idle":"2026-04-29T19:12:59.976263Z","shell.execute_reply.started":"2026-04-29T19:12:59.919004Z","shell.execute_reply":"2026-04-29T19:12:59.975097Z"}},"outputs":[],"execution_count":null},{"id":"83ac3e89","cell_type":"code","source":"# ── Cell · Simple Baselines — Sep-Dec 2017 Holdout ─────────────────────\ndef simple_baselines_for_window(weekly, target, origin_idx, horizon):\n    actual = weekly[target].iloc[origin_idx:origin_idx + horizon].values\n\n    last_val = weekly[target].iloc[origin_idx - 1]\n    pred_last = np.repeat(last_val, len(actual))\n\n    pred_roll4 = np.repeat(\n        weekly[target].iloc[max(0, origin_idx - 4):origin_idx].mean(),\n        len(actual)\n    )\n\n    baselines = {\n        \"actual\": actual,\n        \"Last value\": pred_last,\n        \"4-week mean\": pred_roll4,\n    }\n\n    # seasonal naive only if enough history exists\n    if origin_idx - 52 >= 0:\n        pred_snaive = weekly[target].iloc[\n            origin_idx - 52: origin_idx - 52 + len(actual)\n        ].values\n        baselines[\"Seasonal naive (lag_52)\"] = pred_snaive\n\n    return baselines\n\n\nsept_baseline_rows = []\n\nfor target in TARGETS:\n    xgb_r = test_results_xgb_sept[target]\n    base = simple_baselines_for_window(weekly, target, sept_start_idx, SEPT_WEEKS)\n    actual = base[\"actual\"]\n\n    candidates = {\"XGBoost\": xgb_r[\"pred\"]}\n    for k, v in base.items():\n        if k != \"actual\":\n            candidates[k] = v\n\n    for model_name, pred in candidates.items():\n        valid = ~np.isnan(pred)\n        sept_baseline_rows.append({\n            \"Target\": LABELS[target],\n            \"Model\": model_name,\n            \"MAE\": mean_absolute_error(actual[valid], pred[valid]),\n            \"RMSE\": np.sqrt(mean_squared_error(actual[valid], pred[valid])),\n            \"MAPE %\": np.mean(\n                np.abs(actual[valid] - pred[valid]) / (np.abs(actual[valid]) + 1e-8)\n            ) * 100,\n        })\n\nsept_baseline_df = pd.DataFrame(sept_baseline_rows)\nsept_baseline_df = sept_baseline_df.sort_values([\"Target\", \"MAPE %\", \"MAE\"]).reset_index(drop=True)\nsept_baseline_df\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-04-29T19:13:03.782142Z","iopub.execute_input":"2026-04-29T19:13:03.783003Z","iopub.status.idle":"2026-04-29T19:13:03.855349Z","shell.execute_reply.started":"2026-04-29T19:13:03.782965Z","shell.execute_reply":"2026-04-29T19:13:03.853643Z"}},"outputs":[],"execution_count":null},{"id":"b13c18d3","cell_type":"markdown","source":"---\n## 17 · Feature Ablation\n\nFeatures removed one group at a time to measure individual contribution - ran for XGBoost Only.\n","metadata":{}},{"id":"fe55676f","cell_type":"code","source":"# ── Cell · Feature Ablation — Final 13-Week Holdout ────────────────────\ndef compute_holdout_metrics(result):\n    pred   = result[\"pred\"]\n    actual = result[\"actual\"]\n    valid  = ~np.isnan(pred) & ~np.isnan(actual)\n\n    mae  = mean_absolute_error(actual[valid], pred[valid])\n    rmse = np.sqrt(mean_squared_error(actual[valid], pred[valid]))\n    mape = np.mean(np.abs(actual[valid] - pred[valid]) /\n                   (np.abs(actual[valid]) + 1e-8)) * 100\n    return {\"mae\": mae, \"rmse\": rmse, \"mape\": mape}\n\n\ndef build_feature_cols(use_lag_52=True, use_rolls=True, use_holidays=True):\n    lags = [8, 12, 16, 20] + ([52] if use_lag_52 else [])\n\n    cols = [f\"lag_{l}\" for l in lags]\n\n    if use_rolls:\n        cols += [f\"roll_mean_{w}\" for w in ROLLS]\n        cols += [f\"roll_std_{w}\" for w in ROLLS]\n\n    cols += [\"week_of_year\", \"month\", \"quarter\", \"year\", \"t\"]\n\n    if use_holidays:\n        cols += [\"is_bfcm\", \"is_xmas\", \"is_newyear\"]\n\n    return cols\n\n\ndef run_backtest_ablation(weekly, target, feature_cols):\n    n          = len(weekly)\n    test_start = n - BACKTEST_WEEKS\n    min_train  = 60\n    n_folds    = 8\n\n    df_feat    = add_features(weekly, target).reset_index(drop=True)\n    X_all      = df_feat[feature_cols]\n    feat_valid = X_all.notna().all(axis=1)\n\n    step    = max(1, (test_start - min_train) // n_folds)\n    origins = list(range(min_train, test_start, step))[:n_folds]\n    if test_start not in origins:\n        origins.append(test_start)\n\n    test_result = None\n\n    for origin in origins:\n        preds = []\n        for h in range(1, BACKTEST_WEEKS + 1):\n            y    = pd.Series(weekly[target].shift(-h).values, index=df_feat.index)\n            mask = (df_feat.index < origin) & feat_valid & y.notna()\n            if mask.sum() < 10:\n                preds.append(np.nan)\n                continue\n\n            m = XGBRegressor(**XGB_PARAMS)\n            m.fit(X_all[mask], y[mask])\n            preds.append(m.predict(X_all.iloc[[origin - 1]])[0])\n\n        actual = weekly[target].iloc[origin: origin + BACKTEST_WEEKS].values\n        weeks  = weekly[\"week\"].iloc[origin: origin + BACKTEST_WEEKS].values\n        result = {\n            \"origin\": origin,\n            \"pred\": np.array(preds[:len(actual)]),\n            \"actual\": actual,\n            \"weeks\": weeks,\n        }\n\n        if origin == test_start:\n            test_result = result\n\n    return test_result\n\n\nablations = {\n    \"Full model\": build_feature_cols(use_lag_52=True, use_rolls=True, use_holidays=True),\n    \"No lag_52\": build_feature_cols(use_lag_52=False, use_rolls=True, use_holidays=True),\n    \"No holiday flags\": build_feature_cols(use_lag_52=True, use_rolls=True, use_holidays=False),\n    \"No rolling stats\": build_feature_cols(use_lag_52=True, use_rolls=False, use_holidays=True),\n}\n\nablation_rows = []\n\nfor target in TARGETS:\n    for ablation_name, feature_cols in ablations.items():\n        result = run_backtest_ablation(weekly, target, feature_cols)\n        m = compute_holdout_metrics(result)\n        ablation_rows.append({\n            \"Target\": LABELS[target],\n            \"Model\": ablation_name,\n            \"MAE\": m[\"mae\"],\n            \"RMSE\": m[\"rmse\"],\n            \"MAPE %\": m[\"mape\"],\n        })\n\nablation_df = pd.DataFrame(ablation_rows)\nablation_df = ablation_df.sort_values([\"Target\", \"MAPE %\", \"MAE\"]).reset_index(drop=True)\nablation_df\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-04-29T19:13:07.328186Z","iopub.execute_input":"2026-04-29T19:13:07.328730Z","iopub.status.idle":"2026-04-29T19:16:12.350615Z","shell.execute_reply.started":"2026-04-29T19:13:07.328696Z","shell.execute_reply":"2026-04-29T19:16:12.349596Z"}},"outputs":[],"execution_count":null},{"id":"73730379","cell_type":"code","source":"# ── Cell · Feature Ablation — Sep-Dec 2017 Holdout ─────────────────────\ndef compute_window_metrics(result):\n    pred   = result[\"pred\"]\n    actual = result[\"actual\"]\n    valid  = ~np.isnan(pred) & ~np.isnan(actual)\n\n    mae  = mean_absolute_error(actual[valid], pred[valid])\n    rmse = np.sqrt(mean_squared_error(actual[valid], pred[valid]))\n    mape = np.mean(np.abs(actual[valid] - pred[valid]) /\n                   (np.abs(actual[valid]) + 1e-8)) * 100\n    return {\"mae\": mae, \"rmse\": rmse, \"mape\": mape}\n\n\ndef build_feature_cols_sept(use_rolls=True, use_holidays=True):\n    cols = [f\"lag_{l}\" for l in LAGS_SEPT]\n\n    if use_rolls:\n        cols += [f\"roll_mean_{w}\" for w in ROLLS]\n        cols += [f\"roll_std_{w}\" for w in ROLLS]\n\n    cols += [\"week_of_year\", \"month\", \"quarter\", \"year\", \"t\"]\n\n    if use_holidays:\n        cols += [\"is_bfcm\", \"is_xmas\", \"is_newyear\"]\n\n    return cols\n\n\ndef run_backtest_window_ablation(weekly, target, origin_idx, n_steps, feature_cols, model=\"xgb\"):\n    df_feat    = add_features_sept(weekly, target).reset_index(drop=True)\n    X_all      = df_feat[feature_cols]\n    feat_valid = X_all.notna().all(axis=1)\n\n    preds = []\n    for h in range(1, n_steps + 1):\n        y    = pd.Series(weekly[target].shift(-h).values, index=df_feat.index)\n        mask = (df_feat.index < origin_idx) & feat_valid & y.notna()\n        if mask.sum() < 5:\n            preds.append(np.nan)\n            continue\n\n        if model == \"xgb\":\n            m = XGBRegressor(**XGB_PARAMS)\n        else:\n            m = lgb.LGBMRegressor(**LGBM_PARAMS)\n\n        m.fit(X_all[mask], y[mask])\n        preds.append(m.predict(X_all.iloc[[origin_idx - 1]])[0])\n\n    actual = weekly[target].iloc[origin_idx: origin_idx + n_steps].values\n    weeks  = weekly[\"week\"].iloc[origin_idx: origin_idx + n_steps].values\n\n    return {\n        \"pred\": np.array(preds[:len(actual)]),\n        \"actual\": actual,\n        \"weeks\": weeks,\n    }\n\n\nsept_ablations = {\n    \"Full model\": build_feature_cols_sept(use_rolls=True, use_holidays=True),\n    \"No holiday flags\": build_feature_cols_sept(use_rolls=True, use_holidays=False),\n    \"No rolling stats\": build_feature_cols_sept(use_rolls=False, use_holidays=True),\n}\n\nsept_ablation_rows = []\n\nfor target in TARGETS:\n    for ablation_name, feature_cols in sept_ablations.items():\n        result = run_backtest_window_ablation(\n            weekly, target, sept_start_idx, SEPT_WEEKS, feature_cols, model=\"xgb\"\n        )\n        m = compute_window_metrics(result)\n        sept_ablation_rows.append({\n            \"Target\": LABELS[target],\n            \"Model\": ablation_name,\n            \"MAE\": m[\"mae\"],\n            \"RMSE\": m[\"rmse\"],\n            \"MAPE %\": m[\"mape\"],\n        })\n\nsept_ablation_df = pd.DataFrame(sept_ablation_rows)\nsept_ablation_df = sept_ablation_df.sort_values([\"Target\", \"MAPE %\", \"MAE\"]).reset_index(drop=True)\nsept_ablation_df\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-04-29T19:17:15.290662Z","iopub.execute_input":"2026-04-29T19:17:15.291059Z","iopub.status.idle":"2026-04-29T19:17:33.137149Z","shell.execute_reply.started":"2026-04-29T19:17:15.291028Z","shell.execute_reply":"2026-04-29T19:17:33.136215Z"}},"outputs":[],"execution_count":null},{"id":"0e97b823","cell_type":"markdown","source":"---\n## 18 · Sanity Check — Is the Model Actually Learning?\n\nReal XGBoost vs an identical model trained on **randomly shuffled labels**.","metadata":{}},{"id":"5d59f346-3593-4490-885e-998041ad1e7f","cell_type":"code","source":"# ── Sanity Check — Real Model vs Shuffled Labels ──────────────────────────\nfrom sklearn.utils import shuffle as sk_shuffle\n\ntarget     = \"visits\"\norigin     = len(weekly) - BACKTEST_WEEKS   # same origin as holdout\n\ndf_feat    = add_features(weekly, target).reset_index(drop=True)\nX_all      = df_feat[FEATURE_COLS]\nfeat_valid = X_all.notna().all(axis=1)\n\n# h=1 model only — predicts 1 week ahead from origin\ny    = pd.Series(weekly[target].shift(-1).values, index=df_feat.index)\nmask = (df_feat.index < origin) & feat_valid & y.notna()\n\nX_train = X_all[mask]\ny_train = y[mask]\nactual  = weekly[target].iloc[origin]\n\n# ── real model ─────────────────────────────────────────────────────────────\nm_real = XGBRegressor(**XGB_PARAMS)\nm_real.fit(X_train, y_train)\npred_real = m_real.predict(X_all.iloc[[origin - 1]])[0]\n\n# ── shuffled labels model ──────────────────────────────────────────────────\ny_shuffled = pd.Series(\n    sk_shuffle(y_train.values, random_state=42),\n    index=y_train.index\n)\nm_shuf = XGBRegressor(**XGB_PARAMS)\nm_shuf.fit(X_train, y_shuffled)\npred_shuf = m_shuf.predict(X_all.iloc[[origin - 1]])[0]\n\n# ── results ────────────────────────────────────────────────────────────────\nreal_err = abs(pred_real - actual)\nshuf_err = abs(pred_shuf - actual)\nratio    = shuf_err / real_err\n\nprint(f\"Sanity Check — Real XGBoost vs Shuffled Labels\")\nprint(f\"{'='*50}\")\nprint(f\"Actual visits (week {weekly['week'].iloc[origin].date()}): \"\n      f\"{actual:,.0f}\")\nprint(f\"Real XGBoost prediction:   {pred_real:,.0f}  \"\n      f\"→ error: {real_err:.0f} visits\")\nprint(f\"Shuffled-label prediction: {pred_shuf:,.0f}  \"\n      f\"→ error: {shuf_err:.0f} visits\")\nprint(f\"{'='*50}\")\nprint(f\"Shuffled model errors {ratio:.0f}× more than real model\")\nprint()\nif ratio > 10:\n    print(\"✓ No data leakage — model is learning genuine temporal signal\")\nelse:\n    print(\"⚠ Small gap — investigate potential leakage\")","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-04-29T19:17:50.072569Z","iopub.execute_input":"2026-04-29T19:17:50.072966Z","iopub.status.idle":"2026-04-29T19:17:50.334788Z","shell.execute_reply.started":"2026-04-29T19:17:50.072934Z","shell.execute_reply":"2026-04-29T19:17:50.334065Z"}},"outputs":[],"execution_count":null},{"id":"6cbabe71-eb19-4649-ba20-64a550f03378","cell_type":"markdown","source":"---\n## 19 · Conclusions\n\n### Model Results","metadata":{}},{"id":"d657f146-36b4-43ac-b632-c44c88fe749b","cell_type":"code","source":"# get correct holdout results for both periods — both models\nprint(\"Jan–Apr 2018 Holdout:\")\nprint(f\"{'KPI':<12} {'XGB MAE':>10} {'XGB MAPE':>10} {'LGB MAE':>10} {'LGB MAPE':>10}\")\nprint(\"-\" * 55)\nfor target in TARGETS:\n    xm = compute_holdout_metrics(test_results[target])\n    lm = compute_holdout_metrics(test_results_lgb[target])\n    print(f\"{target:<12} {xm['mae']:>10.2f} {xm['mape']:>9.2f}%  \"\n          f\"{lm['mae']:>10.2f} {lm['mape']:>9.2f}%\")\n\nprint()\nprint(\"Sep–Dec 2017 Holdout:\")\nprint(f\"{'KPI':<12} {'XGB MAE':>10} {'XGB MAPE':>10} {'LGB MAE':>10} {'LGB MAPE':>10}\")\nprint(\"-\" * 55)\nfor target in TARGETS:\n    xm = compute_holdout_metrics(test_results_xgb_sept[target])\n    lm = compute_holdout_metrics(test_results_lgb_sept[target])\n    print(f\"{target:<12} {xm['mae']:>10.2f} {xm['mape']:>9.2f}%  \"\n          f\"{lm['mae']:>10.2f} {lm['mape']:>9.2f}%\")","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-04-29T19:18:04.278339Z","iopub.execute_input":"2026-04-29T19:18:04.278665Z","iopub.status.idle":"2026-04-29T19:18:04.301582Z","shell.execute_reply.started":"2026-04-29T19:18:04.278638Z","shell.execute_reply":"2026-04-29T19:18:04.300610Z"}},"outputs":[],"execution_count":null},{"id":"15816731","cell_type":"markdown","source":"\n\n### Model Winner\n\n| KPI | Jan–Apr 2018 | Sep–Dec 2017 |\n|---|---|---|\n| Visits | XGBoost | XGBoost |\n| Orders | XGBoost | LightGBM |\n| Revenue | XGBoost | LightGBM |\n| CVR | LightGBM | LightGBM |\n\nXGBoost wins consistently on visits. Orders and revenue split by period — XGBoost in Jan-Apr, LightGBM in Sep-Dec. LightGBM wins CVR in both periods. Neither model dominates cleanly across all conditions, suggesting an ensemble would outperform either alone.\n\n### Key Design Decisions\n\n1. **Direct over recursive** — no error propagation across 13 steps\n2. **Walk-forward with fixed holdout** — WF metrics average 8 folds; holdout evaluates the final real test\n3. **Two test windows** — honest picture across stable and volatile conditions\n4. **Sanity check** — 700× shuffled model gap confirms no leakage\n\n### Limitations & Extensions\n\n| Limitation | Potential Extension |\n|---|---|\n| No external signals | Add marketing spend, promotions, ad impressions |\n| Spike weeks under-forecast | Anomaly detection layer before forecasting |\n| 91 weeks of training | More history improves seasonal learning |\n| Single model per horizon | Ensemble XGB + LGB for robustness |\n| CVR modelled directly | Forecast visits and orders separately, derive CVR |\n\n---\n*Built with Python · XGBoost · LightGBM · pandas · matplotlib*\n*Dataset: [Google Analytics Customer Revenue Prediction](https://www.kaggle.com/c/ga-customer-revenue-prediction)*\n","metadata":{}}]}