🙏 Thanks to the WiDS organizers for an excellent and thought‑provoking competition - small data, real constraints, and a genuinely unique and interesting survival setup.
\nThis competition looked simple on paper — survival prediction at multiple horizons — but the real challenge was the data size. With only 221 training rows, most boosted models overfit before they learn anything. Instead of boosting harder, this solution focused on reducing variance by combining 150 survival models, validated with event-time-aware repeated stratified cross‑validation (≤24h / ≤48h / later / censored).
\nCheck out the full notebook HERE.
\nBoosting is powerful on large datasets, but on tiny data it often amplifies noise and fails to generalize.
\nEarly finding: As I also pointed in the discussions, the curse of dimensionality is very real here. With such small data, feature engineering isn't optional - it is the model.
\nGuiding principle: encode the physics, nonlinearities and some of the interactions; keep those with strong signal, that also remain stable across models.
\nCheck the models section for a thorough explanation on the features engineered for each model.
\nThe configs deliberately spread to create model diversity first, then reduce variance by averaging.
\nGuiding principle: Each model is imperfect; the ensemble is (very) stable.
\nlog_distance, proximity_risk (inverse distance transformation) → fire impact diminishes nonlinearly with distancehazard_potential (size x distance) → big things nearby are dangerousalignment_abs (directionality) → is the fire moving towards the asset? could matter more than raw speeddist_slope_ci_0_5h (trend) → captures the change in the first 5 hours (meters/per hour)event_start_month (seasonality) → incorporates any potential seasonal patters log1p_area_first (relative fire size) → grows quickly at first as area increasesA small set of features for trees on data this small led to fewer splitting paths, hence less overfitting.
\nGuiding principle: Cox is often wrong, but wrong differently, which is exactly what the ensemble needs.
\nalignment_abs, log1p_area_first, num_perimeters_0_5hlog_distance, area_to_dist_ratio, fire_radius_km (√(area / π)) → guide nonlinear effects with explicit physics-informed transformslog_eta (basically log(distance / closing speed)), eta_effective (calculated as distance / (closing speed + radial growth)) → time to arrival dynamics; fire risk is time-based and CoxPH naturally models hazard over timenear_speed_rank, far_threat_rank (rank features) → ranks are robust to outliers is_summer, is_afternoon (12-17), zone_near (within 5km), zone_far (over 5km), has_movement (reached more than 1 perimeter) → binary context flags to guide the model to interpretable findingsthreat_score (alignment × speed / log-distance) → composite risk that encodes direction, speed and distance (thus an interaction)Why more features? Regularization shrinks weak signals toward zero.
\n90% GBSA + 10% CoxPH won consistently in cross-validation tests.
\nRandom folds can land badly - too few short-time events in one fold, too many censored cases in another.
\nCross-validation was done by defining an event-time strata (censored / hit by 24h / hit by 48h / hit later) and use RepeatedStratifiedKFold with 3 repeats × 5 folds.
Models: GBSA (8 configs × 10 seeds) + CoxPH (7 alphas × 10 seeds)\nBlend: 90% GBSA + 10% CoxPH, 72h=1.0\nCV: RepeatedStratifiedKFold(5, 3), event+time strata\n c_index : 0.94714 ± 0.01604 (range: 0.92016 - 0.97872)\n brier_12 : 0.05391 ± 0.01179 (range: 0.03554 - 0.07416)\n brier_24 : 0.02572 ± 0.00882 (range: 0.01377 - 0.03897)\n brier_48 : 0.01310 ± 0.00639 (range: 0.00251 - 0.02359)\n brier_72 : 0.00000 ± 0.00000 (range: 0.00000 - 0.00000)\n weighted_brier : 0.01296 ± 0.00243 (range: 0.00893 - 0.01725)\n hybrid : 0.97507 ± 0.00519 (range: 0.96823 - 0.98610)\nprob_72h = 1.0, which quietly wins the weighted BrierFull implementation, details and approach are in the notebook HERE.
", "comments": [], "comment_count": 0, "messages": [ { "id": 3456258, "postDate": "2026-05-11T15:01:52.447Z", "content": "🙏 Thanks to the WiDS organizers for an excellent and thought‑provoking competition - small data, real constraints, and a genuinely unique and interesting survival setup.
\nThis competition looked simple on paper — survival prediction at multiple horizons — but the real challenge was the data size. With only 221 training rows, most boosted models overfit before they learn anything. Instead of boosting harder, this solution focused on reducing variance by combining 150 survival models, validated with event-time-aware repeated stratified cross‑validation (≤24h / ≤48h / later / censored).
\nCheck out the full notebook HERE.
\nBoosting is powerful on large datasets, but on tiny data it often amplifies noise and fails to generalize.
\nEarly finding: As I also pointed in the discussions, the curse of dimensionality is very real here. With such small data, feature engineering isn't optional - it is the model.
\nGuiding principle: encode the physics, nonlinearities and some of the interactions; keep those with strong signal, that also remain stable across models.
\nCheck the models section for a thorough explanation on the features engineered for each model.
\nThe configs deliberately spread to create model diversity first, then reduce variance by averaging.
\nGuiding principle: Each model is imperfect; the ensemble is (very) stable.
\nlog_distance, proximity_risk (inverse distance transformation) → fire impact diminishes nonlinearly with distancehazard_potential (size x distance) → big things nearby are dangerousalignment_abs (directionality) → is the fire moving towards the asset? could matter more than raw speeddist_slope_ci_0_5h (trend) → captures the change in the first 5 hours (meters/per hour)event_start_month (seasonality) → incorporates any potential seasonal patters log1p_area_first (relative fire size) → grows quickly at first as area increasesA small set of features for trees on data this small led to fewer splitting paths, hence less overfitting.
\nGuiding principle: Cox is often wrong, but wrong differently, which is exactly what the ensemble needs.
\nalignment_abs, log1p_area_first, num_perimeters_0_5hlog_distance, area_to_dist_ratio, fire_radius_km (√(area / π)) → guide nonlinear effects with explicit physics-informed transformslog_eta (basically log(distance / closing speed)), eta_effective (calculated as distance / (closing speed + radial growth)) → time to arrival dynamics; fire risk is time-based and CoxPH naturally models hazard over timenear_speed_rank, far_threat_rank (rank features) → ranks are robust to outliers is_summer, is_afternoon (12-17), zone_near (within 5km), zone_far (over 5km), has_movement (reached more than 1 perimeter) → binary context flags to guide the model to interpretable findingsthreat_score (alignment × speed / log-distance) → composite risk that encodes direction, speed and distance (thus an interaction)Why more features? Regularization shrinks weak signals toward zero.
\n90% GBSA + 10% CoxPH won consistently in cross-validation tests.
\nRandom folds can land badly - too few short-time events in one fold, too many censored cases in another.
\nCross-validation was done by defining an event-time strata (censored / hit by 24h / hit by 48h / hit later) and use RepeatedStratifiedKFold with 3 repeats × 5 folds.
Models: GBSA (8 configs × 10 seeds) + CoxPH (7 alphas × 10 seeds)\nBlend: 90% GBSA + 10% CoxPH, 72h=1.0\nCV: RepeatedStratifiedKFold(5, 3), event+time strata\n c_index : 0.94714 ± 0.01604 (range: 0.92016 - 0.97872)\n brier_12 : 0.05391 ± 0.01179 (range: 0.03554 - 0.07416)\n brier_24 : 0.02572 ± 0.00882 (range: 0.01377 - 0.03897)\n brier_48 : 0.01310 ± 0.00639 (range: 0.00251 - 0.02359)\n brier_72 : 0.00000 ± 0.00000 (range: 0.00000 - 0.00000)\n weighted_brier : 0.01296 ± 0.00243 (range: 0.00893 - 0.01725)\n hybrid : 0.97507 ± 0.00519 (range: 0.96823 - 0.98610)\nprob_72h = 1.0, which quietly wins the weighted BrierFull implementation, details and approach are in the notebook HERE.
", "rawMarkdown": "🙏 **Thanks to the WiDS organizers** for an excellent and thought‑provoking competition - small data, real constraints, and a genuinely unique and interesting survival setup.\n\n# 📌 TL;DR\nThis competition looked simple on paper — survival prediction at multiple horizons — but the real challenge was the data size. With only 221 training rows, most boosted models overfit before they learn anything. Instead of boosting harder, this solution focused on reducing variance by combining **150 survival models**, validated with **event-time-aware** repeated stratified cross‑validation (≤24h / ≤48h / later / censored).\n\nCheck out the full notebook **[HERE](https://www.kaggle.com/code/simonagradinaru/wids-26-avoid-boosting-the-flames-top-2).**\n\n---\n#🔥 The core idea - don't boost the flames\nBoosting is powerful on large datasets, but on tiny data it often amplifies noise and fails to generalize.\n\n---\n# 🏗️ Thoughtful feature engineering \n**Early finding**: As I also pointed in the [discussions](https://www.kaggle.com/competitions/WiDSWorldWide_GlobalDathon26/discussion/679133), the curse of dimensionality is *very real* here. With such small data, feature engineering isn't optional - it *is* the model. \n\n**Guiding principle**: encode the physics, nonlinearities and some of the interactions; keep those with strong signal, that also remain stable across models.\n\nCheck the models section for a thorough explanation on the features engineered for each model.\n\n---\n# ⚔️ The models - chosen to disagree\n\n## **1. Gradient Boosting Survival** (GBSA) \n- handles nonlinear effects and feature interactions\n- however, individual fits are noisy\n- Proposed solution: 8 diverse configs × 10 seeds = **80 models averaged** (as inspired by multiple public notebooks throughout the competition).\n\nThe configs deliberately spread to create *model diversity first*, then reduce variance by averaging.\n\n- learning rates from very gentle (0.005) to moderate (0.05),\n- small trees (max_depth = 2–3),\n- wide range of boosting lengths (200 → 2500 estimators),\n- subsampling and min‑leaf sizes to decorrelate trees.\n\n**Guiding principle**: Each model is imperfect; the ensemble is (very) stable.\n\n### Features:\n\n- `log_distance`, `proximity_risk` (inverse distance transformation) → fire impact diminishes nonlinearly with distance\n- `hazard_potential` (size x distance) → big things nearby are dangerous\n- `alignment_abs` (directionality) → is the fire moving towards the asset? could matter more than raw speed\n- `dist_slope_ci_0_5h` (trend) → captures the change in the first 5 hours (meters/per hour)\n- `event_start_month` (seasonality) → incorporates any potential seasonal patters \n- `log1p_area_first` (relative fire size) → grows quickly at first as area increases\n\nA small set of features for trees on data this small led to fewer splitting paths, hence less overfitting.\n\n## **2. Cox Proportional Hazards**\n- linear, regularized, and intentionally biased in a different direction → added for genuine ensemble diversity.\n- coefficients were shrunk by changing alpha values from very weak (.001) to strong regularization (5.0)\n\n**Guiding principle**: Cox is often wrong, but wrong *differently*, which is exactly what the ensemble needs.\n\n### Features:\n\n- `alignment_abs`, `log1p_area_first`, `num_perimeters_0_5h`\n- `log_distance`, `area_to_dist_ratio`, `fire_radius_km` (√(area / π)) → guide nonlinear effects with explicit physics-informed transforms\n- `log_eta` (basically `log(distance / closing speed)`), `eta_effective` (calculated as `distance / (closing speed + radial growth)`) → time to arrival dynamics; fire risk is *time-based* and CoxPH naturally models hazard over time\n- `near_speed_rank`, `far_threat_rank` (rank features) → ranks are robust to outliers \n- `is_summer`, `is_afternoon` (12-17), `zone_near` (within 5km), `zone_far` (over 5km), `has_movement` (reached more than 1 perimeter) → binary context flags to guide the model to interpretable findings\n- `threat_score` (alignment × speed / log-distance) → composite risk that encodes direction, speed and distance (thus an interaction)\n\n**Why more features?** Regularization shrinks weak signals toward zero. \n\n### Final blend:\n**90% GBSA + 10% CoxPH** won consistently in cross-validation tests.\n\n---\n# 🔁 Cross-validation \n\nRandom folds can land badly - too few short-time events in one fold, too many censored cases in another. \n\nCross-validation was done by defining an event-time strata (censored / hit by 24h / hit by 48h / hit later) and use `RepeatedStratifiedKFold` with **3 repeats × 5 folds**. \n\n```\nModels: GBSA (8 configs × 10 seeds) + CoxPH (7 alphas × 10 seeds)\nBlend: 90% GBSA + 10% CoxPH, 72h=1.0\nCV: RepeatedStratifiedKFold(5, 3), event+time strata\n c_index : 0.94714 ± 0.01604 (range: 0.92016 - 0.97872)\n brier_12 : 0.05391 ± 0.01179 (range: 0.03554 - 0.07416)\n brier_24 : 0.02572 ± 0.00882 (range: 0.01377 - 0.03897)\n brier_48 : 0.01310 ± 0.00639 (range: 0.00251 - 0.02359)\n brier_72 : 0.00000 ± 0.00000 (range: 0.00000 - 0.00000)\n weighted_brier : 0.01296 ± 0.00243 (range: 0.00893 - 0.01725)\n hybrid : 0.97507 ± 0.00519 (range: 0.96823 - 0.98610)\n```\n---\n# 👉 What actually mattered \n- **Aggressive feature trimming**: only 7 stable features for trees to reduce any potential overfitting\n- **Different feature space** for CoxPH (16), to ensure it consistently improves CV \n- **Event-time stratified CV** to avoid fold luck\n- Predicting `prob_72h = 1.0`, which quietly wins the weighted Brier\n\n---\n# 💡The lessons\n\n- Trees need few, strong, interaction-ready features \n- Linear models need pre-shaped nonlinear signals \n- Small data needs discipline, not abundance\n- More features ≠ better model. On small data, a small stable set is gold. 🥇 \n\nFull implementation, details and approach are in the notebook **[HERE](https://www.kaggle.com/code/simonagradinaru/wids-26-avoid-boosting-the-flames-top-2)**. ", "votes": 1 } ], "complete": true, "source": "live", "fetched_at": "2026-10-20T08:48:13Z", "competition_id": 125681, "competition_slug": "WiDSWorldWide_GlobalDathon26", "team_id": 15162381, "solution_url": "/competitions/WiDSWorldWide_GlobalDathon26/writeups/avoid-boosting-the-flames" }