# 27th | Avoid Boosting the Flames🔥 🙏 Thanks to the WiDS organizers for an excellent and thought‑provoking competition - small data, real constraints, and a genuinely unique and interesting survival setup. 📌 TL;DR This competition looked simple on paper — survival prediction at multiple horizons — but the real challenge was the data size. With only 221 training rows, most boosted models overfit before they learn anything. Instead of boosting harder, this solution focused on reducing variance by combining 150 survival models , validated with event-time-aware repeated stratified cross‑validation (≤24h / ≤48h / later / censored). Check out the full notebook [HERE](https://www.kaggle.com/code/simonagradinaru/wids-26-avoid-boosting-the-flames-top-2) . 🔥 The core idea - don't boost the flames Boosting is powerful on large datasets, but on tiny data it often amplifies noise and fails to generalize. 🏗️ Thoughtful feature engineering Early finding : As I also pointed in the [discussions](https://www.kaggle.com/competitions/WiDSWorldWide_GlobalDathon26/discussion/679133) , the curse of dimensionality is very real here. With such small data, feature engineering isn't optional - it is the model. Guiding principle : encode the physics, nonlinearities and some of the interactions; keep those with strong signal, that also remain stable across models. Check the models section for a thorough explanation on the features engineered for each model. ⚔️ The models - chosen to disagree 1. Gradient Boosting Survival (GBSA) handles nonlinear effects and feature interactions however, individual fits are noisy Proposed solution: 8 diverse configs × 10 seeds = 80 models averaged (as inspired by multiple public notebooks throughout the competition). The configs deliberately spread to create model diversity first , then reduce variance by averaging. learning rates from very gentle (0.005) to moderate (0.05), small trees (max_depth = 2–3), wide range of boosting lengths (200 → 2500 estimators), subsampling and min‑leaf sizes to decorrelate trees. Guiding principle : Each model is imperfect; the ensemble is (very) stable. Features: log_distance , proximity_risk (inverse distance transformation) → fire impact diminishes nonlinearly with distance hazard_potential (size x distance) → big things nearby are dangerous alignment_abs (directionality) → is the fire moving towards the asset? could matter more than raw speed dist_slope_ci_0_5h (trend) → captures the change in the first 5 hours (meters/per hour) event_start_month (seasonality) → incorporates any potential seasonal patters log1p_area_first (relative fire size) → grows quickly at first as area increases A small set of features for trees on data this small led to fewer splitting paths, hence less overfitting. 2. Cox Proportional Hazards linear, regularized, and intentionally biased in a different direction → added for genuine ensemble diversity. coefficients were shrunk by changing alpha values from very weak (.001) to strong regularization (5.0) Guiding principle : Cox is often wrong, but wrong differently , which is exactly what the ensemble needs. Features: alignment_abs , log1p_area_first , num_perimeters_0_5h log_distance , area_to_dist_ratio , fire_radius_km (√(area / π)) → guide nonlinear effects with explicit physics-informed transforms log_eta (basically log(distance / closing speed) ), eta_effective (calculated as distance / (closing speed + radial growth) ) → time to arrival dynamics; fire risk is time-based and CoxPH naturally models hazard over time near_speed_rank , far_threat_rank (rank features) → ranks are robust to outliers is_summer , is_afternoon (12-17), zone_near (within 5km), zone_far (over 5km), has_movement (reached more than 1 perimeter) → binary context flags to guide the model to interpretable findings threat_score (alignment × speed / log-distance) → composite risk that encodes direction, speed and distance (thus an interaction) Why more features? Regularization shrinks weak signals toward zero. Final blend: 90% GBSA + 10% CoxPH won consistently in cross-validation tests. 🔁 Cross-validation Random folds can land badly - too few short-time events in one fold, too many censored cases in another. Cross-validation was done by defining an event-time strata (censored / hit by 24h / hit by 48h / hit later) and use RepeatedStratifiedKFold with 3 repeats × 5 folds . Models: GBSA (8 configs × 10 seeds) + CoxPH (7 alphas × 10 seeds) Blend: 90% GBSA + 10% CoxPH, 72h=1.0 CV: RepeatedStratifiedKFold(5, 3), event+time strata c_index : 0.94714 ± 0.01604 (range: 0.92016 - 0.97872) brier_12 : 0.05391 ± 0.01179 (range: 0.03554 - 0.07416) brier_24 : 0.02572 ± 0.00882 (range: 0.01377 - 0.03897) brier_48 : 0.01310 ± 0.00639 (range: 0.00251 - 0.02359) brier_72 : 0.00000 ± 0.00000 (range: 0.00000 - 0.00000) weighted_brier : 0.01296 ± 0.00243 (range: 0.00893 - 0.01725) hybrid : 0.97507 ± 0.00519 (range: 0.96823 - 0.98610) 👉 What actually mattered Aggressive feature trimming : only 7 stable features for trees to reduce any potential overfitting Different feature space for CoxPH (16), to ensure it consistently improves CV Event-time stratified CV to avoid fold luck Predicting prob_72h = 1.0 , which quietly wins the weighted Brier 💡The lessons Trees need few, strong, interaction-ready features Linear models need pre-shaped nonlinear signals Small data needs discipline, not abundance More features ≠ better model. On small data, a small stable set is gold. 🥇 Full implementation, details and approach are in the notebook [HERE](https://www.kaggle.com/code/simonagradinaru/wids-26-avoid-boosting-the-flames-top-2) .