Our winning approach combined a LightGBM + CatBoost ensemble with domain-driven feature engineering tailored to the physics of Distributed Energy Resource (DER) systems. The core ideas were:
\nPrivate leaderboard F2: 0.91190.
\nThe dataset contains ~2.36M training rows and ~1.01M test rows with 725 columns of cyber-physical features drawn from SunSpec Modbus register models of a DER simulator.
\nKey column families include:
\ncommon[0].* — device metadata (serial number, model)DERMeasureAC[0].* — AC-side electrical measurements (W, VA, Var, PF, Hz, voltages, currents)DERMeasureDC[0].* — DC-side measurementsDERCapacity[0].* — operational and rated capacity limits (WMax, VAMax, etc.)DERCtlAC[0].* — control setpointsDEREnterService[0].* — enter-service voltage/frequency limitsDERVoltVar[0].* — Volt-Var curve parametersThe target Label is binary (0 = normal, 1 = anomaly) and the evaluation metric is F2-score, which weights recall 4× higher than precision — reflecting the real-world priority of catching attacks over avoiding false alarms.
Two physically distinct DER devices are present in the data, identifiable via common[0].SN:
SN-Three-Phase): ~1.5M rows, strong anomaly signal1100058974): ~820K rows, much weaker signal with very different measurement scales (e.g., W std ~2,500 vs ~24,000)This scale difference motivated per-device normalization features.
\nBefore loading the full dataset, we profiled every column on a 50K-row sample:
\nskip if: null_frac == 1.0 # entirely empty\n nunique <= 1 # constant\n null_frac > 0.95 # nearly empty\nThis reduced 725 → ~260 usable columns, cutting I/O and memory roughly in half.
\nAll object/string columns were label-encoded. Unseen test categories were mapped to a synthetic __MISSING__ class to prevent encoding failures.
We hashed column values on a random 50K-row subsample and dropped any column whose hash collided with an earlier column — removing exact duplicates cheaply.
\nEvery float64 was cast to float32 and int64 to the smallest integer dtype that could hold the data. This kept the full ~3.4M-row combined dataset in memory on a single machine.
We created ~20 lean, domain-grounded features on top of the ~260 surviving raw columns:
\n| Feature Group | \nFeatures Created | \nRationale | \n
|---|---|---|
| Null count | \nnull_count — row-level missing-value count | \nMissing data is itself an attack signature | \n
| AC aggregations | \nac_mean, ac_std, ac_min, ac_max, ac_range, ac_skew | \nSummarize the AC electrical state across all DERMeasureAC[0].* columns | \n
| DC aggregations | \ndc_mean, dc_std | \nDC-side summary | \n
| Power ratios | \nW_VA_ratio, W_Var_ratio | \nDetect power-factor manipulation | \n
| Capacity utilization | \nW_utilization, W_utilization_rtg, VA_utilization, Var_util_inj, Var_util_abs | \nRatio of measured value to operational/rated capacity — flags out-of-bounds operation | \n
| Capacity override detection | \nW_cap_ratio, VA_cap_ratio, VarInj_cap_ratio, VarAbs_cap_ratio + their diffs | \nOperational-vs-rated capacity ratios flag tampered capacity settings | \n
| Frequency deviation | \nhz_dev_60, hz_dev_60_abs | \nUS grid runs at 60 Hz — deviations indicate anomalies | \n
| Control aggregations | \nctl_mean, ctl_std | \nSummarize DERCtlAC[0].* control commands | \n
| Per-device z-scores | \nZ-score of W, VA, Var, PF, Hz, WMax, VAMax grouped by device SN | \nThe two devices operate at completely different scales; z-scoring removes this confound and lets the model compare within-device deviations | \n
The per-device z-score features were a key addition. Without them, a model trained globally would learn thresholds dominated by Device A's larger magnitudes and struggle to detect anomalies in Device B's smaller-scale signals.
\nAfter engineering, we had ~280 total features. Many were noise. We ran a quick LightGBM importance scan to prune them:
\nThis step improved generalization noticeably — removing low-importance features reduced overfitting and sped up the main training loop.
\nWe trained LightGBM with 2 random seeds (42, 2024), each running 5-fold stratified CV for a total of 10 models. Predictions were averaged across all seeds and folds.
\nHyperparameters:
\nobjective: binary\nboosting: gbdt\nlearning_rate: 0.03\nnum_leaves: 255\nmin_child_samples: 100\nfeature_fraction: 0.6\nbagging_fraction: 0.75\nbagging_freq: 5\nreg_alpha: 0.5\nreg_lambda: 3.0\nmin_gain_to_split: 0.02\npath_smooth: 1.0\nis_unbalance: True\nmax boost rounds: 10,000 (early stopping at 200)\nMulti-seed averaging smoothed out the decision boundary, which was particularly helpful near the classification threshold where small probability shifts flip predictions.
\nCatBoost was chosen for ensemble diversity — its ordered boosting and symmetric tree structure make it learn different decision boundaries from LightGBM on the same data.
\nHyperparameters:
\niterations: 5,000 (early stopping at 200)\nlearning_rate: 0.05\ndepth: 8\nl2_leaf_reg: 5.0\nrandom_strength: 1.0\nbagging_temperature: 0.8\nborder_count: 254\nauto_class_weights: Balanced\nCategorical features (integer columns with <20 unique values) were passed natively to CatBoost via cat_features, letting it use its ordered target-encoding rather than plain label encoding.
CatBoost was trained with the same 5-fold stratified CV as LightGBM.
\nWe optimized the LGB/CB blend weight and threshold jointly on the OOF predictions:
\nfor w in np.arange(0.40, 1.01, 0.05): # LGB weight\n for thresh in np.arange(0.05, 0.60, 0.005):\n oof_blend = w * oof_lgb + (1 - w) * oof_cb\n f2 = fbeta_score(y, (oof_blend > thresh).astype(int), beta=2)\nThe optimizer selected the best (w, threshold) pair that maximized OOF F2. A fallback check ensured the blend was only used if it strictly beat LGB-only — if not, pure LGB predictions were used instead.
Since F2 heavily penalizes missed anomalies, the probability-to-binary threshold was critical. We performed a fine-grained grid search over the final OOF predictions:
\nThe optimal threshold was applied to the blended test probabilities. We also generated alternate submissions at ±0.005 and ±0.01 from the optimal threshold to hedge against train/test distribution shift.
\nRaw Data (725 columns, ~3.4M rows)\n │\n ├── Column profiling on 50K sample\n ├── Drop useless columns (all-null, constant, >95% missing) → ~260 cols\n ├── Label-encode categoricals (handle unseen with __MISSING__)\n ├── Drop duplicate columns via hash comparison\n ├── Downcast dtypes (float64→32, int64→smallest)\n │\n ├── Feature Engineering (~20 new features)\n │ ├── Null count, AC/DC aggregations\n │ ├── Power ratios, capacity utilization\n │ ├── Capacity override ratios + diffs\n │ ├── Frequency deviation\n │ ├── Control aggregations\n │ └── Per-device z-score normalization\n │\n ├── Feature Selection (quick LGB → top 120 by gain)\n │\n ├── LightGBM × 2 seeds × 5 folds → averaged OOF & test probs\n ├── CatBoost × 5 folds → OOF & test probs\n │\n ├── Blend Optimization (LGB weight, threshold) on OOF\n │ └── Fallback: use LGB-only if blend doesn't improve F2\n │\n └── Apply threshold → binary submission\n| Technique | \nImpact | \n
|---|---|
| Per-device z-scores | \nNormalized the vastly different scales between the two DER devices, letting the model learn device-agnostic anomaly patterns | \n
| Capacity utilization features | \nW/WMax and similar were consistently top-importance — directly capture out-of-bounds DER operation | \n
| Feature selection (top 120) | \nRemoved noise features that hurt generalization | \n
| CatBoost blending | \nOrdered boosting captures patterns LGB misses; blend outperformed either model alone | \n
| Multi-seed averaging | \nSmoothed decision boundary, reduced variance | \n
| F2 threshold optimization | \nF2's recall emphasis means the optimal threshold is much lower than 0.5 — fine-grained search was essential | \n
| Approach | \nOutcome | \n
|---|---|
| XGBoost in ensemble (v2) | \nHurt LB score — too similar to LGB, added no diversity | \n
| Massive feature engineering (v2, ~60 features) | \nOverfit; lean feature set (v3/v4) generalized better | \n
| Single global model without device normalization | \nGood on Device A, struggled on Device B | \n
| More than 120 selected features | \nMarginal gains in CV, slight LB degradation | \n
Profile before you load. With 725 columns, blindly loading everything wastes memory and adds noise. A 50K-row profile pass let us cut the feature space in half before training.
Domain features beat volume. Twenty carefully designed features (capacity utilization, frequency deviation, per-device z-scores) outperformed 60+ kitchen-sink features from earlier iterations.
Ensemble diversity matters more than ensemble size. LGB + CatBoost (different boosting algorithms) was more effective than LGB + XGB (too similar).
Per-device normalization is critical in multi-population data. The two DER devices had wildly different measurement scales. Z-scoring within device let the model focus on relative anomalies rather than absolute magnitudes.
F2 threshold tuning is not optional. The default 0.5 threshold is far from optimal when recall is weighted 4× over precision. Fine-grained search over OOF predictions found the sweet spot.
Environment: Python 3.x, LightGBM, CatBoost, scikit-learn, pandas, NumPy
\nRuntime: Single machine, CPU only. Full pipeline completes in ~30 minutes on a modern laptop. My computer is equipped with a Ryzen 9 9950x 16 cores, so i trained quite fast locally.
\nCode: Attachted the Kaggle Notebook :)
\nPrivate LB F2: 0.91190
", "comments": [ { "id": 3436535, "author_name": "Akash V P", "author_url": "", "post_date": "2026-04-06T06:24:06.747000", "content": "Great solution, thanks for sharing!
\nI also participated but couldn’t score that well. Learned a lot from your approach. Congrats on 3rd place!
", "votes": 0, "replies": [ { "id": 3436542, "author_name": "Don", "author_url": "", "post_date": "2026-04-06T06:49:48.023000", "content": "Thank you very much! I updated the writeup and added the code as well.
", "votes": 0, "replies": [] } ] } ], "comment_count": 2, "messages": [ { "id": 3436535, "postDate": "2026-04-06T06:24:06.747Z", "content": "Great solution, thanks for sharing!
\nI also participated but couldn’t score that well. Learned a lot from your approach. Congrats on 3rd place!
", "rawMarkdown": "Great solution, thanks for sharing!\n\nI also participated but couldn’t score that well. Learned a lot from your approach. Congrats on 3rd place!\n", "replies": [ { "id": 3436542, "postDate": "2026-04-06T06:49:48.023Z", "content": "Thank you very much! I updated the writeup and added the code as well.
", "rawMarkdown": "Thank you very much! I updated the writeup and added the code as well. " } ] }, { "id": 3436387, "postDate": "2026-04-05T21:10:20.740Z", "content": "Our winning approach combined a LightGBM + CatBoost ensemble with domain-driven feature engineering tailored to the physics of Distributed Energy Resource (DER) systems. The core ideas were:
\nPrivate leaderboard F2: 0.91190.
\nThe dataset contains ~2.36M training rows and ~1.01M test rows with 725 columns of cyber-physical features drawn from SunSpec Modbus register models of a DER simulator.
\nKey column families include:
\ncommon[0].* — device metadata (serial number, model)DERMeasureAC[0].* — AC-side electrical measurements (W, VA, Var, PF, Hz, voltages, currents)DERMeasureDC[0].* — DC-side measurementsDERCapacity[0].* — operational and rated capacity limits (WMax, VAMax, etc.)DERCtlAC[0].* — control setpointsDEREnterService[0].* — enter-service voltage/frequency limitsDERVoltVar[0].* — Volt-Var curve parametersThe target Label is binary (0 = normal, 1 = anomaly) and the evaluation metric is F2-score, which weights recall 4× higher than precision — reflecting the real-world priority of catching attacks over avoiding false alarms.
Two physically distinct DER devices are present in the data, identifiable via common[0].SN:
SN-Three-Phase): ~1.5M rows, strong anomaly signal1100058974): ~820K rows, much weaker signal with very different measurement scales (e.g., W std ~2,500 vs ~24,000)This scale difference motivated per-device normalization features.
\nBefore loading the full dataset, we profiled every column on a 50K-row sample:
\nskip if: null_frac == 1.0 # entirely empty\n nunique <= 1 # constant\n null_frac > 0.95 # nearly empty\nThis reduced 725 → ~260 usable columns, cutting I/O and memory roughly in half.
\nAll object/string columns were label-encoded. Unseen test categories were mapped to a synthetic __MISSING__ class to prevent encoding failures.
We hashed column values on a random 50K-row subsample and dropped any column whose hash collided with an earlier column — removing exact duplicates cheaply.
\nEvery float64 was cast to float32 and int64 to the smallest integer dtype that could hold the data. This kept the full ~3.4M-row combined dataset in memory on a single machine.
We created ~20 lean, domain-grounded features on top of the ~260 surviving raw columns:
\n| Feature Group | \nFeatures Created | \nRationale | \n
|---|---|---|
| Null count | \nnull_count — row-level missing-value count | \nMissing data is itself an attack signature | \n
| AC aggregations | \nac_mean, ac_std, ac_min, ac_max, ac_range, ac_skew | \nSummarize the AC electrical state across all DERMeasureAC[0].* columns | \n
| DC aggregations | \ndc_mean, dc_std | \nDC-side summary | \n
| Power ratios | \nW_VA_ratio, W_Var_ratio | \nDetect power-factor manipulation | \n
| Capacity utilization | \nW_utilization, W_utilization_rtg, VA_utilization, Var_util_inj, Var_util_abs | \nRatio of measured value to operational/rated capacity — flags out-of-bounds operation | \n
| Capacity override detection | \nW_cap_ratio, VA_cap_ratio, VarInj_cap_ratio, VarAbs_cap_ratio + their diffs | \nOperational-vs-rated capacity ratios flag tampered capacity settings | \n
| Frequency deviation | \nhz_dev_60, hz_dev_60_abs | \nUS grid runs at 60 Hz — deviations indicate anomalies | \n
| Control aggregations | \nctl_mean, ctl_std | \nSummarize DERCtlAC[0].* control commands | \n
| Per-device z-scores | \nZ-score of W, VA, Var, PF, Hz, WMax, VAMax grouped by device SN | \nThe two devices operate at completely different scales; z-scoring removes this confound and lets the model compare within-device deviations | \n
The per-device z-score features were a key addition. Without them, a model trained globally would learn thresholds dominated by Device A's larger magnitudes and struggle to detect anomalies in Device B's smaller-scale signals.
\nAfter engineering, we had ~280 total features. Many were noise. We ran a quick LightGBM importance scan to prune them:
\nThis step improved generalization noticeably — removing low-importance features reduced overfitting and sped up the main training loop.
\nWe trained LightGBM with 2 random seeds (42, 2024), each running 5-fold stratified CV for a total of 10 models. Predictions were averaged across all seeds and folds.
\nHyperparameters:
\nobjective: binary\nboosting: gbdt\nlearning_rate: 0.03\nnum_leaves: 255\nmin_child_samples: 100\nfeature_fraction: 0.6\nbagging_fraction: 0.75\nbagging_freq: 5\nreg_alpha: 0.5\nreg_lambda: 3.0\nmin_gain_to_split: 0.02\npath_smooth: 1.0\nis_unbalance: True\nmax boost rounds: 10,000 (early stopping at 200)\nMulti-seed averaging smoothed out the decision boundary, which was particularly helpful near the classification threshold where small probability shifts flip predictions.
\nCatBoost was chosen for ensemble diversity — its ordered boosting and symmetric tree structure make it learn different decision boundaries from LightGBM on the same data.
\nHyperparameters:
\niterations: 5,000 (early stopping at 200)\nlearning_rate: 0.05\ndepth: 8\nl2_leaf_reg: 5.0\nrandom_strength: 1.0\nbagging_temperature: 0.8\nborder_count: 254\nauto_class_weights: Balanced\nCategorical features (integer columns with <20 unique values) were passed natively to CatBoost via cat_features, letting it use its ordered target-encoding rather than plain label encoding.
CatBoost was trained with the same 5-fold stratified CV as LightGBM.
\nWe optimized the LGB/CB blend weight and threshold jointly on the OOF predictions:
\nfor w in np.arange(0.40, 1.01, 0.05): # LGB weight\n for thresh in np.arange(0.05, 0.60, 0.005):\n oof_blend = w * oof_lgb + (1 - w) * oof_cb\n f2 = fbeta_score(y, (oof_blend > thresh).astype(int), beta=2)\nThe optimizer selected the best (w, threshold) pair that maximized OOF F2. A fallback check ensured the blend was only used if it strictly beat LGB-only — if not, pure LGB predictions were used instead.
Since F2 heavily penalizes missed anomalies, the probability-to-binary threshold was critical. We performed a fine-grained grid search over the final OOF predictions:
\nThe optimal threshold was applied to the blended test probabilities. We also generated alternate submissions at ±0.005 and ±0.01 from the optimal threshold to hedge against train/test distribution shift.
\nRaw Data (725 columns, ~3.4M rows)\n │\n ├── Column profiling on 50K sample\n ├── Drop useless columns (all-null, constant, >95% missing) → ~260 cols\n ├── Label-encode categoricals (handle unseen with __MISSING__)\n ├── Drop duplicate columns via hash comparison\n ├── Downcast dtypes (float64→32, int64→smallest)\n │\n ├── Feature Engineering (~20 new features)\n │ ├── Null count, AC/DC aggregations\n │ ├── Power ratios, capacity utilization\n │ ├── Capacity override ratios + diffs\n │ ├── Frequency deviation\n │ ├── Control aggregations\n │ └── Per-device z-score normalization\n │\n ├── Feature Selection (quick LGB → top 120 by gain)\n │\n ├── LightGBM × 2 seeds × 5 folds → averaged OOF & test probs\n ├── CatBoost × 5 folds → OOF & test probs\n │\n ├── Blend Optimization (LGB weight, threshold) on OOF\n │ └── Fallback: use LGB-only if blend doesn't improve F2\n │\n └── Apply threshold → binary submission\n| Technique | \nImpact | \n
|---|---|
| Per-device z-scores | \nNormalized the vastly different scales between the two DER devices, letting the model learn device-agnostic anomaly patterns | \n
| Capacity utilization features | \nW/WMax and similar were consistently top-importance — directly capture out-of-bounds DER operation | \n
| Feature selection (top 120) | \nRemoved noise features that hurt generalization | \n
| CatBoost blending | \nOrdered boosting captures patterns LGB misses; blend outperformed either model alone | \n
| Multi-seed averaging | \nSmoothed decision boundary, reduced variance | \n
| F2 threshold optimization | \nF2's recall emphasis means the optimal threshold is much lower than 0.5 — fine-grained search was essential | \n
| Approach | \nOutcome | \n
|---|---|
| XGBoost in ensemble (v2) | \nHurt LB score — too similar to LGB, added no diversity | \n
| Massive feature engineering (v2, ~60 features) | \nOverfit; lean feature set (v3/v4) generalized better | \n
| Single global model without device normalization | \nGood on Device A, struggled on Device B | \n
| More than 120 selected features | \nMarginal gains in CV, slight LB degradation | \n
Profile before you load. With 725 columns, blindly loading everything wastes memory and adds noise. A 50K-row profile pass let us cut the feature space in half before training.
Domain features beat volume. Twenty carefully designed features (capacity utilization, frequency deviation, per-device z-scores) outperformed 60+ kitchen-sink features from earlier iterations.
Ensemble diversity matters more than ensemble size. LGB + CatBoost (different boosting algorithms) was more effective than LGB + XGB (too similar).
Per-device normalization is critical in multi-population data. The two DER devices had wildly different measurement scales. Z-scoring within device let the model focus on relative anomalies rather than absolute magnitudes.
F2 threshold tuning is not optional. The default 0.5 threshold is far from optimal when recall is weighted 4× over precision. Fine-grained search over OOF predictions found the sweet spot.
Environment: Python 3.x, LightGBM, CatBoost, scikit-learn, pandas, NumPy
\nRuntime: Single machine, CPU only. Full pipeline completes in ~30 minutes on a modern laptop. My computer is equipped with a Ryzen 9 9950x 16 cores, so i trained quite fast locally.
\nCode: Attachted the Kaggle Notebook :)
\nPrivate LB F2: 0.91190
", "rawMarkdown": "# Cyber-Physical Anomaly Detection for DER Systems — Winning Solution (Private LB: 0.91190 Public: 0.91149)\n\n## Summary\n\nOur winning approach combined a **LightGBM + CatBoost ensemble** with domain-driven feature engineering tailored to the physics of Distributed Energy Resource (DER) systems. The core ideas were:\n\n1. **Aggressive column pruning** — profiling all 725 columns on a sample and discarding noise before training\n2. **Domain-specific feature engineering** — capacity utilization ratios, power factor anomalies, frequency deviations, control aggregations, and **per-device z-score normalization** to account for fundamentally different device scales\n3. **Gain-based feature selection** — reducing to the top 120 most informative features\n4. **Multi-seed LightGBM** (2 seeds × 5 folds) for stable probability estimates\n5. **CatBoost** as a secondary learner with ordered boosting for ensemble diversity\n6. **Optimized LGB/CB blending** with fine-grained F2 threshold tuning\n\nPrivate leaderboard F2: **0.91190**.\n\n---\n\n## 1. Data Understanding\n\nThe dataset contains **~2.36M training rows** and **~1.01M test rows** with **725 columns** of cyber-physical features drawn from SunSpec Modbus register models of a DER simulator.\n\nKey column families include:\n- `common[0].*` — device metadata (serial number, model)\n- `DERMeasureAC[0].*` — AC-side electrical measurements (W, VA, Var, PF, Hz, voltages, currents)\n- `DERMeasureDC[0].*` — DC-side measurements\n- `DERCapacity[0].*` — operational and rated capacity limits (WMax, VAMax, etc.)\n- `DERCtlAC[0].*` — control setpoints\n- `DEREnterService[0].*` — enter-service voltage/frequency limits\n- `DERVoltVar[0].*` — Volt-Var curve parameters\n\nThe target `Label` is binary (0 = normal, 1 = anomaly) and the evaluation metric is **F2-score**, which weights recall 4× higher than precision — reflecting the real-world priority of catching attacks over avoiding false alarms.\n\nTwo physically distinct DER devices are present in the data, identifiable via `common[0].SN`:\n- **Device A** (`SN-Three-Phase`): ~1.5M rows, strong anomaly signal\n- **Device B** (`1100058974`): ~820K rows, much weaker signal with very different measurement scales (e.g., W std ~2,500 vs ~24,000)\n\nThis scale difference motivated per-device normalization features.\n\n---\n\n## 2. Preprocessing\n\n### 2.1 Column Profiling & Filtering\n\nBefore loading the full dataset, we profiled every column on a 50K-row sample:\n\n```python\nskip if: null_frac == 1.0 # entirely empty\n nunique <= 1 # constant\n null_frac > 0.95 # nearly empty\n```\n\nThis reduced **725 → ~260 usable columns**, cutting I/O and memory roughly in half.\n\n### 2.2 Label Encoding\n\nAll object/string columns were label-encoded. Unseen test categories were mapped to a synthetic `__MISSING__` class to prevent encoding failures.\n\n### 2.3 Duplicate Column Removal\n\nWe hashed column values on a random 50K-row subsample and dropped any column whose hash collided with an earlier column — removing exact duplicates cheaply.\n\n### 2.4 Memory Downcasting\n\nEvery `float64` was cast to `float32` and `int64` to the smallest integer dtype that could hold the data. This kept the full ~3.4M-row combined dataset in memory on a single machine.\n\n---\n\n## 3. Feature Engineering\n\nWe created **~20 lean, domain-grounded features** on top of the ~260 surviving raw columns:\n\n| Feature Group | Features Created | Rationale |\n|--------------|-----------------|-----------|\n| **Null count** | `null_count` — row-level missing-value count | Missing data is itself an attack signature |\n| **AC aggregations** | `ac_mean`, `ac_std`, `ac_min`, `ac_max`, `ac_range`, `ac_skew` | Summarize the AC electrical state across all `DERMeasureAC[0].*` columns |\n| **DC aggregations** | `dc_mean`, `dc_std` | DC-side summary |\n| **Power ratios** | `W_VA_ratio`, `W_Var_ratio` | Detect power-factor manipulation |\n| **Capacity utilization** | `W_utilization`, `W_utilization_rtg`, `VA_utilization`, `Var_util_inj`, `Var_util_abs` | Ratio of measured value to operational/rated capacity — flags out-of-bounds operation |\n| **Capacity override detection** | `W_cap_ratio`, `VA_cap_ratio`, `VarInj_cap_ratio`, `VarAbs_cap_ratio` + their diffs | Operational-vs-rated capacity ratios flag tampered capacity settings |\n| **Frequency deviation** | `hz_dev_60`, `hz_dev_60_abs` | US grid runs at 60 Hz — deviations indicate anomalies |\n| **Control aggregations** | `ctl_mean`, `ctl_std` | Summarize `DERCtlAC[0].*` control commands |\n| **Per-device z-scores** | Z-score of W, VA, Var, PF, Hz, WMax, VAMax grouped by device SN | The two devices operate at completely different scales; z-scoring removes this confound and lets the model compare within-device deviations |\n\nThe per-device z-score features were a key addition. Without them, a model trained globally would learn thresholds dominated by Device A's larger magnitudes and struggle to detect anomalies in Device B's smaller-scale signals.\n\n---\n\n## 4. Feature Selection\n\nAfter engineering, we had ~280 total features. Many were noise. We ran a **quick LightGBM importance scan** to prune them:\n\n1. Train a fast model (500 rounds, LR=0.1) on an 80/20 split\n2. Rank all features by gain importance\n3. Keep only the **top 120 features** with importance > 0\n\nThis step improved generalization noticeably — removing low-importance features reduced overfitting and sped up the main training loop.\n\n---\n\n## 5. Modeling\n\n### 5.1 LightGBM (Multi-Seed)\n\nWe trained LightGBM with **2 random seeds** (42, 2024), each running **5-fold stratified CV** for a total of 10 models. Predictions were averaged across all seeds and folds.\n\n**Hyperparameters:**\n\n```\nobjective: binary\nboosting: gbdt\nlearning_rate: 0.03\nnum_leaves: 255\nmin_child_samples: 100\nfeature_fraction: 0.6\nbagging_fraction: 0.75\nbagging_freq: 5\nreg_alpha: 0.5\nreg_lambda: 3.0\nmin_gain_to_split: 0.02\npath_smooth: 1.0\nis_unbalance: True\nmax boost rounds: 10,000 (early stopping at 200)\n```\n\nMulti-seed averaging smoothed out the decision boundary, which was particularly helpful near the classification threshold where small probability shifts flip predictions.\n\n### 5.2 CatBoost\n\nCatBoost was chosen for **ensemble diversity** — its ordered boosting and symmetric tree structure make it learn different decision boundaries from LightGBM on the same data.\n\n**Hyperparameters:**\n\n```\niterations: 5,000 (early stopping at 200)\nlearning_rate: 0.05\ndepth: 8\nl2_leaf_reg: 5.0\nrandom_strength: 1.0\nbagging_temperature: 0.8\nborder_count: 254\nauto_class_weights: Balanced\n```\n\nCategorical features (integer columns with <20 unique values) were passed natively to CatBoost via `cat_features`, letting it use its ordered target-encoding rather than plain label encoding.\n\nCatBoost was trained with the same 5-fold stratified CV as LightGBM.\n\n### 5.3 Ensemble Blending\n\nWe optimized the LGB/CB blend weight and threshold jointly on the OOF predictions:\n\n```python\nfor w in np.arange(0.40, 1.01, 0.05): # LGB weight\n for thresh in np.arange(0.05, 0.60, 0.005):\n oof_blend = w * oof_lgb + (1 - w) * oof_cb\n f2 = fbeta_score(y, (oof_blend > thresh).astype(int), beta=2)\n```\n\nThe optimizer selected the best `(w, threshold)` pair that maximized OOF F2. A fallback check ensured the blend was only used if it strictly beat LGB-only — if not, pure LGB predictions were used instead.\n\n---\n\n## 6. Threshold Optimization\n\nSince F2 heavily penalizes missed anomalies, the probability-to-binary threshold was critical. We performed a fine-grained grid search over the final OOF predictions:\n\n- **Range:** 0.05 to 0.60\n- **Step size:** 0.001\n\nThe optimal threshold was applied to the blended test probabilities. We also generated **alternate submissions** at ±0.005 and ±0.01 from the optimal threshold to hedge against train/test distribution shift.\n\n---\n\n## 7. Full Pipeline\n\n```\nRaw Data (725 columns, ~3.4M rows)\n │\n ├── Column profiling on 50K sample\n ├── Drop useless columns (all-null, constant, >95% missing) → ~260 cols\n ├── Label-encode categoricals (handle unseen with __MISSING__)\n ├── Drop duplicate columns via hash comparison\n ├── Downcast dtypes (float64→32, int64→smallest)\n │\n ├── Feature Engineering (~20 new features)\n │ ├── Null count, AC/DC aggregations\n │ ├── Power ratios, capacity utilization\n │ ├── Capacity override ratios + diffs\n │ ├── Frequency deviation\n │ ├── Control aggregations\n │ └── Per-device z-score normalization\n │\n ├── Feature Selection (quick LGB → top 120 by gain)\n │\n ├── LightGBM × 2 seeds × 5 folds → averaged OOF & test probs\n ├── CatBoost × 5 folds → OOF & test probs\n │\n ├── Blend Optimization (LGB weight, threshold) on OOF\n │ └── Fallback: use LGB-only if blend doesn't improve F2\n │\n └── Apply threshold → binary submission\n```\n\n---\n\n## 8. What Worked\n\n| Technique | Impact |\n|-----------|--------|\n| **Per-device z-scores** | Normalized the vastly different scales between the two DER devices, letting the model learn device-agnostic anomaly patterns |\n| **Capacity utilization features** | W/WMax and similar were consistently top-importance — directly capture out-of-bounds DER operation |\n| **Feature selection (top 120)** | Removed noise features that hurt generalization |\n| **CatBoost blending** | Ordered boosting captures patterns LGB misses; blend outperformed either model alone |\n| **Multi-seed averaging** | Smoothed decision boundary, reduced variance |\n| **F2 threshold optimization** | F2's recall emphasis means the optimal threshold is much lower than 0.5 — fine-grained search was essential |\n\n## 9. What Didn't Work / We Tried\n\n| Approach | Outcome |\n|----------|---------|\n| XGBoost in ensemble (v2) | Hurt LB score — too similar to LGB, added no diversity |\n| Massive feature engineering (v2, ~60 features) | Overfit; lean feature set (v3/v4) generalized better |\n| Single global model without device normalization | Good on Device A, struggled on Device B |\n| More than 120 selected features | Marginal gains in CV, slight LB degradation |\n\n---\n\n## 10. Key Takeaways\n\n1. **Profile before you load.** With 725 columns, blindly loading everything wastes memory and adds noise. A 50K-row profile pass let us cut the feature space in half before training.\n\n2. **Domain features beat volume.** Twenty carefully designed features (capacity utilization, frequency deviation, per-device z-scores) outperformed 60+ kitchen-sink features from earlier iterations.\n\n3. **Ensemble diversity matters more than ensemble size.** LGB + CatBoost (different boosting algorithms) was more effective than LGB + XGB (too similar).\n\n4. **Per-device normalization is critical in multi-population data.** The two DER devices had wildly different measurement scales. Z-scoring within device let the model focus on *relative* anomalies rather than *absolute* magnitudes.\n\n5. **F2 threshold tuning is not optional.** The default 0.5 threshold is far from optimal when recall is weighted 4× over precision. Fine-grained search over OOF predictions found the sweet spot.\n\n---\n\n## Reproducibility\n\n**Environment:** Python 3.x, LightGBM, CatBoost, scikit-learn, pandas, NumPy\n\n**Runtime:** Single machine, CPU only. Full pipeline completes in ~30 minutes on a modern laptop. My computer is equipped with a Ryzen 9 9950x 16 cores, so i trained quite fast locally. \n\n**Code:** Attachted the Kaggle Notebook :)\n\n**Private LB F2: 0.91190**\n" } ], "complete": true, "source": "live", "fetched_at": "2026-10-22T12:48:25Z", "competition_id": 119083, "competition_slug": "cyber-physical-anomaly-detection-for-der-systems", "team_id": 15420476, "solution_url": "" }