{
  "id": 703229,
  "title": "8th place solution",
  "content": "<p>Thanks to the host for this interesting competition. We really learned a lot about survival prediction from this competition.</p>\n<h1>TL;DR</h1>\n<p>In the competition, we need to predict the probability of wildfires spreading to evacuation zones at various points in the future. Moreover, the dataset of this competition is very small, with only 221 samples. So, how to avoid overfitting will be a problem.</p>\n<p>My final solution inlcude ensembling Tabm with GBSA (Gradient Boosting Survival) and some physics feature engineering.</p>\n<h1>Feature engineering</h1>\n<p>Feature engineering mainly includes six aspects:</p>\n<ul>\n<li>Distance Transformations</li>\n<li>Geometric &amp; Spatial Ratios</li>\n<li>Kinematics &amp; ETA (Estimated Time of Arrival)</li>\n<li>Composite Threat &amp; Urgency Scores</li>\n<li>Temporal &amp; Environmental Context</li>\n<li>Stratified &amp; Zonal Logic</li>\n</ul>\n<p>To prevent overfitting, We did not explore deeper features.</p>\n<h1>Cross Validation</h1>\n<p>We use stratified cross validation with prob_12h to enable the model to learn some urgent situations.</p>\n<p>Final predictions are the average of every fold. </p>\n<h1>Models</h1>\n<p>We use two models in the final ensemble—Tabm and GBSA.</p>\n<h2>Gradient Boosting Survival (GBSA)</h2>\n<p>To capture the nonlinear effect between the features.</p>\n<pre><code>model_pra = {\n        \"random_state\": 8888,\n        \"learning_rate\": 0.02,\n        \"n_estimators\": 500,\n        \"max_depth\": 3,\n        \"loss\": \"coxph\",\n        \"min_samples_leaf\": 3,\n        \"min_samples_split\": 12,\n        \"subsample\": 0.85,\n        \"dropout_rate\": 0.0,\n}\n</code></pre>\n<h2>TabM</h2>\n<p>Neural network models can learn more continuous and smooth nonlinear relationships. Furthermore, the model can perform multi-target regression very conveniently.\nThe model parameters I use in the training :</p>\n<pre><code>my_config = {\n    'lr': 0.0007612359851151996,\n    'weight_decay': 0.0381,\n    'k': 28,\n    'num_embeddings': {\n                'type': 'PiecewiseLinearEmbeddings',\n                'd_embedding': 32,\n                'activation': True,\n                'version': 'A',\n            },\n    'backbone': {\n            'type': 'MLP',\n            'n_blocks': 3 ,\n            'd_block': [304, 304, 304],\n            'dropout': 0.35099603708131766,\n            'activation' : 'SiLU',\n        },\n    'batch_size': 16,\n    'arch_type': 'tabm',\n    'folds': 5,\n    'n_bins' : 121,\n    'n_classes' : 4,\n    'device' : 'cuda',\n    'epochs' : 80,\n}\n</code></pre>\n<h2>Monotonicity constraints</h2>\n<p>The outputs of all models were subject to monotonicity constraints: \nprob_12h &lt;= prob_24h &lt;= prob_48h &lt;= prob_72h</p>\n<h2>Optuna</h2>\n<p>We use optuna to find the suitable parameters.</p>\n<h2>Ensemble</h2>\n<p>Because of concerns that the neural network might overfit this small dataset, the final weights are 60% GBSA + 40% TabM.</p>\n<h1>What doesn't work</h1>\n<ul>\n<li>XGBoost or LightGBM. I try to train 4 xgb or lgb models with different  target, but it doesn't perform well.</li>\n<li>Create more features , such as time feature, crossing feature.</li>\n<li>Calibrating the probability with threshold.</li>\n</ul>\n<h1>Training code</h1>\n<p><a href=\"https://www.kaggle.com/code/i2nfinit3y/wids-tabm-training\" target=\"_blank\">TabM training code</a></p>\n<p><a href=\"https://www.kaggle.com/code/i2nfinit3y/wids-gbsa-training\" target=\"_blank\">GBSA training code</a></p>",
  "comments": [],
  "comment_count": 0,
  "messages": [
    {
      "id": 3464287,
      "postDate": "2026-05-29T12:10:39.717Z",
      "content": "<p>Thanks to the host for this interesting competition. We really learned a lot about survival prediction from this competition.</p>\n<h1>TL;DR</h1>\n<p>In the competition, we need to predict the probability of wildfires spreading to evacuation zones at various points in the future. Moreover, the dataset of this competition is very small, with only 221 samples. So, how to avoid overfitting will be a problem.</p>\n<p>My final solution inlcude ensembling Tabm with GBSA (Gradient Boosting Survival) and some physics feature engineering.</p>\n<h1>Feature engineering</h1>\n<p>Feature engineering mainly includes six aspects:</p>\n<ul>\n<li>Distance Transformations</li>\n<li>Geometric &amp; Spatial Ratios</li>\n<li>Kinematics &amp; ETA (Estimated Time of Arrival)</li>\n<li>Composite Threat &amp; Urgency Scores</li>\n<li>Temporal &amp; Environmental Context</li>\n<li>Stratified &amp; Zonal Logic</li>\n</ul>\n<p>To prevent overfitting, We did not explore deeper features.</p>\n<h1>Cross Validation</h1>\n<p>We use stratified cross validation with prob_12h to enable the model to learn some urgent situations.</p>\n<p>Final predictions are the average of every fold. </p>\n<h1>Models</h1>\n<p>We use two models in the final ensemble—Tabm and GBSA.</p>\n<h2>Gradient Boosting Survival (GBSA)</h2>\n<p>To capture the nonlinear effect between the features.</p>\n<pre><code>model_pra = {\n        \"random_state\": 8888,\n        \"learning_rate\": 0.02,\n        \"n_estimators\": 500,\n        \"max_depth\": 3,\n        \"loss\": \"coxph\",\n        \"min_samples_leaf\": 3,\n        \"min_samples_split\": 12,\n        \"subsample\": 0.85,\n        \"dropout_rate\": 0.0,\n}\n</code></pre>\n<h2>TabM</h2>\n<p>Neural network models can learn more continuous and smooth nonlinear relationships. Furthermore, the model can perform multi-target regression very conveniently.\nThe model parameters I use in the training :</p>\n<pre><code>my_config = {\n    'lr': 0.0007612359851151996,\n    'weight_decay': 0.0381,\n    'k': 28,\n    'num_embeddings': {\n                'type': 'PiecewiseLinearEmbeddings',\n                'd_embedding': 32,\n                'activation': True,\n                'version': 'A',\n            },\n    'backbone': {\n            'type': 'MLP',\n            'n_blocks': 3 ,\n            'd_block': [304, 304, 304],\n            'dropout': 0.35099603708131766,\n            'activation' : 'SiLU',\n        },\n    'batch_size': 16,\n    'arch_type': 'tabm',\n    'folds': 5,\n    'n_bins' : 121,\n    'n_classes' : 4,\n    'device' : 'cuda',\n    'epochs' : 80,\n}\n</code></pre>\n<h2>Monotonicity constraints</h2>\n<p>The outputs of all models were subject to monotonicity constraints: \nprob_12h &lt;= prob_24h &lt;= prob_48h &lt;= prob_72h</p>\n<h2>Optuna</h2>\n<p>We use optuna to find the suitable parameters.</p>\n<h2>Ensemble</h2>\n<p>Because of concerns that the neural network might overfit this small dataset, the final weights are 60% GBSA + 40% TabM.</p>\n<h1>What doesn't work</h1>\n<ul>\n<li>XGBoost or LightGBM. I try to train 4 xgb or lgb models with different  target, but it doesn't perform well.</li>\n<li>Create more features , such as time feature, crossing feature.</li>\n<li>Calibrating the probability with threshold.</li>\n</ul>\n<h1>Training code</h1>\n<p><a href=\"https://www.kaggle.com/code/i2nfinit3y/wids-tabm-training\" target=\"_blank\">TabM training code</a></p>\n<p><a href=\"https://www.kaggle.com/code/i2nfinit3y/wids-gbsa-training\" target=\"_blank\">GBSA training code</a></p>",
      "rawMarkdown": "Thanks to the host for this interesting competition. We really learned a lot about survival prediction from this competition.\n\n# TL;DR\n\nIn the competition, we need to predict the probability of wildfires spreading to evacuation zones at various points in the future. Moreover, the dataset of this competition is very small, with only 221 samples. So, how to avoid overfitting will be a problem.\n\nMy final solution inlcude ensembling Tabm with GBSA (Gradient Boosting Survival) and some physics feature engineering.\n\n# Feature engineering\n\nFeature engineering mainly includes six aspects:\n- Distance Transformations\n- Geometric & Spatial Ratios\n- Kinematics & ETA (Estimated Time of Arrival)\n- Composite Threat & Urgency Scores\n- Temporal & Environmental Context\n- Stratified & Zonal Logic\n\nTo prevent overfitting, We did not explore deeper features.\n\n# Cross Validation\n\nWe use stratified cross validation with prob_12h to enable the model to learn some urgent situations.\n\nFinal predictions are the average of every fold. \n\n# Models\nWe use two models in the final ensemble—Tabm and GBSA.\n## Gradient Boosting Survival (GBSA)\nTo capture the nonlinear effect between the features.\n```\nmodel_pra = {\n        \"random_state\": 8888,\n        \"learning_rate\": 0.02,\n        \"n_estimators\": 500,\n        \"max_depth\": 3,\n        \"loss\": \"coxph\",\n        \"min_samples_leaf\": 3,\n        \"min_samples_split\": 12,\n        \"subsample\": 0.85,\n        \"dropout_rate\": 0.0,\n}\n```\n## TabM\nNeural network models can learn more continuous and smooth nonlinear relationships. Furthermore, the model can perform multi-target regression very conveniently.\nThe model parameters I use in the training :\n```\nmy_config = {\n    'lr': 0.0007612359851151996,\n    'weight_decay': 0.0381,\n    'k': 28,\n    'num_embeddings': {\n                'type': 'PiecewiseLinearEmbeddings',\n                'd_embedding': 32,\n                'activation': True,\n                'version': 'A',\n            },\n    'backbone': {\n            'type': 'MLP',\n            'n_blocks': 3 ,\n            'd_block': [304, 304, 304],\n            'dropout': 0.35099603708131766,\n            'activation' : 'SiLU',\n        },\n    'batch_size': 16,\n    'arch_type': 'tabm',\n    'folds': 5,\n    'n_bins' : 121,\n    'n_classes' : 4,\n    'device' : 'cuda',\n    'epochs' : 80,\n}\n```\n\n## Monotonicity constraints\nThe outputs of all models were subject to monotonicity constraints: \nprob_12h <= prob_24h <= prob_48h <= prob_72h\n\n## Optuna\nWe use optuna to find the suitable parameters.\n\n## Ensemble\nBecause of concerns that the neural network might overfit this small dataset, the final weights are 60% GBSA + 40% TabM.\n\n# What doesn't work\n- XGBoost or LightGBM. I try to train 4 xgb or lgb models with different  target, but it doesn't perform well.\n- Create more features , such as time feature, crossing feature.\n- Calibrating the probability with threshold.\n\n# Training code\n[TabM training code](https://www.kaggle.com/code/i2nfinit3y/wids-tabm-training)\n\n[GBSA training code](https://www.kaggle.com/code/i2nfinit3y/wids-gbsa-training)",
      "votes": 1
    }
  ],
  "complete": true,
  "source": "live",
  "fetched_at": "2026-10-20T08:48:39Z",
  "competition_id": 125681,
  "competition_slug": "WiDSWorldWide_GlobalDathon26",
  "team_id": 15198870,
  "solution_url": "/competitions/WiDSWorldWide_GlobalDathon26/writeups/8th-place-solution"
}