{
  "id": 566563,
  "title": "100th Place Silver Medal Solution",
  "content": "<p>Hi Everyone, I just wanted to share my solution as well as some tricks I used to boost my score.</p>\n<h1>Overview</h1>\n<ul>\n<li>Kaplan–Meier Catboost with event-masking + PRL NN with event-masking</li>\n<li>CV: 0.690</li>\n<li>Public LB Score: 0.689</li>\n<li>Private LB Score: 0.693</li>\n</ul>\n<h1>Tricks</h1>\n<ul>\n<li>Cross Validation Methodology</li>\n</ul>\n<ol>\n<li>StratifiedKFold using both race group and binned target values. This ensures balanced representation of race groups and target distributions across folds.</li>\n<li>10 folds across 30 different seeds were used for cross validation to train a total of 300 models for each unique model variant. The idea was to make the model ensemble more diverse and robust.</li>\n</ol>\n<pre><code>train[] = pd.cut(train[], =NUM_BINS, =)\ntrain[] = train[].astype(str) +  + train[].astype(str)\n\n seed  SEEDS:\n    skf = StratifiedKFold(=FOLDS, =, =seed)\n</code></pre>\n<p>This method significantly boosted my CV score—the more seeds I added, the higher the score. However, I stopped at 30 seeds, as my testing showed that adding more than 30 no longer improved the CV score.</p>\n<p>Running such a meticulous CV also served another purpose: preventing me from overfitting to specific folds during testing. This methodology was both stable and reliable, giving me confidence in the feedback from my CV.</p>\n<ul>\n<li>Event-Masking Using Classifiers</li>\n</ul>\n<p>Classifier ensemble consisting of catboost, xgboost, and lightgbm is used for event-masking.</p>\n<pre><code>[pred_ensemble == ] += .\n[pred_ensemble == ] += .\n</code></pre>\n<p>When applying classifier event-masking to a Catboost regressor trained on a Kaplan–Meier target, there was a huge CV improvement of about 0.01 (0.6781455227816864 --&gt; 0.6864895241696849). Applying event-masking usually does result in improvement for many of my models, but on this Kaplan–Meier catboost the improvement is very significant. </p>\n<p>This brought the CV of this catboost model on par to that of the PRL NN and the resulting ensemble of Kaplan–Meier catboost with event-masking + PRL NN with event-masking yielded a silver medal solution.</p>\n<h1>Closing Remarks</h1>\n<p>I learnt a lot from this competition and would like to specially give credit to <a href=\"https://www.kaggle.com/Albanito\" target=\"_blank\">@Albanito</a> for his notebook on event-masking. (<a href=\"https://www.kaggle.com/code/albansteff/event-masked-prl-nn\" target=\"_blank\">https://www.kaggle.com/code/albansteff/event-masked-prl-nn</a>)</p>\n<p>Thank you.</p>\n<h1>Code and Model Files</h1>\n<p>Inference Notebook: <a href=\"https://www.kaggle.com/code/sshiyu/silver-solution-100th\" target=\"_blank\">https://www.kaggle.com/code/sshiyu/silver-solution-100th</a><br>\nModel Files: <a href=\"https://www.kaggle.com/datasets/sshiyu/cibmtr-final-models\" target=\"_blank\">https://www.kaggle.com/datasets/sshiyu/cibmtr-final-models</a><br>\nModel Training Code: <a href=\"https://www.kaggle.com/datasets/sshiyu/cibmtr-final-model-training-code\" target=\"_blank\">https://www.kaggle.com/datasets/sshiyu/cibmtr-final-model-training-code</a></p>",
  "comments": [
    {
      "id": 3144438,
      "author_name": "minerppdy",
      "author_url": "",
      "post_date": "2025-03-08T11:14:45.857000",
      "content": "<p>incredible! just with one shot!</p>",
      "votes": 1,
      "replies": []
    }
  ],
  "comment_count": 1,
  "messages": [
    {
      "id": 3141996,
      "postDate": "2025-03-06T02:57:40.713Z",
      "content": "<p>Hi Everyone, I just wanted to share my solution as well as some tricks I used to boost my score.</p>\n<h1>Overview</h1>\n<ul>\n<li>Kaplan–Meier Catboost with event-masking + PRL NN with event-masking</li>\n<li>CV: 0.690</li>\n<li>Public LB Score: 0.689</li>\n<li>Private LB Score: 0.693</li>\n</ul>\n<h1>Tricks</h1>\n<ul>\n<li>Cross Validation Methodology</li>\n</ul>\n<ol>\n<li>StratifiedKFold using both race group and binned target values. This ensures balanced representation of race groups and target distributions across folds.</li>\n<li>10 folds across 30 different seeds were used for cross validation to train a total of 300 models for each unique model variant. The idea was to make the model ensemble more diverse and robust.</li>\n</ol>\n<pre><code>train[] = pd.cut(train[], =NUM_BINS, =)\ntrain[] = train[].astype(str) +  + train[].astype(str)\n\n seed  SEEDS:\n    skf = StratifiedKFold(=FOLDS, =, =seed)\n</code></pre>\n<p>This method significantly boosted my CV score—the more seeds I added, the higher the score. However, I stopped at 30 seeds, as my testing showed that adding more than 30 no longer improved the CV score.</p>\n<p>Running such a meticulous CV also served another purpose: preventing me from overfitting to specific folds during testing. This methodology was both stable and reliable, giving me confidence in the feedback from my CV.</p>\n<ul>\n<li>Event-Masking Using Classifiers</li>\n</ul>\n<p>Classifier ensemble consisting of catboost, xgboost, and lightgbm is used for event-masking.</p>\n<pre><code>[pred_ensemble == ] += .\n[pred_ensemble == ] += .\n</code></pre>\n<p>When applying classifier event-masking to a Catboost regressor trained on a Kaplan–Meier target, there was a huge CV improvement of about 0.01 (0.6781455227816864 --&gt; 0.6864895241696849). Applying event-masking usually does result in improvement for many of my models, but on this Kaplan–Meier catboost the improvement is very significant. </p>\n<p>This brought the CV of this catboost model on par to that of the PRL NN and the resulting ensemble of Kaplan–Meier catboost with event-masking + PRL NN with event-masking yielded a silver medal solution.</p>\n<h1>Closing Remarks</h1>\n<p>I learnt a lot from this competition and would like to specially give credit to <a href=\"https://www.kaggle.com/Albanito\" target=\"_blank\">@Albanito</a> for his notebook on event-masking. (<a href=\"https://www.kaggle.com/code/albansteff/event-masked-prl-nn\" target=\"_blank\">https://www.kaggle.com/code/albansteff/event-masked-prl-nn</a>)</p>\n<p>Thank you.</p>\n<h1>Code and Model Files</h1>\n<p>Inference Notebook: <a href=\"https://www.kaggle.com/code/sshiyu/silver-solution-100th\" target=\"_blank\">https://www.kaggle.com/code/sshiyu/silver-solution-100th</a><br>\nModel Files: <a href=\"https://www.kaggle.com/datasets/sshiyu/cibmtr-final-models\" target=\"_blank\">https://www.kaggle.com/datasets/sshiyu/cibmtr-final-models</a><br>\nModel Training Code: <a href=\"https://www.kaggle.com/datasets/sshiyu/cibmtr-final-model-training-code\" target=\"_blank\">https://www.kaggle.com/datasets/sshiyu/cibmtr-final-model-training-code</a></p>",
      "rawMarkdown": "Hi Everyone, I just wanted to share my solution as well as some tricks I used to boost my score.\n\n\n# Overview\n- Kaplan–Meier Catboost with event-masking + PRL NN with event-masking\n- CV: 0.690\n- Public LB Score: 0.689\n- Private LB Score: 0.693\n\n\n# Tricks\n- Cross Validation Methodology\n\n1. StratifiedKFold using both race group and binned target values. This ensures balanced representation of race groups and target distributions across folds.\n2. 10 folds across 30 different seeds were used for cross validation to train a total of 300 models for each unique model variant. The idea was to make the model ensemble more diverse and robust.\n```\ntrain['y_bins'] = pd.cut(train['y'], bins=NUM_BINS, labels=False)\ntrain['stratify_col'] = train['race_group'].astype(str) + '_' + train['y_bins'].astype(str)\n\nfor seed in SEEDS:\n    skf = StratifiedKFold(n_splits=FOLDS, shuffle=True, random_state=seed)\n```\nThis method significantly boosted my CV score—the more seeds I added, the higher the score. However, I stopped at 30 seeds, as my testing showed that adding more than 30 no longer improved the CV score.\n\nRunning such a meticulous CV also served another purpose: preventing me from overfitting to specific folds during testing. This methodology was both stable and reliable, giving me confidence in the feedback from my CV.\n\n\n- Event-Masking Using Classifiers\n\nClassifier ensemble consisting of catboost, xgboost, and lightgbm is used for event-masking.\n\n```\npred_cat[pred_ensemble == 1] += 0.13\ntest_pred_nn_reloaded[pred_ensemble == 1] += 0.16\n```\n\nWhen applying classifier event-masking to a Catboost regressor trained on a Kaplan–Meier target, there was a huge CV improvement of about 0.01 (0.6781455227816864 --> 0.6864895241696849). Applying event-masking usually does result in improvement for many of my models, but on this Kaplan–Meier catboost the improvement is very significant. \n\nThis brought the CV of this catboost model on par to that of the PRL NN and the resulting ensemble of Kaplan–Meier catboost with event-masking + PRL NN with event-masking yielded a silver medal solution.\n\n\n# Closing Remarks\nI learnt a lot from this competition and would like to specially give credit to @Albanito for his notebook on event-masking. (https://www.kaggle.com/code/albansteff/event-masked-prl-nn)\n\nThank you.\n\n\n# Code and Model Files\nInference Notebook: https://www.kaggle.com/code/sshiyu/silver-solution-100th\nModel Files: https://www.kaggle.com/datasets/sshiyu/cibmtr-final-models\nModel Training Code: https://www.kaggle.com/datasets/sshiyu/cibmtr-final-model-training-code",
      "votes": 10
    },
    {
      "id": 3144438,
      "postDate": "2025-03-08T11:14:45.857Z",
      "content": "<p>incredible! just with one shot!</p>",
      "rawMarkdown": "incredible! just with one shot!",
      "votes": 1
    }
  ],
  "complete": true,
  "source": "live",
  "fetched_at": "2026-10-19T19:46:33Z",
  "competition_id": 70942,
  "competition_slug": "equity-post-HCT-survival-predictions",
  "team_id": 13111515,
  "solution_url": "/competitions/equity-post-HCT-survival-predictions/writeups/sshiyu-silver-solution-100th"
}