Full model* code: https://www.kaggle.com/code/herrahuu/4th-place-solution
\nPrivate score: 0.69936, Public score: 0.69724, total runtime: 4h
\n*this version also includes LightGBM model not mentioned below, but impact is minimal in ensemble
For comparison solo model scores (without bagging):
\nCatBoost: 0.69784, 0.69500, 30min
\nXGBoost: 0.69765, 0.69538, 5min
\nTabM: 0.69636, 0.69383, 6min
\nLightGBM: 0.69793, 0.69516, 5min
~~~~
\nIt was fun to make a small comeback to Kaggle after a long break. Especially given this nice \"old school\" competition, not too big & tabular dataset. Below is a short summary of my solution.
\nThe goal of this competition was to rank patients based on their risk scores for allogeneic HCT transplantation events. As a first step it's useful to think what a perfect solution would look like. In this case that would be:
\nClearly this ranking problem is non-differentiable, especially as the 1) introduces step function like behavior for the desired ranks. One common way to handle this type of problems is to introduce some sort of soft/smoothed version of the original task. However, in my experience, such approaches tend to make things unstable during training, or just don't really match to the original problem that well if smoothed too much.
\nSo instead, my main idea for the competition was to follow the steps 1-2) directly and divide the problem into two parts:
\nFinally, given these two predictions, we can calculate the expected ranking position as a risk score.
\nrisk_score = P(event = )*(s0_group/) + P(event = )*(s0_group + (-s0_group)*E[rank% | event = ])\n, where s0_group = sum_{race_group = group} P(event = ) / N_group \nFor both of these tasks, three different models are implemented using the following packages/algorithms:
\nSo, in total six models are trained. First their predictions are merged by just calculating weighted sums (separately for both time and event using different weights), and then finally risk scores are calculated by the the formula given above.
", "comments": [ { "id": 3157197, "author_name": "MJeremy", "author_url": "", "post_date": "2025-03-23T05:36:52.177000", "content": "hi thanks for sharing. But I got quite confused at the risk score formulae, I feel mathematically the
\nrisk_score = P() * P(rank | )\nHow did you come up with the s0_group / 2 and also the additions of the formulae?
", "votes": 1, "replies": [ { "id": 3157420, "author_name": "Herra Huu", "author_url": "", "post_date": "2025-03-23T12:26:28.997000", "content": "Your formula would represents the joint probability distribution rather than the expected value.
\nTo build intuition around the original formulation, consider that we would like to emulate the function scipy.rankdata(data, method='average') (https://docs.scipy.org/doc/scipy/reference/generated/scipy.stats.rankdata.html ). In that context, the term s0_group/2 corresponds to the contribution of the parameter method='average'.
", "votes": 1, "replies": [] } ] }, { "id": 3141870, "author_name": "Aleksandr Larko", "author_url": "", "post_date": "2025-03-06T00:50:53.807000", "content": "This is awesome , good job! Congratulations！
", "votes": 1, "replies": [] }, { "id": 3150721, "author_name": "Robert Hatch", "author_url": "", "post_date": "2025-03-15T20:29:35.830000", "content": "I notice you did training on full train data. How did you tune hyperparameters for each model and ensemble weights for the final predictions? Using normal CV methods and later retrain on full with tuned params and weights?
", "votes": 0, "replies": [ { "id": 3150741, "author_name": "Herra Huu", "author_url": "", "post_date": "2025-03-15T21:29:03.017000", "content": "Yeah, exactly that for both hyperparameters&ensemble weights.
", "votes": 1, "replies": [] } ] }, { "id": 3141889, "author_name": "Robert Hatch", "author_url": "", "post_date": "2025-03-06T01:09:04.433000", "content": "Nice, that's basically exactly what I did as well for 5th! Congrats!
\nI did go ahead and use a million models, and blend in predictions from an ensemble of 'normal' risk score predictions. And I post-processed for a really nice boost… but the core was the same formula basis.
", "votes": 0, "replies": [] }, { "id": 3141842, "author_name": "SCRIPTCHEF", "author_url": "", "post_date": "2025-03-06T00:23:52.380000", "content": "This is awesome, thanks for sharing! Congrats!
", "votes": 0, "replies": [] }, { "id": 3141840, "author_name": "", "author_url": "", "post_date": "2025-03-06T00:21:56.520000", "content": "", "votes": 0, "replies": [] } ], "comment_count": 8, "messages": [ { "id": 3141829, "postDate": "2025-03-06T00:15:34.587Z", "content": "Full model* code: https://www.kaggle.com/code/herrahuu/4th-place-solution
\nPrivate score: 0.69936, Public score: 0.69724, total runtime: 4h
\n*this version also includes LightGBM model not mentioned below, but impact is minimal in ensemble
For comparison solo model scores (without bagging):
\nCatBoost: 0.69784, 0.69500, 30min
\nXGBoost: 0.69765, 0.69538, 5min
\nTabM: 0.69636, 0.69383, 6min
\nLightGBM: 0.69793, 0.69516, 5min
~~~~
\nIt was fun to make a small comeback to Kaggle after a long break. Especially given this nice \"old school\" competition, not too big & tabular dataset. Below is a short summary of my solution.
\nThe goal of this competition was to rank patients based on their risk scores for allogeneic HCT transplantation events. As a first step it's useful to think what a perfect solution would look like. In this case that would be:
\nClearly this ranking problem is non-differentiable, especially as the 1) introduces step function like behavior for the desired ranks. One common way to handle this type of problems is to introduce some sort of soft/smoothed version of the original task. However, in my experience, such approaches tend to make things unstable during training, or just don't really match to the original problem that well if smoothed too much.
\nSo instead, my main idea for the competition was to follow the steps 1-2) directly and divide the problem into two parts:
\nFinally, given these two predictions, we can calculate the expected ranking position as a risk score.
\nrisk_score = P(event = )*(s0_group/) + P(event = )*(s0_group + (-s0_group)*E[rank% | event = ])\n, where s0_group = sum_{race_group = group} P(event = ) / N_group \nFor both of these tasks, three different models are implemented using the following packages/algorithms:
\nSo, in total six models are trained. First their predictions are merged by just calculating weighted sums (separately for both time and event using different weights), and then finally risk scores are calculated by the the formula given above.
", "rawMarkdown": "Full model* code: https://www.kaggle.com/code/herrahuu/4th-place-solution\nPrivate score: 0.69936, Public score: 0.69724, total runtime: 4h\n*this version also includes LightGBM model not mentioned below, but impact is minimal in ensemble\n\nFor comparison solo model scores (without bagging):\nCatBoost: 0.69784, 0.69500, 30min\nXGBoost: 0.69765, 0.69538, 5min\nTabM: 0.69636, 0.69383, 6min\nLightGBM: 0.69793, 0.69516, 5min\n\n~~~~\n\nIt was fun to make a small comeback to Kaggle after a long break. Especially given this nice \"old school\" competition, not too big & tabular dataset. Below is a short summary of my solution.\n\n# Introduction\nThe goal of this competition was to rank patients based on their risk scores for allogeneic HCT transplantation events. As a first step it's useful to think what a perfect solution would look like. In this case that would be:\n1. all patients with no event (efs=0) should be at the bottom of the list (no order defined between them)\n2. for efs = 1, patients are ordered by the event times\n\nClearly this ranking problem is non-differentiable, especially as the 1) introduces step function like behavior for the desired ranks. One common way to handle this type of problems is to introduce some sort of soft/smoothed version of the original task. However, in my experience, such approaches tend to make things unstable during training, or just don't really match to the original problem that well if smoothed too much. \n\nSo instead, my main idea for the competition was to follow the steps 1-2) directly and divide the problem into two parts: \n1. predict the probability of the event\n2. predict the expected ranking position among patients with events\n\nFinally, given these two predictions, we can calculate the expected ranking position as a risk score.\n\n# Model formulation:\n```python\nrisk_score = P(event = 0)*(s0_group/2) + P(event = 1)*(s0_group + (1-s0_group)*E[rank% | event = 1])\n, where s0_group = sum_{race_group = group} P(event = 0) / N_group \n\n```\n## P(event):\n- binary classification for efs events\n- censored data points are treated as a partial observations\n- weights for those data points are calculated as cumulative densities given by Kaplan-Meier estimator and scaled to (0,1) range.\n\n## E[rank% | event = 1]:\n- conditional regression model to predict time based rankings for data points where event did happen\n- the model is trained using only data points which have efs = 1\n- target is rank% = rank(-time)/N. \n- so, target values are between 0 and 1, 0 = longest time and 1 = shortest\n- with all algorithms, I'm using target transformation of the form: inverse_normal_cdf(rank%) \"z-score\", and then predictions are transformed back to percentages by normal_cdf(pred)\n\n# Algorithms:\nFor both of these tasks, three different models are implemented using the following packages/algorithms:\n- TabM\n- CatBoost\n- XGBoost\n\nSo, in total six models are trained. First their predictions are merged by just calculating weighted sums (separately for both time and event using different weights), and then finally risk scores are calculated by the the formula given above.\n\n\n", "votes": 53 }, { "id": 3157197, "postDate": "2025-03-23T05:36:52.177Z", "content": "hi thanks for sharing. But I got quite confused at the risk score formulae, I feel mathematically the
\nrisk_score = P() * P(rank | )\nHow did you come up with the s0_group / 2 and also the additions of the formulae?
", "rawMarkdown": "hi thanks for sharing. But I got quite confused at the risk score formulae, I feel mathematically the \n\n```\nrisk_score = P(event) * P(rank | event)\n```\n\nHow did you come up with the s0_group / 2 and also the additions of the formulae?", "votes": 1, "replies": [ { "id": 3157420, "postDate": "2025-03-23T12:26:28.997Z", "content": "Your formula would represents the joint probability distribution rather than the expected value.
\nTo build intuition around the original formulation, consider that we would like to emulate the function scipy.rankdata(data, method='average') (https://docs.scipy.org/doc/scipy/reference/generated/scipy.stats.rankdata.html ). In that context, the term s0_group/2 corresponds to the contribution of the parameter method='average'.
", "rawMarkdown": "Your formula would represents the joint probability distribution rather than the expected value.\n\nTo build intuition around the original formulation, consider that we would like to emulate the function scipy.rankdata(data, method='average') (https://docs.scipy.org/doc/scipy/reference/generated/scipy.stats.rankdata.html ). In that context, the term s0_group/2 corresponds to the contribution of the parameter method='average'.", "votes": 1 } ] }, { "id": 3141870, "postDate": "2025-03-06T00:50:53.807Z", "content": "This is awesome , good job! Congratulations！
", "rawMarkdown": "This is awesome , good job! Congratulations！", "votes": 1 }, { "id": 3150721, "postDate": "2025-03-15T20:29:35.830Z", "content": "I notice you did training on full train data. How did you tune hyperparameters for each model and ensemble weights for the final predictions? Using normal CV methods and later retrain on full with tuned params and weights?
", "rawMarkdown": "I notice you did training on full train data. How did you tune hyperparameters for each model and ensemble weights for the final predictions? Using normal CV methods and later retrain on full with tuned params and weights?", "replies": [ { "id": 3150741, "postDate": "2025-03-15T21:29:03.017Z", "content": "Yeah, exactly that for both hyperparameters&ensemble weights.
", "rawMarkdown": "Yeah, exactly that for both hyperparameters&ensemble weights.", "votes": 1 } ] }, { "id": 3141889, "postDate": "2025-03-06T01:09:04.433Z", "content": "Nice, that's basically exactly what I did as well for 5th! Congrats!
\nI did go ahead and use a million models, and blend in predictions from an ensemble of 'normal' risk score predictions. And I post-processed for a really nice boost… but the core was the same formula basis.
", "rawMarkdown": "Nice, that's basically exactly what I did as well for 5th! Congrats!\n\nI did go ahead and use a million models, and blend in predictions from an ensemble of 'normal' risk score predictions. And I post-processed for a really nice boost... but the core was the same formula basis." }, { "id": 3141842, "postDate": "2025-03-06T00:23:52.380Z", "content": "This is awesome, thanks for sharing! Congrats!
", "rawMarkdown": "This is awesome, thanks for sharing! Congrats!" }, { "id": 3141840, "postDate": "2025-03-06T00:21:56.520Z", "rawMarkdown": "", "isDeleted": true } ], "complete": true, "source": "live", "fetched_at": "2026-10-19T19:44:54Z", "competition_id": 70942, "competition_slug": "equity-post-HCT-survival-predictions", "team_id": 13137282, "solution_url": "/competitions/equity-post-HCT-survival-predictions/writeups/herra-huu-4th-place-factorized-modelling-approach" }