Thank you Kaggle staff and competition organizers for such a interesting competition. Also, thanks to everyone for sharing during the competition. I learned lot of things in this competition from exploring different tabular model to ensembling them and about survival analysis data.
\nI have open sourced almost all my training notebooks in the GitHub account. here is the link : https://github.com/jaytonde/Kaggle-CIBMTR-2024
\nI have also made my final inference notebook public here : https://www.kaggle.com/code/jaytonde/hill-climbing-submission
\nI did not maintained the code well cause of shortage of time so reach out to me in case of any doubts.
\nMy solution is very simple yet generalized cause I do not have much gap in CV, LB and Private LB.
\nI have performed multiple experiments with different model and target preparation techniques and my final solution is just hill climbing of this experiments.
As per the discussions in the competition about folding strategies, I decided to do 10 random folds, as 5 folds stratified by race wasn't beneficial.
\nFinal solution have diverse ensemble of neural networks, XGBoost, CatBoost, LGBM, Lasso regression, TabFPN, Support Vector Regression, Tablenet, torch surv, Ridge Regression, TabM, Linear Regression, Table Transformer.
\nFollowing is my experiments with respective weights in hill climbing ensemble.
\nFirst best model having experiment name prlnn-exp-01 which is Refactoring Pairwise Ranking Network with classification mask and 0.2 shift.
| Experiment | \nDescription | \nWeight | \n
|---|---|---|
| catboost-exp-05 | \nCatBoost with Kaplan Meier targets and classification mask | \n0.48000000000000087 | \n
| xgboost-exp-09 | \nXGBoost with Monotone constraints and Kaplan Meier targets | \n0.2700000000000007 | \n
| lgbm-exp-08 | \nLGBM with BreslowFlemingHarringtonFitter targets | \n-0.20999999999999974 | \n
| catboost-exp-01 | \nCatBoost with Kaplan Meier targets | \n0.10000000000000053 | \n
| lasso-exp-01 | \nLasso regressor with Kaplan Meier targets | \n-0.03999999999999959 | \n
| nn-exp-04 | \nNeural Network having classification mask and 0.1 shift | \n0.12000000000000055 | \n
| svr-exp-06 | \nSupport vector regression with Nelson Aalen Fitter targets and classification mask | \n-0.0499999999999996 | \n
| ds-exp-01 | \nCoxPH model | \n0.04000000000000048 | \n
| rf-exp-05 | \nRandom Forest with Kaplan Meier targets and classification mask | \n-0.05999999999999961 | \n
| svr-exp-01 | \nSupport vector regression with Kaplan Meier targets | \n-0.03999999999999959 | \n
| tf-exp-01 | \ntabpfn with Kaplan Meier targets | \n0.04000000000000048 | \n
| catboost-exp-03 | \nCatBoost with Nelson Alen targets | \n-0.03999999999999959 | \n
| catboost-exp-06 | \nCatBoost with Nelson Aalen Fitter targets and classification mask | \n0.0600000000000005 | \n
| tn-exp-02 | \nTablenet | \n0.03000000000000047 | \n
| xgboost-exp-02 | \nXGBoost with Cox Loss | \n-0.0499999999999996 | \n
| catboost-exp-04 | \nCatBoost with CoxPHFitter targets | \n0.020000000000000462 | \n
| lgbm-exp-03 | \nLGBM with Nelson Alen targets | \n-0.029999999999999583 | \n
| ts-exp-01 | \nTorch Surv model | \n-0.019999999999999574 | \n
| xgboost-exp-10 | \n\n | 0.04000000000000048 | \n
| xgboost-exp-06 | \nXGBoost with Nelson Aalen Fitter targets and classification mask | \n-0.029999999999999583 | \n
| en-exp-02 | \nElastiNet with classification mask | \n0.020000000000000462 | \n
| ri-exp-06 | \nRidge regression with Nelson Alen targets and classification mask | \n-0.029999999999999583 | \n
| tabm-exp-02 | \nTabM Model | \n0.0600000000000005 | \n
| nn-exp-01 | \nNeural Network with 2 targets into 1 conversion | \n-0.05999999999999961 | \n
| lgbm-exp-01 | \nLGBM with Kaplan Meier targets | \n-0.0499999999999996 | \n
| nn-exp-06 | \nRefactoring Pairwise Ranking Network, thanks @albansteff | \n-0.0499999999999996 | \n
| xgboost-exp-05 | \nXGBoost with Kaplan Meier targets tuned parameters | \n0.04000000000000048 | \n
| lir-exp-01 | \nLinear regression with Kaplan Meier targets | \n-0.019999999999999574 | \n
| lgbm-exp-04 | \nLGBM with CoxPHFitter targets | \n0.020000000000000462 | \n
| nn-exp-05 | \nNeural Network with some pre processing applied | \n0.010000000000000453 | \n
| lgbm-exp-06 | \nLGBM with Nelson Alen Targets | \n-0.009999999999999565 | \n
| nn-exp-02 | \nNeural network with varied data pre processing | \n0.010000000000000453 | \n
| xgboost-exp-01 | \nXGBoost with Kaplan Meier targets | \n4.440892098500626e-16 | \n
| tt-exp-01 | \nTable Transformers Kaplan Meier Targets | \n4.440892098500626e-16 | \n
For more details about model training and target preparation look out the corresponding notebooks in my GitHub repo. Experiment name is the notebook name in the repo.
\n| Description | \nCV | \nPublic LB | \nPrivate LB | \n
|---|---|---|---|
| Ensemble of above models with weights and rankdata | \n0.688 | \n0.693 | \n0.694 | \n
I would like to thank everyone who shared during the competition. I learned a lot and will try to apply all the learning in the next competition. Also, a special thanks to amazing notebooks by @cdeotte for his detailed discussions and starter notebooks.
", "comments": [ { "id": 3150363, "author_name": "Darwish 2030", "author_url": "", "post_date": "2025-03-15T12:09:04.327000", "content": "A simple yet powerful approach with diverse models and solid generalization
", "votes": 1, "replies": [] } ], "comment_count": 1, "messages": [ { "id": 3142916, "postDate": "2025-03-06T18:46:51.337Z", "content": "Thank you Kaggle staff and competition organizers for such a interesting competition. Also, thanks to everyone for sharing during the competition. I learned lot of things in this competition from exploring different tabular model to ensembling them and about survival analysis data.
\nI have open sourced almost all my training notebooks in the GitHub account. here is the link : https://github.com/jaytonde/Kaggle-CIBMTR-2024
\nI have also made my final inference notebook public here : https://www.kaggle.com/code/jaytonde/hill-climbing-submission
\nI did not maintained the code well cause of shortage of time so reach out to me in case of any doubts.
\nMy solution is very simple yet generalized cause I do not have much gap in CV, LB and Private LB.
\nI have performed multiple experiments with different model and target preparation techniques and my final solution is just hill climbing of this experiments.
As per the discussions in the competition about folding strategies, I decided to do 10 random folds, as 5 folds stratified by race wasn't beneficial.
\nFinal solution have diverse ensemble of neural networks, XGBoost, CatBoost, LGBM, Lasso regression, TabFPN, Support Vector Regression, Tablenet, torch surv, Ridge Regression, TabM, Linear Regression, Table Transformer.
\nFollowing is my experiments with respective weights in hill climbing ensemble.
\nFirst best model having experiment name prlnn-exp-01 which is Refactoring Pairwise Ranking Network with classification mask and 0.2 shift.
| Experiment | \nDescription | \nWeight | \n
|---|---|---|
| catboost-exp-05 | \nCatBoost with Kaplan Meier targets and classification mask | \n0.48000000000000087 | \n
| xgboost-exp-09 | \nXGBoost with Monotone constraints and Kaplan Meier targets | \n0.2700000000000007 | \n
| lgbm-exp-08 | \nLGBM with BreslowFlemingHarringtonFitter targets | \n-0.20999999999999974 | \n
| catboost-exp-01 | \nCatBoost with Kaplan Meier targets | \n0.10000000000000053 | \n
| lasso-exp-01 | \nLasso regressor with Kaplan Meier targets | \n-0.03999999999999959 | \n
| nn-exp-04 | \nNeural Network having classification mask and 0.1 shift | \n0.12000000000000055 | \n
| svr-exp-06 | \nSupport vector regression with Nelson Aalen Fitter targets and classification mask | \n-0.0499999999999996 | \n
| ds-exp-01 | \nCoxPH model | \n0.04000000000000048 | \n
| rf-exp-05 | \nRandom Forest with Kaplan Meier targets and classification mask | \n-0.05999999999999961 | \n
| svr-exp-01 | \nSupport vector regression with Kaplan Meier targets | \n-0.03999999999999959 | \n
| tf-exp-01 | \ntabpfn with Kaplan Meier targets | \n0.04000000000000048 | \n
| catboost-exp-03 | \nCatBoost with Nelson Alen targets | \n-0.03999999999999959 | \n
| catboost-exp-06 | \nCatBoost with Nelson Aalen Fitter targets and classification mask | \n0.0600000000000005 | \n
| tn-exp-02 | \nTablenet | \n0.03000000000000047 | \n
| xgboost-exp-02 | \nXGBoost with Cox Loss | \n-0.0499999999999996 | \n
| catboost-exp-04 | \nCatBoost with CoxPHFitter targets | \n0.020000000000000462 | \n
| lgbm-exp-03 | \nLGBM with Nelson Alen targets | \n-0.029999999999999583 | \n
| ts-exp-01 | \nTorch Surv model | \n-0.019999999999999574 | \n
| xgboost-exp-10 | \n\n | 0.04000000000000048 | \n
| xgboost-exp-06 | \nXGBoost with Nelson Aalen Fitter targets and classification mask | \n-0.029999999999999583 | \n
| en-exp-02 | \nElastiNet with classification mask | \n0.020000000000000462 | \n
| ri-exp-06 | \nRidge regression with Nelson Alen targets and classification mask | \n-0.029999999999999583 | \n
| tabm-exp-02 | \nTabM Model | \n0.0600000000000005 | \n
| nn-exp-01 | \nNeural Network with 2 targets into 1 conversion | \n-0.05999999999999961 | \n
| lgbm-exp-01 | \nLGBM with Kaplan Meier targets | \n-0.0499999999999996 | \n
| nn-exp-06 | \nRefactoring Pairwise Ranking Network, thanks @albansteff | \n-0.0499999999999996 | \n
| xgboost-exp-05 | \nXGBoost with Kaplan Meier targets tuned parameters | \n0.04000000000000048 | \n
| lir-exp-01 | \nLinear regression with Kaplan Meier targets | \n-0.019999999999999574 | \n
| lgbm-exp-04 | \nLGBM with CoxPHFitter targets | \n0.020000000000000462 | \n
| nn-exp-05 | \nNeural Network with some pre processing applied | \n0.010000000000000453 | \n
| lgbm-exp-06 | \nLGBM with Nelson Alen Targets | \n-0.009999999999999565 | \n
| nn-exp-02 | \nNeural network with varied data pre processing | \n0.010000000000000453 | \n
| xgboost-exp-01 | \nXGBoost with Kaplan Meier targets | \n4.440892098500626e-16 | \n
| tt-exp-01 | \nTable Transformers Kaplan Meier Targets | \n4.440892098500626e-16 | \n
For more details about model training and target preparation look out the corresponding notebooks in my GitHub repo. Experiment name is the notebook name in the repo.
\n| Description | \nCV | \nPublic LB | \nPrivate LB | \n
|---|---|---|---|
| Ensemble of above models with weights and rankdata | \n0.688 | \n0.693 | \n0.694 | \n
I would like to thank everyone who shared during the competition. I learned a lot and will try to apply all the learning in the next competition. Also, a special thanks to amazing notebooks by @cdeotte for his detailed discussions and starter notebooks.
", "rawMarkdown": "Thank you Kaggle staff and competition organizers for such a interesting competition. Also, thanks to everyone for sharing during the competition. I learned lot of things in this competition from exploring different tabular model to ensembling them and about survival analysis data.\n\nI have open sourced almost all my training notebooks in the GitHub account. here is the link : https://github.com/jaytonde/Kaggle-CIBMTR-2024\n\nI have also made my final inference notebook public here : https://www.kaggle.com/code/jaytonde/hill-climbing-submission\n\nI did not maintained the code well cause of shortage of time so reach out to me in case of any doubts. \n\nMy solution is very simple yet generalized cause I do not have much gap in CV, LB and Private LB.\nI have performed multiple experiments with different model and target preparation techniques and my final solution is just hill climbing of this experiments.\n\n\n## Folding strategy\nAs per the discussions in the competition about folding strategies, I decided to do 10 random folds, as 5 folds stratified by race wasn't beneficial.\n\n\n## Ensemble\nFinal solution have diverse ensemble of neural networks, XGBoost, CatBoost, LGBM, Lasso regression, TabFPN, Support Vector Regression, Tablenet, torch surv, Ridge Regression, TabM, Linear Regression, Table Transformer.\n\nFollowing is my experiments with respective weights in hill climbing ensemble.\nFirst best model having experiment name prlnn-exp-01 which is Refactoring Pairwise Ranking Network with classification mask and 0.2 shift.\n\n| Experiment | Description | Weight |\n|------------|--------------------|--------|\n| catboost-exp-05 | CatBoost with Kaplan Meier targets and classification mask | 0.48000000000000087 |\n| xgboost-exp-09 | XGBoost with Monotone constraints and Kaplan Meier targets| 0.2700000000000007 |\n| lgbm-exp-08 | LGBM with BreslowFlemingHarringtonFitter targets | -0.20999999999999974 |\n| catboost-exp-01 | CatBoost with Kaplan Meier targets | 0.10000000000000053 |\n| lasso-exp-01 | Lasso regressor with Kaplan Meier targets | -0.03999999999999959 |\n| nn-exp-04 | Neural Network having classification mask and 0.1 shift | 0.12000000000000055 |\n| svr-exp-06 | Support vector regression with Nelson Aalen Fitter targets and classification mask| -0.0499999999999996 |\n| ds-exp-01 | CoxPH model | 0.04000000000000048 |\n| rf-exp-05 | Random Forest with Kaplan Meier targets and classification mask | -0.05999999999999961 |\n| svr-exp-01 | Support vector regression with Kaplan Meier targets | -0.03999999999999959 |\n| tf-exp-01 | tabpfn with Kaplan Meier targets | 0.04000000000000048 |\n| catboost-exp-03 | CatBoost with Nelson Alen targets | -0.03999999999999959 |\n| catboost-exp-06 | CatBoost with Nelson Aalen Fitter targets and classification mask | 0.0600000000000005 |\n| tn-exp-02 | Tablenet | 0.03000000000000047 |\n| xgboost-exp-02 | XGBoost with Cox Loss| -0.0499999999999996 |\n| catboost-exp-04 | CatBoost with CoxPHFitter targets | 0.020000000000000462 |\n| lgbm-exp-03 | LGBM with Nelson Alen targets| -0.029999999999999583 |\n| ts-exp-01 | Torch Surv model | -0.019999999999999574 |\n| xgboost-exp-10 | | 0.04000000000000048 |\n| xgboost-exp-06 | XGBoost with Nelson Aalen Fitter targets and classification mask | -0.029999999999999583 |\n| en-exp-02 | ElastiNet with classification mask | 0.020000000000000462 |\n| ri-exp-06 | Ridge regression with Nelson Alen targets and classification mask | -0.029999999999999583 |\n| tabm-exp-02 | TabM Model | 0.0600000000000005 |\n| nn-exp-01 | Neural Network with 2 targets into 1 conversion | -0.05999999999999961 |\n| lgbm-exp-01 | LGBM with Kaplan Meier targets | -0.0499999999999996 |\n| nn-exp-06 | Refactoring Pairwise Ranking Network, thanks @albansteff | -0.0499999999999996 |\n| xgboost-exp-05 | XGBoost with Kaplan Meier targets tuned parameters | 0.04000000000000048 |\n| lir-exp-01 | Linear regression with Kaplan Meier targets | -0.019999999999999574 |\n| lgbm-exp-04 | LGBM with CoxPHFitter targets | 0.020000000000000462 |\n| nn-exp-05 | Neural Network with some pre processing applied | 0.010000000000000453 |\n| lgbm-exp-06 | LGBM with Nelson Alen Targets | -0.009999999999999565 |\n| nn-exp-02 | Neural network with varied data pre processing | 0.010000000000000453 |\n| xgboost-exp-01 | XGBoost with Kaplan Meier targets | 4.440892098500626e-16 |\n| tt-exp-01 | Table Transformers Kaplan Meier Targets | 4.440892098500626e-16 |\n\nFor more details about model training and target preparation look out the corresponding notebooks in my GitHub repo. Experiment name is the notebook name in the repo.\n\n## Final Submission CV and LB : \n| Description | CV | Public LB | Private LB |\n| --- | ---------------------------------- |---------|--------|\n| Ensemble of above models with weights and rankdata | 0.688 | 0.693 | 0.694 |\n\n\n## Acknowledgments\nI would like to thank everyone who shared during the competition. I learned a lot and will try to apply all the learning in the next competition. Also, a special thanks to amazing notebooks by @cdeotte for his detailed discussions and starter notebooks.", "votes": 7 }, { "id": 3150363, "postDate": "2025-03-15T12:09:04.327Z", "content": "A simple yet powerful approach with diverse models and solid generalization
", "rawMarkdown": "A simple yet powerful approach with diverse models and solid generalization", "votes": 1 } ], "complete": true, "source": "live", "fetched_at": "2026-10-19T19:49:29Z", "competition_id": 70942, "competition_slug": "equity-post-HCT-survival-predictions", "team_id": 13072603, "solution_url": "/competitions/equity-post-HCT-survival-predictions/writeups/jaydev-tonde-22nd-place-solution-with-code" }