\n
![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F7421663%2F2eacebe9e5e564735c30383a7490e35d%2Fimportance.jpg?generation=1741265067440316&alt=media\")
Very happy to get my first silver medal, and this was also the first time I attended a competition from the beginning to the end without giving up halfway.
\nMy solution consists of three parts:
\nPart 1. Models derived from Chris's notebooks. I use XGBoost, LightGBM, and CatBoost with KM, NA, and Cox target transformation. Furthermore, each model is trained on two random seeds and then averaged.
\nPart 2. LitNN with pairwise ranking loss. This model can boost my CV and LB significantly, so I fine-tune them on three random seeds. One of these models can reach a CV of 0.683, but it always harms my ensemble's CV, so I don't include it in my final submission.
\nPart 3. Yunbase. I use the public yunbase without changing anything. This model increases my CV from 0.6869 to 0.6876. I think its unique feature engineering and training settings boost diversity in my ensemble.
\nFinally, I use a hill climbing algorithm to ensemble these models with the search step of 0.01. In my case, hill climbing is better than Optuna.
This graph shows how the model's standardized out-of-fold prediction is added to the ensemble. Interestingly, many models with low CV are added to the ensemble first:
\n
More importantly, I cannot reach this place without excellent public notebooks and enlightening discussions, I really learnt a lot of models and tricks in this competition. Thank you for your self-giving sharing!!
", "comments": [], "comment_count": 0, "messages": [ { "id": 3142561, "postDate": "2025-03-06T13:27:36.363Z", "content": "Very happy to get my first silver medal, and this was also the first time I attended a competition from the beginning to the end without giving up halfway.
\nMy solution consists of three parts:
\nPart 1. Models derived from Chris's notebooks. I use XGBoost, LightGBM, and CatBoost with KM, NA, and Cox target transformation. Furthermore, each model is trained on two random seeds and then averaged.
\nPart 2. LitNN with pairwise ranking loss. This model can boost my CV and LB significantly, so I fine-tune them on three random seeds. One of these models can reach a CV of 0.683, but it always harms my ensemble's CV, so I don't include it in my final submission.
\nPart 3. Yunbase. I use the public yunbase without changing anything. This model increases my CV from 0.6869 to 0.6876. I think its unique feature engineering and training settings boost diversity in my ensemble.
\nFinally, I use a hill climbing algorithm to ensemble these models with the search step of 0.01. In my case, hill climbing is better than Optuna.
This graph shows how the model's standardized out-of-fold prediction is added to the ensemble. Interestingly, many models with low CV are added to the ensemble first:
\n
More importantly, I cannot reach this place without excellent public notebooks and enlightening discussions, I really learnt a lot of models and tricks in this competition. Thank you for your self-giving sharing!!
", "rawMarkdown": "Very happy to get my first silver medal, and this was also the first time I attended a competition from the beginning to the end without giving up halfway. \n# Solution Overview\nMy solution consists of three parts: \nPart 1. Models derived from Chris's notebooks. I use XGBoost, LightGBM, and CatBoost with KM, NA, and Cox target transformation. Furthermore, each model is trained on two random seeds and then averaged. \nPart 2. LitNN with pairwise ranking loss. This model can boost my CV and LB significantly, so I fine-tune them on three random seeds. One of these models can reach a CV of 0.683, but it always harms my ensemble's CV, so I don't include it in my final submission. \nPart 3. Yunbase. I use the public yunbase without changing anything. This model increases my CV from 0.6869 to 0.6876. I think its unique feature engineering and training settings boost diversity in my ensemble.\nFinally, I use a hill climbing algorithm to ensemble these models with the search step of 0.01. In my case, hill climbing is better than Optuna.\n\n# Useful Tricks\n1. **Power transformation (scipy.stats.yeojohnson)**. In Part 1, I apply power transformation to the efs_time before any other target transformation. This thought comes from Chris, because RMSE and MSE Loss are suitable to the normalized target. I employ it separately in the event/survival group.\n2. **Enable negative weights in hill climbing**. I find that averaging Part 1 and Part 2 models directly leads to the degradation of CV and LB. I guess that's because some models are biased (tend to predict higher/lower than the ground truth). Therefore, I enable negative weights when ensembling, which significantly improve my CV and LB.\n\n# Model Importance\nThis graph shows how the model's standardized out-of-fold prediction is added to the ensemble. Interestingly, many models with low CV are added to the ensemble first:\n \n\n# Thanks\nMore importantly, I cannot reach this place without excellent public notebooks and enlightening discussions, I really learnt a lot of models and tricks in this competition. Thank you for your self-giving sharing!!", "votes": 5 } ], "complete": true, "source": "live", "fetched_at": "2026-10-19T19:48:00Z", "competition_id": 70942, "competition_slug": "equity-post-HCT-survival-predictions", "team_id": 13074402, "solution_url": "/competitions/equity-post-HCT-survival-predictions/writeups/tlhoerd-35th-place-solution-ensemble-10-models-wit" }