# 84th Place Solution

Congratulations to Kaggle and CIBMTR for hosting this competition and all Kagglers who shared their work and progress! In particular, I want to thank
[@albansteff](https://www.kaggle.com/albansteff)
and
[@abdmental01](https://www.kaggle.com/abdmental01)
for the notebooks behind the:
target transformation (
[notebook](https://www.kaggle.com/code/abdmental01/xgb-monotone-fork-cv-improvement)
)
Event Mask PRL-NN (
[notebook](https://www.kaggle.com/code/albansteff/event-masked-prl-nn)
)
and handling missing values in HLA columns (
[notebook](https://www.kaggle.com/code/albansteff/cibmtr-eda-ensemble-model-recalculate-hla)
)
I used in my best ensembles.
For all my in-house models I did not do feature aggregation and used 5-fold KFold CV scheme. My
best selected
submission is an ensemble of:
16 in-house models (8x GBDT and 8x MLP), based on my public
[baseline](https://www.kaggle.com/code/andreasbis/cibmtr-eda-ensemble-model)
2 variations of the
[Event-masked PRL-NN](https://www.kaggle.com/code/albansteff/event-masked-prl-nn)
notebook: one with 5-fold and one with 10-fold CV (an ensemble of these 2 achieves a CV of 0.68415)
My best overall solution (still 0.693) also used the
[XGB Monotone_ Fork _CV_Improvement](https://www.kaggle.com/code/abdmental01/xgb-monotone-fork-cv-improvement)
notebook; the CV of my ensemble improved to 0.68748, but because the public LB dropped by 0.001 (0.693 -> 0.692), I did not choose it.
GBDT Models
The GBDT models I used are based on the Kaplan-Meier target of
[this](https://www.kaggle.com/code/abdmental01/xgb-monotone-fork-cv-improvement)
notebook. The targets I used were: Kaplan-Meier, Nelson-Aalen and the transformation suggested by
[@cdeotte](https://www.kaggle.com/cdeotte)
which is suitable for the Cox loss function:
:

     ():

        .data = data
        .cat_cols = cat_cols

        ._threshold = 
        ._shift = 

     ():

        data = .data.loc[(.data.efs_time < ._threshold) | (.data.efs == )]

        km = KaplanMeierFitter()        
        km.fit(durations=data[], event_observed=data[])

        target = km.survival_function_at_times(.data[]).values        
        target[.data.efs == ] = target[.data.efs == ].() - ._shift
        target /= np.std(target)

        .data[] = target

         .data

     ():

        data = .data.loc[(.data.efs_time < ._threshold) | (.data.efs == )]

        na = NelsonAalenFitter()    
        na.fit(durations=data[], event_observed=data[])

        target = -na.cumulative_hazard_at_times(.data[]).values 
        target[.data.efs == ] = target[.data.efs == ].() - ._shift
        target /= np.std(target)

        .data[] = target

         .data

     ():

        .data[] = .data.efs_time.copy()
        .data.loc[.data.efs == , ] *= -

         .data
For the Kaplan-Meier and Nelson-Aalen targets, I used: CatBoost, LightGBM, XGBoost
For the Cox-loss target, I used two CatBoost models with different parameters.
MLP Models
The MLP models were useful
only
if I used the OOF predictions of GBDT models; in order to have some variety in predictions, I trained a different set of 14 GBDT models (without target post-processing as I do with the GBDT models above) and use their predictions as meta features.
The targets I used were: CoxPH, Kaplan-Meier, Nelson-Aalen, Quantile transform. I also used two loss functions:
stratified MSE loss, calculated as:
def (self, outputs, y, race_groups, alpha=):

        unique_races = race_groups.()
        group_losses = []

        for race in unique_races:

            race_mask = (race_groups.() == race)
            race_mask = torch.(race_mask, device=outputs.device, dtype=torch.bool)

            race_outputs = outputs[race_mask]
            race_y = y[race_mask]

            group_loss = torch.nn.functional.(race_outputs.(), race_y)
            group_losses.(group_loss)

        group_losses_tensor = torch.(group_losses)

        return group_losses_tensor.() + alpha * group_losses_tensor.()
and Pairwise Ranking loss, which is calculated in the same fashion as the loss above.
Unsuccessful Experiments
Some of my unsuccessful (that did not improve both CV and LB) were:
Implementing GNN: while I implemented an architecture using pytorch (not torch_geometric) it generalizes well only on the local CV, as the ensemble which had GNN models dropped from 0.693 to0.692 (luckily I didn't choose this one as best-LB!)
As per the following
[post](https://www.kaggle.com/competitions/equity-post-HCT-survival-predictions/discussion/555490)
, I had tried generating additional data using SurvivalGAN, but the resulting models did not perform well on the LB.
Early in the competition, I had tried TTA (Test Time Augmentation) by adding guassian noise to the (only) two continuous features of the dataset (age_at_hct, donor_age) but the LB degraded.
DeBERTa; my idea was to convert all categorical columns (and some numeric columns that describe patients' age and health) into one text column and feed this column into DeBERTa using custom Pooling layers. While the CV of these models did not exceed 0.654, I learned to use pytorch and huggingface to fine-tune models and can use this code implementation for future competitions.