{
  "id": 566530,
  "title": "Silver Solution [59th]",
  "content": "<p>I'll describe it later, it's a late night for me.</p>\n<p>Mostly feature engineering, bunch of models with different targets and stable validation.<br>\nHere is my code:<br>\n<a href=\"https://www.kaggle.com/code/viktoriaskorik/silver-solution-59th\" target=\"_blank\">https://www.kaggle.com/code/viktoriaskorik/silver-solution-59th</a></p>\n<h1>UPD.</h1>\n<h2>I want to thank people behind this notebooks:</h2>\n<ul>\n<li>I used <a href=\"https://www.kaggle.com/code/albansteff/refactoring-nn-pairwise-ranking-loss\" target=\"_blank\">Pairwise NN </a> in my solution.</li>\n<li>My best model has Quantile Target that I saw <a href=\"https://www.kaggle.com/code/ambrosm/esp-eda-which-makes-sense\" target=\"_blank\">here</a></li>\n<li>I used Nelson target that I saw <a href=\"https://www.kaggle.com/code/tanujtaneja/single-model-xgboost-nelson-cv-681-lb-685\" target=\"_blank\">here</a></li>\n</ul>\n<h1>Validation</h1>\n<p>I used Stratified Kfold with 10 folds, each fold has own random_seed.<br>\nHere isthe code for my stratification column:</p>\n<pre><code>stratify_labels = (\n        data.race_group.astype() +\n        (data.age_at_hct == ).astype() +\n        data.efs.astype()\n    )\n</code></pre>\n<p>Also I didn't leak val data while target creation. You can read about this <a href=\"https://www.kaggle.com/competitions/equity-post-HCT-survival-predictions/discussion/566702\" target=\"_blank\">here</a>.</p>\n<h1>Feature Engineering</h1>\n<p>I did a lot of FE. For the most part, it was target encoding. There is some standard techniques as Target Mean/Median/STD/Min/Max, but also some custom encoding (they were the most helpful).</p>\n<h3>KDE-based features</h3>\n<p>Kernel Density Estimation features allows capturing density distributions of both event and censored observations.<br>\nThere is multiple KDE related features:</p>\n<ul>\n<li>Density Estimates: Calculated probability distributions for event times and censored times</li>\n<li>Density Ratio: Ratio between event and censoring densities to identify high-risk time regions</li>\n<li>Crossing Time Indicator: Binary feature marking proximity to the median event time</li>\n</ul>\n<p>Each of this features was created relatevely to given group_col and then for each feature was calculated group statistic (Aggregated KDE features (mean, std, min, max) across categorical groups)</p>\n<pre><code> ():\n    \n    event_times = train_df[train_df[event_col] == ][time_col]\n    censor_times = train_df[train_df[event_col] == ][time_col]\n\n     (event_times) &gt;   (censor_times) &gt; :  \n        event_kde = stats.gaussian_kde(event_times)\n        censor_kde = stats.gaussian_kde(censor_times)\n\n        train_df[] = event_kde(train_df[time_col])\n        train_df[] = censor_kde(train_df[time_col])\n\n        \n        train_df[] = train_df[] / (train_df[] + )\n    :\n        train_df[] = \n        train_df[] = \n        train_df[] = \n\n    \n      event_times.empty:\n        crossing_time = train_df.iloc[(train_df[time_col] - event_times.median()).().argmin()][time_col]\n        train_df[] = (np.(train_df[time_col] - crossing_time) &lt; ).astype()\n    :\n        train_df[] = \n\n    \n     group_cols:\n         group_col  group_cols:\n            group_stats = (\n                train_df\n                .groupby(group_col)[[, , , ]]\n                .agg([, , , ])\n            )\n            group_stats.columns = [  feat, stat  group_stats.columns]\n\n            \n            train_df = train_df.merge(group_stats, on=group_col, how=)\n            val_df = val_df.merge(group_stats, on=group_col, how=)\n            test_df = test_df.merge(group_stats, on=group_col, how=)\n\n     train_df, val_df, test_df\n\n\n ():\n    \n    event_times = train_df[train_df[event_col] == ][time_col]\n    censor_times = train_df[train_df[event_col] == ][time_col]\n\n     (event_times) &gt;   (censor_times) &gt; :  \n        event_kde = stats.gaussian_kde(event_times)\n        censor_kde = stats.gaussian_kde(censor_times)\n\n        train_df[] = event_kde(train_df[time_col])\n        train_df[] = censor_kde(train_df[time_col])\n\n        \n        train_df[] = train_df[] / (train_df[] + )\n    :\n        train_df[] = \n        train_df[] = \n        train_df[] = \n\n    \n      event_times.empty:\n        crossing_time = train_df.iloc[(train_df[time_col] - event_times.median()).().argmin()][time_col]\n        train_df[] = (np.(train_df[time_col] - crossing_time) &lt; ).astype()\n    :\n        train_df[] = \n\n    \n     group_cols:\n         group_col  group_cols:\n            group_stats = (\n                train_df[cols2calc + [, , , ]]\n                .groupby(group_col)[[, , , ]]\n                .agg([, , , ])\n            )\n            group_stats.columns = [  feat, stat  group_stats.columns]\n\n            \n            train_df = train_df.merge(group_stats, on=group_col, how=)\n            val_df = val_df.merge(group_stats, on=group_col, how=)\n            test_df = test_df.merge(group_stats, on=group_col, how=)\n\n     train_df, val_df, test_df\n</code></pre>\n<h3>Survival propobility rank features</h3>\n<p>This feature engineering method transforms categorical variables into numerical features based on their survival outcomes using Kaplan-Meier estimators. The approach ranks categorical variables according to their association with survival probability.</p>\n<ol>\n<li>Calculates non-parametric survival estimates for each time point</li>\n<li>Computes survival probability (Kaplan-Meier)</li>\n<li>Category Ranking:<br>\n-- For each categorical variable, calculates survival curves per category<br>\n-- Ranks categories based on final survival probability<br>\nHigher ranks indicate better survival outcomes</li>\n</ol>\n<pre><code> ():\n    survival_df = df[[, ]].groupby().agg([, ]).droplevel(, axis=).astype()\n    survival_df[] = survival_df[].() - survival_df[].shift().fillna().cumsum().astype()\n    hazard = survival_df[] / survival_df[]\n    survival_df[] = np.cumsum(hazard)  \n    survival_df[] = ( - hazard).cumprod()  \n     survival_df\n\n ():\n    \n    \n    survival_probs = {}\n     category, group  train.groupby(column, observed=):\n        survival_df = survival_function(group)\n         survival_df  :\n             \n        final_prob = survival_df[].iloc[-]   survival_df.empty  np.nan\n        survival_probs[category] = final_prob\n\n    \n    ranking_df = pd.DataFrame.from_dict(\n        survival_probs,\n        orient=,\n        columns=[]\n    ).dropna()\n\n    \n    ranking_df[] = ranking_df[].rank(\n        ascending=,\n        method=\n    ).astype()\n\n     ranking_df[].to_dict()\n\n\n () -&gt; pd.DataFrame:\n    \n    ()\n    ref_copy = train_df.copy(deep=)\n     col  [\n        , , , , ,\n        , , , ,\n        , , , , ,\n        , , , , ,\n        ,  \n        , , , ,\n        , , , ,\n        , , , ,\n        , , , , , , ,\n        , , , , , ,\n        , , , , , ,\n        , , , , , , ,\n        , , , , , , ,\n    ]:\n         col  test_df:\n            ref_copy[col] = ref_copy[col].astype()\n\n            ranks_km = rank_categories_by_survival_kaplan_meier(ref_copy, col, survival_function)\n            train_df[] = train_df[col].(ranks_km)\n            val_df[] = val_df[col].(ranks_km)\n            test_df[] = test_df[col].(ranks_km)\n               ranks_km:\n                train_df[] = train_df[col].fillna(ranks_km.get())\n                val_df[] = val_df[col].fillna(ranks_km.get())\n                test_df[] = test_df[col].fillna(ranks_km.get())\n     train_df, val_df, test_df\n</code></pre>\n<h1>Label Propagation</h1>\n<ol>\n<li>I used GNN for prediction of efs.</li>\n<li>I used this predictions to choose which data points to shift.</li>\n<li>I adjusted shift value for each model with Optuna.</li>\n</ol>",
  "comments": [
    {
      "id": 3141988,
      "author_name": "klog",
      "author_url": "",
      "post_date": "2025-03-06T02:45:30.053000",
      "content": "<p>Hi! Congratulations on winning the silver medal! 🎉<br>\nIf possible, could you share a Notebook explaining the feature selection method you mentioned? (Apologies if it's already included.)<a href=\"https://www.kaggle.com/competitions/equity-post-HCT-survival-predictions/discussion/564679#3132591\" target=\"_blank\">https://www.kaggle.com/competitions/equity-post-HCT-survival-predictions/discussion/564679#3132591</a></p>\n<p>It seems that many participants, including myself, struggled with feature engineering, so I believe it would be incredibly helpful for learning!</p>",
      "votes": 1,
      "replies": [
        {
          "id": 3142840,
          "author_name": "Viktoria",
          "author_url": "",
          "post_date": "2025-03-06T17:48:19.807000",
          "content": "<p>Thank you!</p>\n<p>I apologize, perhaps I did not explain my approach clearly enough. It's very simple, I used forward and backward selection <a href=\"https://scikit-learn.org/stable/modules/feature_selection.html#sequential-feature-selection\" target=\"_blank\">methods</a>.</p>\n<ol>\n<li>I started with forward selection.</li>\n<li>Then I'd tuned hp with selected features.</li>\n<li>Then I'd done backward selection.</li>\n</ol>\n<p>I only do this with features that I created. I keep original features in training data.</p>",
          "votes": 1,
          "replies": [
            {
              "id": 3143087,
              "author_name": "klog",
              "author_url": "",
              "post_date": "2025-03-06T23:31:41.533000",
              "content": "<p>Thanks for your reply! I’m not so familiar with it, but when adding features in forward selection, are they added in a greedy manner to improve the score?</p>",
              "votes": 0,
              "replies": []
            },
            {
              "id": 3143099,
              "author_name": "Viktoria",
              "author_url": "",
              "post_date": "2025-03-06T23:49:59.370000",
              "content": "<p>Yep, in greedy manner.</p>",
              "votes": 1,
              "replies": []
            }
          ]
        }
      ]
    },
    {
      "id": 3141920,
      "author_name": "Sheikh Muhammad Abdullah",
      "author_url": "",
      "post_date": "2025-03-06T01:28:22.377000",
      "content": "<p>Congratulations <a href=\"https://www.kaggle.com/viktoriaskorik\" target=\"_blank\">@viktoriaskorik</a> That's good.</p>",
      "votes": 1,
      "replies": [
        {
          "id": 3142664,
          "author_name": "Viktoria",
          "author_url": "",
          "post_date": "2025-03-06T15:21:59.303000",
          "content": "<p>Thank you!</p>",
          "votes": 0,
          "replies": []
        }
      ]
    },
    {
      "id": 3142867,
      "author_name": "Viktoria",
      "author_url": "",
      "post_date": "2025-03-06T18:10:05.130000",
      "content": "<p>For those who are interested, I wrote a wrap-up of my solution.</p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 3141873,
      "author_name": "Viktoria",
      "author_url": "",
      "post_date": "2025-03-06T00:53:15.753000",
      "content": "<p>Also, I leave here my completely unsuccessful experiment with GNN.<br>\n<a href=\"https://www.kaggle.com/code/viktoriaskorik/my-first-ever-try-with-gnn\" target=\"_blank\">https://www.kaggle.com/code/viktoriaskorik/my-first-ever-try-with-gnn</a></p>\n<p>I'm looking forward to hearing from people who have used GNN and gotten good results, how they did it.</p>",
      "votes": 0,
      "replies": [
        {
          "id": 3142098,
          "author_name": "Tom",
          "author_url": "",
          "post_date": "2025-03-06T05:03:26.570000",
          "content": "<p><a href=\"https://www.kaggle.com/viktoriaskorik\" target=\"_blank\">@viktoriaskorik</a>  I have written  my <a href=\"https://www.kaggle.com/competitions/equity-post-HCT-survival-predictions/discussion/566544\" target=\"_blank\">solution</a>. You can check it. </p>",
          "votes": 1,
          "replies": [
            {
              "id": 3142680,
              "author_name": "Viktoria",
              "author_url": "",
              "post_date": "2025-03-06T15:42:42.967000",
              "content": "<p>Thank you, this is trulery interesting approach!</p>",
              "votes": 0,
              "replies": []
            }
          ]
        }
      ]
    }
  ],
  "comment_count": 10,
  "messages": [
    {
      "id": 3141843,
      "postDate": "2025-03-06T00:24:35.900Z",
      "content": "<p>I'll describe it later, it's a late night for me.</p>\n<p>Mostly feature engineering, bunch of models with different targets and stable validation.<br>\nHere is my code:<br>\n<a href=\"https://www.kaggle.com/code/viktoriaskorik/silver-solution-59th\" target=\"_blank\">https://www.kaggle.com/code/viktoriaskorik/silver-solution-59th</a></p>\n<h1>UPD.</h1>\n<h2>I want to thank people behind this notebooks:</h2>\n<ul>\n<li>I used <a href=\"https://www.kaggle.com/code/albansteff/refactoring-nn-pairwise-ranking-loss\" target=\"_blank\">Pairwise NN </a> in my solution.</li>\n<li>My best model has Quantile Target that I saw <a href=\"https://www.kaggle.com/code/ambrosm/esp-eda-which-makes-sense\" target=\"_blank\">here</a></li>\n<li>I used Nelson target that I saw <a href=\"https://www.kaggle.com/code/tanujtaneja/single-model-xgboost-nelson-cv-681-lb-685\" target=\"_blank\">here</a></li>\n</ul>\n<h1>Validation</h1>\n<p>I used Stratified Kfold with 10 folds, each fold has own random_seed.<br>\nHere isthe code for my stratification column:</p>\n<pre><code>stratify_labels = (\n        data.race_group.astype() +\n        (data.age_at_hct == ).astype() +\n        data.efs.astype()\n    )\n</code></pre>\n<p>Also I didn't leak val data while target creation. You can read about this <a href=\"https://www.kaggle.com/competitions/equity-post-HCT-survival-predictions/discussion/566702\" target=\"_blank\">here</a>.</p>\n<h1>Feature Engineering</h1>\n<p>I did a lot of FE. For the most part, it was target encoding. There is some standard techniques as Target Mean/Median/STD/Min/Max, but also some custom encoding (they were the most helpful).</p>\n<h3>KDE-based features</h3>\n<p>Kernel Density Estimation features allows capturing density distributions of both event and censored observations.<br>\nThere is multiple KDE related features:</p>\n<ul>\n<li>Density Estimates: Calculated probability distributions for event times and censored times</li>\n<li>Density Ratio: Ratio between event and censoring densities to identify high-risk time regions</li>\n<li>Crossing Time Indicator: Binary feature marking proximity to the median event time</li>\n</ul>\n<p>Each of this features was created relatevely to given group_col and then for each feature was calculated group statistic (Aggregated KDE features (mean, std, min, max) across categorical groups)</p>\n<pre><code> ():\n    \n    event_times = train_df[train_df[event_col] == ][time_col]\n    censor_times = train_df[train_df[event_col] == ][time_col]\n\n     (event_times) &gt;   (censor_times) &gt; :  \n        event_kde = stats.gaussian_kde(event_times)\n        censor_kde = stats.gaussian_kde(censor_times)\n\n        train_df[] = event_kde(train_df[time_col])\n        train_df[] = censor_kde(train_df[time_col])\n\n        \n        train_df[] = train_df[] / (train_df[] + )\n    :\n        train_df[] = \n        train_df[] = \n        train_df[] = \n\n    \n      event_times.empty:\n        crossing_time = train_df.iloc[(train_df[time_col] - event_times.median()).().argmin()][time_col]\n        train_df[] = (np.(train_df[time_col] - crossing_time) &lt; ).astype()\n    :\n        train_df[] = \n\n    \n     group_cols:\n         group_col  group_cols:\n            group_stats = (\n                train_df\n                .groupby(group_col)[[, , , ]]\n                .agg([, , , ])\n            )\n            group_stats.columns = [  feat, stat  group_stats.columns]\n\n            \n            train_df = train_df.merge(group_stats, on=group_col, how=)\n            val_df = val_df.merge(group_stats, on=group_col, how=)\n            test_df = test_df.merge(group_stats, on=group_col, how=)\n\n     train_df, val_df, test_df\n\n\n ():\n    \n    event_times = train_df[train_df[event_col] == ][time_col]\n    censor_times = train_df[train_df[event_col] == ][time_col]\n\n     (event_times) &gt;   (censor_times) &gt; :  \n        event_kde = stats.gaussian_kde(event_times)\n        censor_kde = stats.gaussian_kde(censor_times)\n\n        train_df[] = event_kde(train_df[time_col])\n        train_df[] = censor_kde(train_df[time_col])\n\n        \n        train_df[] = train_df[] / (train_df[] + )\n    :\n        train_df[] = \n        train_df[] = \n        train_df[] = \n\n    \n      event_times.empty:\n        crossing_time = train_df.iloc[(train_df[time_col] - event_times.median()).().argmin()][time_col]\n        train_df[] = (np.(train_df[time_col] - crossing_time) &lt; ).astype()\n    :\n        train_df[] = \n\n    \n     group_cols:\n         group_col  group_cols:\n            group_stats = (\n                train_df[cols2calc + [, , , ]]\n                .groupby(group_col)[[, , , ]]\n                .agg([, , , ])\n            )\n            group_stats.columns = [  feat, stat  group_stats.columns]\n\n            \n            train_df = train_df.merge(group_stats, on=group_col, how=)\n            val_df = val_df.merge(group_stats, on=group_col, how=)\n            test_df = test_df.merge(group_stats, on=group_col, how=)\n\n     train_df, val_df, test_df\n</code></pre>\n<h3>Survival propobility rank features</h3>\n<p>This feature engineering method transforms categorical variables into numerical features based on their survival outcomes using Kaplan-Meier estimators. The approach ranks categorical variables according to their association with survival probability.</p>\n<ol>\n<li>Calculates non-parametric survival estimates for each time point</li>\n<li>Computes survival probability (Kaplan-Meier)</li>\n<li>Category Ranking:<br>\n-- For each categorical variable, calculates survival curves per category<br>\n-- Ranks categories based on final survival probability<br>\nHigher ranks indicate better survival outcomes</li>\n</ol>\n<pre><code> ():\n    survival_df = df[[, ]].groupby().agg([, ]).droplevel(, axis=).astype()\n    survival_df[] = survival_df[].() - survival_df[].shift().fillna().cumsum().astype()\n    hazard = survival_df[] / survival_df[]\n    survival_df[] = np.cumsum(hazard)  \n    survival_df[] = ( - hazard).cumprod()  \n     survival_df\n\n ():\n    \n    \n    survival_probs = {}\n     category, group  train.groupby(column, observed=):\n        survival_df = survival_function(group)\n         survival_df  :\n             \n        final_prob = survival_df[].iloc[-]   survival_df.empty  np.nan\n        survival_probs[category] = final_prob\n\n    \n    ranking_df = pd.DataFrame.from_dict(\n        survival_probs,\n        orient=,\n        columns=[]\n    ).dropna()\n\n    \n    ranking_df[] = ranking_df[].rank(\n        ascending=,\n        method=\n    ).astype()\n\n     ranking_df[].to_dict()\n\n\n () -&gt; pd.DataFrame:\n    \n    ()\n    ref_copy = train_df.copy(deep=)\n     col  [\n        , , , , ,\n        , , , ,\n        , , , , ,\n        , , , , ,\n        ,  \n        , , , ,\n        , , , ,\n        , , , ,\n        , , , , , , ,\n        , , , , , ,\n        , , , , , ,\n        , , , , , , ,\n        , , , , , , ,\n    ]:\n         col  test_df:\n            ref_copy[col] = ref_copy[col].astype()\n\n            ranks_km = rank_categories_by_survival_kaplan_meier(ref_copy, col, survival_function)\n            train_df[] = train_df[col].(ranks_km)\n            val_df[] = val_df[col].(ranks_km)\n            test_df[] = test_df[col].(ranks_km)\n               ranks_km:\n                train_df[] = train_df[col].fillna(ranks_km.get())\n                val_df[] = val_df[col].fillna(ranks_km.get())\n                test_df[] = test_df[col].fillna(ranks_km.get())\n     train_df, val_df, test_df\n</code></pre>\n<h1>Label Propagation</h1>\n<ol>\n<li>I used GNN for prediction of efs.</li>\n<li>I used this predictions to choose which data points to shift.</li>\n<li>I adjusted shift value for each model with Optuna.</li>\n</ol>",
      "rawMarkdown": "I'll describe it later, it's a late night for me.\n\nMostly feature engineering, bunch of models with different targets and stable validation.\nHere is my code:\nhttps://www.kaggle.com/code/viktoriaskorik/silver-solution-59th\n\n# UPD.\n\n## I want to thank people behind this notebooks:\n- I used [Pairwise NN ](https://www.kaggle.com/code/albansteff/refactoring-nn-pairwise-ranking-loss) in my solution.\n- My best model has Quantile Target that I saw [here](https://www.kaggle.com/code/ambrosm/esp-eda-which-makes-sense)\n- I used Nelson target that I saw [here](https://www.kaggle.com/code/tanujtaneja/single-model-xgboost-nelson-cv-681-lb-685)\n\n# Validation\nI used Stratified Kfold with 10 folds, each fold has own random_seed.\nHere isthe code for my stratification column:\n```python\nstratify_labels = (\n        data.race_group.astype(str) +\n        (data.age_at_hct == 0.044).astype(str) +\n        data.efs.astype(str)\n    )\n```\n\nAlso I didn't leak val data while target creation. You can read about this [here](https://www.kaggle.com/competitions/equity-post-HCT-survival-predictions/discussion/566702).\n\n# Feature Engineering\nI did a lot of FE. For the most part, it was target encoding. There is some standard techniques as Target Mean/Median/STD/Min/Max, but also some custom encoding (they were the most helpful).\n### KDE-based features\n\nKernel Density Estimation features allows capturing density distributions of both event and censored observations.\nThere is multiple KDE related features:\n- Density Estimates: Calculated probability distributions for event times and censored times\n- Density Ratio: Ratio between event and censoring densities to identify high-risk time regions\n- Crossing Time Indicator: Binary feature marking proximity to the median event time\n\nEach of this features was created relatevely to given group_col and then for each feature was calculated group statistic (Aggregated KDE features (mean, std, min, max) across categorical groups)\n\n```python\ndef compute_kde_features(train_df, val_df, test_df, time_col='efs_time', event_col='efs', group_cols=None):\n    # Compute KDE for event and censor times\n    event_times = train_df[train_df[event_col] == 1][time_col]\n    censor_times = train_df[train_df[event_col] == 0][time_col]\n\n    if len(event_times) > 1 and len(censor_times) > 1:  # Avoid KDE errors with too few points\n        event_kde = stats.gaussian_kde(event_times)\n        censor_kde = stats.gaussian_kde(censor_times)\n\n        train_df['event_density'] = event_kde(train_df[time_col])\n        train_df['censor_density'] = censor_kde(train_df[time_col])\n\n        # Density ratio: event_density / (censor_density + small epsilon)\n        train_df['density_ratio'] = train_df['event_density'] / (train_df['censor_density'] + 1e-6)\n    else:\n        train_df['event_density'] = 0\n        train_df['censor_density'] = 0\n        train_df['density_ratio'] = 0\n\n    # Identify key crossing time\n    if not event_times.empty:\n        crossing_time = train_df.iloc[(train_df[time_col] - event_times.median()).abs().argmin()][time_col]\n        train_df['crossing_time_indicator'] = (np.abs(train_df[time_col] - crossing_time) < 1).astype(int)\n    else:\n        train_df['crossing_time_indicator'] = 0\n\n    # Group-based statistics (instead of mean target encoding)\n    if group_cols:\n        for group_col in group_cols:\n            group_stats = (\n                train_df\n                .groupby(group_col)[['event_density', 'censor_density', 'density_ratio', 'crossing_time_indicator']]\n                .agg(['mean', 'std', 'min', 'max'])\n            )\n            group_stats.columns = [f\"kde_features_{group_col}_{feat}_{stat}\" for feat, stat in group_stats.columns]\n\n            # Merge with test_df\n            train_df = train_df.merge(group_stats, on=group_col, how=\"left\")\n            val_df = val_df.merge(group_stats, on=group_col, how=\"left\")\n            test_df = test_df.merge(group_stats, on=group_col, how=\"left\")\n\n    return train_df, val_df, test_df\n\n\ndef compute_kde_features(train_df, val_df, test_df, time_col='efs_time', event_col='efs', cols2calc=[], group_cols=None):\n    # Compute KDE for event and censor times\n    event_times = train_df[train_df[event_col] == 1][time_col]\n    censor_times = train_df[train_df[event_col] == 0][time_col]\n\n    if len(event_times) > 1 and len(censor_times) > 1:  # Avoid KDE errors with too few points\n        event_kde = stats.gaussian_kde(event_times)\n        censor_kde = stats.gaussian_kde(censor_times)\n\n        train_df['event_density'] = event_kde(train_df[time_col])\n        train_df['censor_density'] = censor_kde(train_df[time_col])\n\n        # Density ratio: event_density / (censor_density + small epsilon)\n        train_df['density_ratio'] = train_df['event_density'] / (train_df['censor_density'] + 1e-6)\n    else:\n        train_df['event_density'] = 0\n        train_df['censor_density'] = 0\n        train_df['density_ratio'] = 0\n\n    # Identify key crossing time\n    if not event_times.empty:\n        crossing_time = train_df.iloc[(train_df[time_col] - event_times.median()).abs().argmin()][time_col]\n        train_df['crossing_time_indicator'] = (np.abs(train_df[time_col] - crossing_time) < 1).astype(int)\n    else:\n        train_df['crossing_time_indicator'] = 0\n\n    # Group-based statistics (instead of mean target encoding)\n    if group_cols:\n        for group_col in group_cols:\n            group_stats = (\n                train_df[cols2calc + ['event_density', 'density_ratio', 'censor_density', 'crossing_time_indicator']]\n                .groupby(group_col)[['event_density', 'censor_density', 'density_ratio', 'crossing_time_indicator']]\n                .agg(['mean', 'std', 'min', 'max'])\n            )\n            group_stats.columns = [f\"kde_features_{group_col}_{feat}_{stat}\" for feat, stat in group_stats.columns]\n\n            # Merge with test_df\n            train_df = train_df.merge(group_stats, on=group_col, how=\"left\")\n            val_df = val_df.merge(group_stats, on=group_col, how=\"left\")\n            test_df = test_df.merge(group_stats, on=group_col, how=\"left\")\n\n    return train_df, val_df, test_df\n```\n### Survival propobility rank features\n\nThis feature engineering method transforms categorical variables into numerical features based on their survival outcomes using Kaplan-Meier estimators. The approach ranks categorical variables according to their association with survival probability.\n\n\n1. Calculates non-parametric survival estimates for each time point\n2. Computes survival probability (Kaplan-Meier)\n3. Category Ranking:\n-- For each categorical variable, calculates survival curves per category\n-- Ranks categories based on final survival probability\nHigher ranks indicate better survival outcomes\n\n```python\ndef survival_function(df):\n    survival_df = df[['efs', 'efs_time']].groupby('efs_time').agg(['size', 'sum']).droplevel(0, axis=1).astype(int)\n    survival_df['n_at_risk'] = survival_df['size'].sum() - survival_df['size'].shift().fillna(0).cumsum().astype(int)\n    hazard = survival_df['sum'] / survival_df['n_at_risk']\n    survival_df['cumulative_hazard'] = np.cumsum(hazard)  # nelson_alen_estimator\n    survival_df['survival_probability'] = (1 - hazard).cumprod()  # kaplan_meier_estimator\n    return survival_df\n    \ndef rank_categories_by_survival_kaplan_meier(train: pd.DataFrame, column: str, survival_function):\n    \"\"\"Rank categories based on final survival probability using vectorized operations.\n\n    Args:\n        train: DataFrame containing survival data\n        column: Categorical column to rank\n\n    Returns:\n        Dictionary mapping {category: survival_rank}\n    \"\"\"\n    # Calculate survival probabilities for each category\n    survival_probs = {}\n    for category, group in train.groupby(column, observed=True):\n        survival_df = survival_function(group)\n        if survival_df is None:\n            return None\n        final_prob = survival_df['survival_probability'].iloc[-1] if not survival_df.empty else np.nan\n        survival_probs[category] = final_prob\n\n    # Create ranking DataFrame\n    ranking_df = pd.DataFrame.from_dict(\n        survival_probs,\n        orient='index',\n        columns=['final_survival_probability']\n    ).dropna()\n\n    # Calculate ranks (higher probability = better rank)\n    ranking_df['rank'] = ranking_df['final_survival_probability'].rank(\n        ascending=False,\n        method='min'\n    ).astype(int)\n\n    return ranking_df['rank'].to_dict()\n    \n\ndef add_survival_features(train_df, val_df, test_df) -> pd.DataFrame:\n    \"\"\"Add survival-related features using training data as reference.\"\"\"\n    print(\"Adding survival features...\")\n    ref_copy = train_df.copy(deep=True)\n    for col in [\n        'dri_score', 'sex_match', 'cyto_score', 'tbi_status', 'graft_type',\n        'prim_disease_hct', 'cmv_status', 'tce_imm_match', 'cyto_score_detail',\n        'conditioning_intensity', 'ethnicity', 'year_hct', 'mrd_hct', 'in_vivo_tcd',\n        'tce_match', 'gvhd_proph', 'race_group', 'tce_div_match', 'donor_related',\n        'melphalan_dose',  # 'frst_sex_match', 'frst_tce_imm_match', 'frst_cmv_status',\n        'prim_disease_group', 'gvhd_proph_group', 'comorbidity_score', 'age_bin',\n        'race_group|sex_match', 'race_group|cmv_status', 'race_group|tce_imm_match', 'race_group|age_bin',\n        'race_group|frst_sex_match', 'race_group|frst_tce_imm_match', 'race_group|frst_cmv_status', 'psych_disturb',\n        'diabetes', 'hla_match_c_high', 'hla_high_res_8', 'arrhythmia', 'hla_low_res_6', 'vent_hist', 'renal_issue',\n        'pulm_severe', 'hla_high_res_6', 'hla_high_res_10', 'hla_match_dqb1_high', 'hla_nmdp_6', 'hla_match_c_low',\n        'rituximab', 'hla_match_drb1_low', 'hla_match_dqb1_low', 'prod_type', 'obesity', 'hla_match_a_high',\n        'hepatic_severe', 'prior_tumor', 'hla_match_b_low', 'peptic_ulcer', 'hla_match_a_low', 'rheum_issue', 'hla_match_b_high',\n        'karnofsky_score', 'hepatic_mild', 'hla_low_res_8', 'cardiac', 'hla_match_drb1_high', 'pulm_moderate', 'hla_low_res_10',\n    ]:\n        if col in test_df:\n            ref_copy[col] = ref_copy[col].astype(str)\n    \n            ranks_km = rank_categories_by_survival_kaplan_meier(ref_copy, col, survival_function)\n            train_df[f'{col}_survival_rank'] = train_df[col].map(ranks_km)\n            val_df[f'{col}_survival_rank'] = val_df[col].map(ranks_km)\n            test_df[f'{col}_survival_rank'] = test_df[col].map(ranks_km)\n            if 'nan' in ranks_km:\n                train_df[f'{col}_survival_rank'] = train_df[col].fillna(ranks_km.get('nan'))\n                val_df[f'{col}_survival_rank'] = val_df[col].fillna(ranks_km.get('nan'))\n                test_df[f'{col}_survival_rank'] = test_df[col].fillna(ranks_km.get('nan'))\n    return train_df, val_df, test_df\n```\n\n# Label Propagation\n1. I used GNN for prediction of efs.\n2. I used this predictions to choose which data points to shift.\n3. I adjusted shift value for each model with Optuna.",
      "votes": 13
    },
    {
      "id": 3141988,
      "postDate": "2025-03-06T02:45:30.053Z",
      "content": "<p>Hi! Congratulations on winning the silver medal! 🎉<br>\nIf possible, could you share a Notebook explaining the feature selection method you mentioned? (Apologies if it's already included.)<a href=\"https://www.kaggle.com/competitions/equity-post-HCT-survival-predictions/discussion/564679#3132591\" target=\"_blank\">https://www.kaggle.com/competitions/equity-post-HCT-survival-predictions/discussion/564679#3132591</a></p>\n<p>It seems that many participants, including myself, struggled with feature engineering, so I believe it would be incredibly helpful for learning!</p>",
      "rawMarkdown": "Hi! Congratulations on winning the silver medal! 🎉\nIf possible, could you share a Notebook explaining the feature selection method you mentioned? (Apologies if it's already included.)https://www.kaggle.com/competitions/equity-post-HCT-survival-predictions/discussion/564679#3132591\n\n It seems that many participants, including myself, struggled with feature engineering, so I believe it would be incredibly helpful for learning!",
      "votes": 1,
      "replies": [
        {
          "id": 3142840,
          "postDate": "2025-03-06T17:48:19.807Z",
          "content": "<p>Thank you!</p>\n<p>I apologize, perhaps I did not explain my approach clearly enough. It's very simple, I used forward and backward selection <a href=\"https://scikit-learn.org/stable/modules/feature_selection.html#sequential-feature-selection\" target=\"_blank\">methods</a>.</p>\n<ol>\n<li>I started with forward selection.</li>\n<li>Then I'd tuned hp with selected features.</li>\n<li>Then I'd done backward selection.</li>\n</ol>\n<p>I only do this with features that I created. I keep original features in training data.</p>",
          "rawMarkdown": "Thank you!\n\nI apologize, perhaps I did not explain my approach clearly enough. It's very simple, I used forward and backward selection [methods](https://scikit-learn.org/stable/modules/feature_selection.html#sequential-feature-selection).\n\n1. I started with forward selection.\n2. Then I'd tuned hp with selected features.\n3. Then I'd done backward selection.\n\nI only do this with features that I created. I keep original features in training data.",
          "votes": 1,
          "replies": [
            {
              "id": 3143087,
              "postDate": "2025-03-06T23:31:41.533Z",
              "content": "<p>Thanks for your reply! I’m not so familiar with it, but when adding features in forward selection, are they added in a greedy manner to improve the score?</p>",
              "rawMarkdown": "Thanks for your reply! I’m not so familiar with it, but when adding features in forward selection, are they added in a greedy manner to improve the score?"
            },
            {
              "id": 3143099,
              "postDate": "2025-03-06T23:49:59.370Z",
              "content": "<p>Yep, in greedy manner.</p>",
              "rawMarkdown": "Yep, in greedy manner.",
              "votes": 1
            }
          ]
        }
      ]
    },
    {
      "id": 3141920,
      "postDate": "2025-03-06T01:28:22.377Z",
      "content": "<p>Congratulations <a href=\"https://www.kaggle.com/viktoriaskorik\" target=\"_blank\">@viktoriaskorik</a> That's good.</p>",
      "rawMarkdown": "Congratulations @viktoriaskorik That's good.",
      "votes": 1,
      "replies": [
        {
          "id": 3142664,
          "postDate": "2025-03-06T15:21:59.303Z",
          "content": "<p>Thank you!</p>",
          "rawMarkdown": "Thank you!"
        }
      ]
    },
    {
      "id": 3142867,
      "postDate": "2025-03-06T18:10:05.130Z",
      "content": "<p>For those who are interested, I wrote a wrap-up of my solution.</p>",
      "rawMarkdown": "For those who are interested, I wrote a wrap-up of my solution."
    },
    {
      "id": 3141873,
      "postDate": "2025-03-06T00:53:15.753Z",
      "content": "<p>Also, I leave here my completely unsuccessful experiment with GNN.<br>\n<a href=\"https://www.kaggle.com/code/viktoriaskorik/my-first-ever-try-with-gnn\" target=\"_blank\">https://www.kaggle.com/code/viktoriaskorik/my-first-ever-try-with-gnn</a></p>\n<p>I'm looking forward to hearing from people who have used GNN and gotten good results, how they did it.</p>",
      "rawMarkdown": "Also, I leave here my completely unsuccessful experiment with GNN.\nhttps://www.kaggle.com/code/viktoriaskorik/my-first-ever-try-with-gnn\n\nI'm looking forward to hearing from people who have used GNN and gotten good results, how they did it.",
      "replies": [
        {
          "id": 3142098,
          "postDate": "2025-03-06T05:03:26.570Z",
          "content": "<p><a href=\"https://www.kaggle.com/viktoriaskorik\" target=\"_blank\">@viktoriaskorik</a>  I have written  my <a href=\"https://www.kaggle.com/competitions/equity-post-HCT-survival-predictions/discussion/566544\" target=\"_blank\">solution</a>. You can check it. </p>",
          "rawMarkdown": "@viktoriaskorik  I have written  my [solution](https://www.kaggle.com/competitions/equity-post-HCT-survival-predictions/discussion/566544). You can check it. ",
          "votes": 1,
          "replies": [
            {
              "id": 3142680,
              "postDate": "2025-03-06T15:42:42.967Z",
              "content": "<p>Thank you, this is trulery interesting approach!</p>",
              "rawMarkdown": "Thank you, this is trulery interesting approach!"
            }
          ]
        }
      ]
    }
  ],
  "complete": true,
  "source": "live",
  "fetched_at": "2026-10-19T19:45:01Z",
  "competition_id": 70942,
  "competition_slug": "equity-post-HCT-survival-predictions",
  "team_id": 13149137,
  "solution_url": "/competitions/equity-post-HCT-survival-predictions/writeups/viktoria-skorik-silver-solution-59th"
}