## Evaluation Criteria

The evaluation of prediction accuracy in the competition will involve a specialized metric known as the Stratified Concordance Index (*C-index*), adapted to consider different racial groups independently. This method allows us to gauge the predictive performance of models in a way that emphasizes equitability across diverse patient populations, particularly focusing on racial disparities in transplant outcomes.

### Concordance index

It represents the global assessment of the model discrimination power: this is the model’s ability to correctly provide a reliable ranking of the survival times based on the individual risk scores. It can be computed with the following formula:

![](https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F16861288%2F91fb00b64ff1919fbb3e76a54ad2672e%2Fc-index-formula.png?generation=1715363648236558&alt=media)

The concordance index is a value between 0 and 1 where:

- 0.5 is the expected result from random predictions,
- 1.0 is a perfect concordance and,
- 0.0 is perfect anti-concordance (multiply predictions with -1 to get 1.0)

### Stratified Concordance Index

For this competition, we adjust the standard C-index to account for racial stratification, thus ensuring that each racial group's outcomes are weighed equally in the model evaluation. The stratified c-index is calculated as the mean minus the standard deviation of the c-index scores calculated within the recipient race categories, i.e., the score will be better if the mean c-index over the different race categories is large and the standard deviation of the c-indices over the race categories is small. This value will range from 0 to 1, 1 is the theoretical perfect score, but this value will practically be lower due to censored outcomes.  

The submitted risk scores will be evaluated using the `score` function. This evaluation process involves comparing the submitted risk scores against actual observed values (i.e., survival times and event occurrences) from a test dataset. The function specifically calculates the stratified concordance index across different racial groups, ensuring that the predictions are not only accurate overall but also equitable across diverse patient demographics.

The implementation of the metric is wound in this [notebook](https://www.kaggle.com/code/metric/eefs-concordance-index).
----

## Submission File

Participants must submit their predictions for the test dataset as real-valued risk scores. These scores represent the model's assessment of each patient's risk following transplantation. A higher risk score typically indicates a higher likelihood of the target event occurrence.

The submission file must include a header and follow this format:

```
ID,prediction
28800,0.5
28801,1.2
28802,0.8
etc.
```

where:

`ID` refers to the identifier for each patient in the test dataset.
`prediction` is the corresponding risk score generated by your model.