For each `repertoire_id` across all test datasets, the participants has to return a probability for the repertoire being label-positive. In addition, a ranked list of the top 50,000 unique rows (including junction_aa, v_call, and j_call) that best contribute to the optimal classification for each training dataset has to be returned, regardless of the data encoding used. Note that these label-associated sequences have to be sorted based on some form of importance scores from most important to less important; we may use only top-n sequences from the ordered list of 50k sequences for evaluation. These will be used to compute the performance metrics [**area under the ROC curve**](http://en.wikipedia.org/wiki/Receiver_operating_characteristic) and [**Jaccard similarity**](https://en.wikipedia.org/wiki/Jaccard_index), respectively, for each of the datasets. A **weighted average** of both measures across all the included datasets will be used as the basis for ranking on the leaderboard for the competition. 

![](https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F14003908%2F13d3d8ca174bb1cb8efd2bb9df671813%2Fevaluation_illustration_for_kaggle_page.png?generation=1756380205170091&alt=media)

## Submission File

There are a total of 4213 repertoires across all test datasets. The submission file should contain a total of **404213** rows (4213 repertoire ids across test datasets and their predicted probabilities plus 50,000 rank-ordered rows per each of the training datasets with junction_aa, v_call, and j_call).

The submission file should look like this (row-index is not needed, shown only for information):

![](https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F14003908%2F6c942487fa34b4f28c0bb7b1ca540e6f%2Fsubmissions_file_example.png?generation=1756381500552522&alt=media)

**Note:** A sample file of submissions (sample_submissions.csv) is provided under `Data` tab. As you can notice from the above image and the sample submissions file, where relevant, missing values in the submission file are to be represented by a special float '-999.0'. This is strictly needed as Kaggle's validation does not allow missing values such as 'NaN' in submission files.

Given that the prediction probabilities across all the test datasets are in `df1 `(4213 rows), and the list of all rank-ordered label-associated sequences are in `df2` (400,000 rows), we assume you would combine both in a fashion similar to: `combined_df = pd.concat([df1, df2], axis=0, ignore_index=True)`