Submissions are evaluated on [Recall@3](https://ir-measur.es/en/latest/measures.html#r), since the dataset contains at most 3 relevant documents per query.

## Submission File
The submission file must contain two columns: `id` (taken from the column with the same name in the `test.csv` file) and `documents` (a JSON-encoded list of file paths). Documents will be treated as a set (i.e. their order in the list is irrelevant). See the `sample_submission.csv` file for an example.

If using Sage, you can simply run the [retrieve_kaggle.py](https://github.com/Storia-AI/sage/blob/main/benchmarks/retrieval/retrieve_kaggle.py) script to get a submission file in the right format.