The final test score is the mean of the hierarchical distances calculated across the test set, where the best average score is 0 and the worst is 12. Read below to see how the hierarchical distance is calculated. 

## Submission Format
Submission format should resemble the following:

| annotation_id | concept_name |
| --- | --- |
| 1 | Ophiuroidea |

Where an annotation_id is provided alongside a classification, or concept_name, for each ROI. 

Submissions can be any of the following taxonomic ranks: kingdom, phylum, class, order, family,  genus, and species. These ranks are part of a taxonomic tree, which organizes living things into a branching structure, starting with broad groups and narrowing down to specific species. A visual representation of this can be seen below for a specific animal Abyssocucumis abyssorum,

![](https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F23590972%2Fdc76332649aee257aaf5051e04cfd338%2Ftaxonomic_tree_example.png?generation=1738879903021487&alt=media)

Submissions will be evaluated on taxonomic distance from the ground truth answer. To do so we use the [FathomNet WoRMS Module](https://fathomnet-py.readthedocs.io/en/latest/api.html#module-fathomnet.api.worms) to populate the full taxonomic name and a reference library of these taxonomic names is stored in the evaluation script. This reference library only considered the 79 categories within the dataset. Each submission mush correspond to one of the original 79 categories to be considered. For example, if the submission concept name is Abyssocucumis abyssorum, the score function will populate a tree like what can be seen in the visual above.

This submission is compared with the ground truth taxonomic name, and a score is given corresponding to the distance between the submission and the ground truth answer along the taxonomic tree. Example scores and visual representations are provided below.

In the following example a score of one is given, and is represented visually via the bolded line. In this case, the submission is Asteronyx of genus rank while the ground truth is Asteronyx loveni, which is at the species rank. The distance between the rank of the ground truth and the submission is 1, so the resulting score is 1. 

![](https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F23590972%2Fd11b659dae9f99834b04f6aa0e4fea29%2Fsame%20line_example.png?generation=1738879926734632&alt=media)

In the following example a score of six is given, and visually this can be seen in the bolded line. In this case the submission taxon is Ophiacanthidae of family rank while the ground truth is Asteronyx loveni, which is of the species rank. The distance between the rank of the ground truth and the submission is 6, so the resulting score is 6. 

![](https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F23590972%2Fb1696618c501b7e779278e29cccb7b76%2Fdifferent_rank_example.png?generation=1738879941390101&alt=media)]

## Notes:
- Each submission must be one of the 79 categories that encompasses the dataset. If a submission name is not discoverable within the map of taxonomic names referenced in the evaluation metric, then a maximum score of 12 is assigned. This includes null submissions and misspellings.
- Some classifications have more complex taxonomic ranks such as suborders and infraclasses. For this task, we will only consider the 7 taxonomic rankings: kingdom, phylum, class, order, family, genus, and species.
- The ground truth categories vary in taxonomic rank.
- An important note on the test set: the images in the test set are on average smaller than those in the training. For more context on this please review [this](https://www.kaggle.com/competitions/fathomnet-2025/discussion/573371) discussion.