Submissions are evaluated using a language-balanced weighted F1 metric designed to reward both overall performance and robustness across languages.

The evaluation process proceeds as follows:

1. The test set is partitioned by language.
2. For each language, a weighted F1 score is computed across topic classes. The weighting follows the standard definition used in classification tasks, where each class F1 is weighted by its support within that language.
3. Two summary statistics are computed:

   * The mean of the per-language F1 scores.
   * The minimum (worst) per-language F1 score.
4. The final competition score is computed as a weighted combination of these two quantities.

Formally, let $$ ( F1_{\text{lang}} ) $$ denote the weighted F1 score for a given language. If there are ( L ) languages, then:

$$
F1_{\text{mean}} = \frac{1}{L} \sum_{\text{lang}} F1_{\text{lang}}
$$
$$
F1_{\text{worst}} = \min_{\text{lang}} F1_{\text{lang}}
$$

The final score is defined as:

$$
\text{Final Score} = 0.80 \times F1_{\text{mean}} + 0.20 \times F1_{\text{worst}}
$$

This formulation ensures that strong average performance alone is not sufficient. Models must also avoid collapsing on lower-resource languages. A system that performs extremely well on dominant languages but poorly on others will receive a reduced final score.

The public leaderboard reflects performance on the public portion of the test data during the competition. Final rankings are determined using the private portion of the test data after the competition closes.

Higher scores indicate better performance.