Submissions are evaluated according to the Mean Average Precision @ 25 (MAP@25):

$$\text{MAP@25} = \frac{1}{U} \sum\_{u=1}^{U} \sum\_{k=1}^{min(n,25)} P(k) \times rel(k)$$

where \\( U \\)  is the number of observations, \\( P(k) \\) is the precision at cutoff \\( k \\), \\( n \\) is the number predictions submitted per observation, and \\( rel(k) \\) is an indicator function equaling 1 if the item at rank \\( k \\) is a relevant (correct) label, zero otherwise. 

Once a correct label has been scored for *an observation*, that label is no longer considered relevant for that observation, and additional predictions of that label are skipped in the calculation. For example, if the correct label is `A` for an observation, the following predictions all score an average precision of `1.0`.

    [A, B, C, D, E]
    [A, A, A, A, A]
    [A, B, A, C, A]

There is only one correct label per observation (hence no divisor term in front of the inner summation.)

## Submission File
For each `QuestionId_Answer` row in the test set, you must predict the corresponding `MisconceptionId`. You can predict up to 25 `MisconceptionId` values per row, and these should be space-delimited. The file should contain a header and have the following format:

```
QuestionId_Answer,MisconceptionId
1869_A,1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25
1869_B,1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25
1869_C,1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25
1870_B,1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25
...
```