This competition uses Average Precision at k (AP@k) as the evaluation metric. It is calculated as:
\[AP@K = \frac{\sum_{i=1}^K P@i}{K} \]
where
\[ P@i = \frac{ \text{#relevant - forbidden ads retrieved in the first ith places} }{i} \]
Submission File
The submission file should contain a header row and be a single column of ad id's, giving 1,351,243 rows in total (including the header row). Ad ids are sorted with those ads most likely to contain illicit content at the top and those most likely to not contain illicit content at the bottom.
Id
385 <-- most likely to have illicit content
924691
85893
2719
93752
...
83543
3481
4838 <- least likely to have illicit content