Submissions are evaluated by computing mean [Average Precision](https://en.wikipedia.org/wiki/Evaluation_measures_(information_retrieval)#Average_precision) (AP), modified to take into account the annotation process of [Open Images dataset](https://storage.googleapis.com/openimages/web/index.html) (mean is taken over per-class APs). The metric is described on the [Open Images Challenge website](https://storage.googleapis.com/openimages/web/object_detection_metric.html). 

The final mAP is computed as the average AP over the 500 classes. The participants will be ranked on this final metric.
 
Kaggle's production code in C# can be viewed [here](https://gist.github.com/wendykan/8b800e70251aa784b2f73283d1315ebd). The metric is also implemented as a part of [Tensorflow Object Detection API](https://github.com/tensorflow/models/tree/master/research/object_detection). See [this Tutorial](https://github.com/tensorflow/models/blob/master/research/object_detection/g3doc/challenge_evaluation.md) on running the evaluation in Python.

## Kernel Submissions
You can make submissions directly from Kaggle Kernels. By adding your teammates as collaborators on a kernel, you can share and edit code privately with them.

## Submission File
For each image in the test set, you must predict a list of boxes describing objects in the image. Each box is described as 

    ImageID,PredictionString
    ImageID,{Label Confidence XMin YMin XMax YMax},{...}