### Special Notice

The computation resources used for the metric used in this competition is significantly higher than other Kaggle competitions. We would like to remind you to be patient for your submissions to be scored. Please understand that since this metric puts pressure on our evaluation server, only one submission per user is allowed per day. Please use your submissions wisely. 

## Evaluation Metric
In this competition, we evaluate seven different instance-level annotations, which are **car**, **motorcycle**, **bicycle**, **pedestrian**, **truck**, **bus**, and **tricycle**. The corresponding groups, such as car group and bicycle group, are annotated when boundaries cannot be distinguished by labelers. These groups are not evaluated currently.

We use the interpolated average precision (AP) as the metric for object segmentation [1]. The mean AP (mAP) is computed for all the video clips and all the classes for evaluation at different intersection-over-union (IoU) thresholds. The IoU between a predicted instance A and a ground truth instance B is computed by

 
$$
IoU(A,B) = \frac {A \cap B}  {A \cup B}.
$$
                                         

To obtain the Precision-Recall curve, we choose ten IoU thresholds in range [0.5, 1.0) with step 0.05. We match ground truth instances with predicted instances at different IoU thresholds. For example, given an IoU threshold 0.5, a predicted instance is considered as “matched” if the IoU with a ground truth instance is greater than 0.5. If there are multiple predicted instances matched to a ground truth instance, the predicted instance that is larger than the IoU threshold and has the largest confidence is considered as the true positive, and remaining predicted instances are false positives. The predicted instances that are not matched with any ground truth instances are counted as false positives. If IoU between a predicted instance and ignoring labels is larger than the IoU threshold, this predicted instance is removed from the evaluation. Notice that the group classes, such as car group and bicycle group, are also ignored in the evaluation.

 

Reference:

[1] Microsoft coco: Common objects in context, http://cocodataset.org/

## Submission File

---
**Important:**

* **Each row in submission must have all the required fields with the exact order.**
* **Each image should be encoded in the row-major order.**

---

The submission should be in Run Length Encoding format. Each submission line should represent one object instance, with the following columns:
`ImageId,LabelId,Confidence,PixelCount,EncodedPixels`
where 

- `ImageId` is the file name, 
- `LabelId` is the class of that object (car, person, etc), 
- `Confidence` is your confidence of the prediction, 
- `PixelCount` is the total number of pixels in that object (this is to help our evaluation execution speed), 
- `EncodedPixels` is Run-length encoded, with each pair delimited by `|`, for example: `1 3|10 5|` implies pixels `1,2,3,10,11,12,13,14` are to be included in the mask. The pixels are zero-indexed and numbered from left to right, then top to bottom.

The file should contain a header and have the following format:

    ImageId,LabelId,Confidence,PixelCount,EncodedPixels
    47fef111a243f92c3895510e6fef932b,33,0.7,8,1 3|10 5|
    47fef111a243f92c3895510e6fef932b,35,0.9,20,100 5|200 15|
    etc.