Submissions are evaluated on [area under the ROC curve](http://en.wikipedia.org/wiki/Receiver_operating_characteristic) between the predicted probability and the observed target.

## Submission File Format

For each record ID in the test set, you must predict a binary value for the **is_anomaly** field. The file should contain a header and have the following format:

ID,is_anomaly?

1,0

2,0

3,0

4,0

5,0

6,0

7,0

8,0

9,0

10,0

...

## Understanding the evaluation metric
In addition to the reference to Wikipedia, you may use the following rules-of-thumb to understand the performance of your outlier detection model:

**AUC = 1**: a perfect outlier detection model. All anomalies and normal records are predicted correctly.

**0.5 < AUC < 1**: in most cases, your detection model would fall into this category. It means that your detection result is better than the random-guess, and is effective in practice.

**AUC = 0.5**: the detection result is roughly like random-guess, it is like flipping a coin to detect outliers.

**AUC < 0.5**: the detection model is worse than random-guess. Although you could flip the result of your model, i.e., report the anomalies detected by your model as normal and similarly for normal records in the submission file to get higher AUC, such behavior is forbidden because of two reasons:  a). you cannot explain why this is the case and b). your detection result, in essence, is worse than random-guess, which makes it inapplicable in practice. **We will check your final uploaded file and the generated file running your code to make sure such behavior does not happen**.

You may compute AUC offline when you implement your outlier detection model.