The evaluation metric for this challenge is
[Mean F1-Score](https://www.kaggle.com/wiki/MeanFScore)
. The F1 score, commonly used in information retrieval, measures accuracy using the statistics precision p and recall r. Precision is the ratio of true positives (tp) to all predicted positives (tp + fp). Recall is the ratio of true positives to all actual positives (tp + fn). The F1 score is given by:
\[ F1 = 2\frac{p \cdot r}{p+r}\ \ \mathrm{where}\ \ p = \frac{tp}{tp+fp},\ \ r = \frac{tp}{tp+fn} \]
The F1 metric weights recall and precision equally, and a good retrieval algorithm will maximize both precision and recall simultaneously. Thus, moderately good performance on both will be favored over extremely good performance on one and poor performance on the other.
50% of the test images will be used for public leaderboard performance calculation and the remaining 50% will be used for private leaderboard. Public scores will be shown on the public leaderboard, while only the private scores will be used for final evaluation. The private leaderboard score is not shown until the end of the competititon. You can hand-select any 2 of the eligible submissions for private leaderboard score, or will otherwise default to the best public scoring submissions.
Submission Format
Submission files should be in .csv format and contain two columns: filename and class. ‘filename’ should be in string format and will contain the image filename (1.jpg, 2. jpg, etc.). The ‘class’ will also be in string format and contain the corresponding class name. It can take two values: ‘cracked’ or ‘uncracked’. Since there are 2000 images in the test dataset, your submission should have 2000 rows (excluding the header), each with 2 columns. . A sample submission file is also provided in the dataset.
The file should contain a header and have the following format:
filename,class
1.jpg,cracked