The evaluation metric for this competition is  FBeta-Score with Beta set to 1, which makes the score equal to the F1 score. The F1 score, commonly used in information retrieval, measures accuracy using the statistics precision p and recall r. Precision is the ratio of true positives (tp) to all predicted positives (tp + fp). Recall is the ratio of true positives to all actual positives (tp + fn). The F1 score is given by:
\[ F1 = 2\frac{p \cdot r}{p+r}\ \ \mathrm{where}\ \ p = \frac{tp}{tp+fp},\ \ r = \frac{tp}{tp+fn} \]
The F1 metric weights recall and precision equally, and a good retrieval algorithm will maximize both precision and recall simultaneously. Thus, moderately good performance on both will be favored over extremely good performance on one and poor performance on the other.
Since we are dealing with multi-class classification problem, we use `MeanFScore` metric of Kaggle, which is equivalent to `sklearn.metrics.f1_score(y_true, y_pred, average='micro')`. For more information, see [sklearn's documentation](https://scikit-learn.org/stable/modules/generated/sklearn.metrics.fbeta_score.html).
<\p>
Submission Format
For every time series sample in the testSet.json file
, submission files should contain two columns: 'Id' and 'Label'. The 'Label' is the class membership of each of the samples. The results are evaluated based on the 'Id' and 'Label' pairs. So, ordering the results by 'Id' is not necessary.
The file should contain a header and have the following format:
Id,Label
1,X