The first track of this competition focuses on accuracy of classification. Submissions for this track are evaluated using **multi-class logarithmic loss**. Each row in the dataset has been labeled with one true effectiveness label. For each row, you must submit the predicted probabilities that the product belongs to each quality label. The formula is:

$$ \text{log loss} = -\frac{1}{N}\sum\_{i=1}^N\sum\_{j=1}^My_{ij}\log(p\_{ij}), $$

where \\(N\\) is the number of rows in the test set, \\(M\\) is the number of class labels, \\( \text{log}\\) is the natural logarithm, \\(y\_{ij}\\) is 1 if observation \\(i\\) is in class \\(j\\) and 0 otherwise, and \\(p\_{ij}\\) is the predicted probability that observation \\(i\\) belongs to class \\(j\\). The logarithm in this case is the natural logarithm.

The submitted probabilities for a given discourse element are not required to sum to one: they are rescaled prior to being scored, each row being divided by the row sum. In order to avoid the extremes of the \\(\text{log}\\) function, predicted probabilities are replaced with \\(max(min(p,1-10^{-15}),10^{-15})\\).

## Submission File ##

You must submit a CSV file with each discourse element's `discourse_id` and a predicted probability for each of the three effectiveness ratings. The order of the rows does not matter. The file must have a header and should have the following format:

```
discourse_id,Ineffective,Adequate,Effective
a261b6e14276,0.2,0.6,0.4
5a88900e7dc1,3.0,6.0,1.0
9790d835736b,1.0,2.0,3.0
75ce6d68b67b,0.33,0.34,0.33
93578d946723,0.01,0.24,0.47
2e214524dbe3,0.2,0.6,0.4
```