The evaluation of your team's predictions accuracy is measured the percentage of correct predictions as aggregated across all test set authors across all personality axis. That is, the performance in computed as
\[
\sqrt{{1\over 5N}\sum_{a \in Axes} \sum_{i\in\text{Test}} (y_{ia} - \hat y_{ia})^2}.
\]
Here 
\[
\begin{align}
Axes &\qquad \text{is the set $\{Extr, Emot, Agr, Cons, Open\}$}, \\
Test &\qquad \text{is the set of vlogger IDs in the test set}, \\
y_{ia} &\qquad \text{is the observed score for personality axis $a$ for the $i$-th vlogger in the test set, and} \\
\hat y_{ia} &\qquad \text{is the corresponding predicted score.}
\end{align}
\]
N is the size of the test set.
Submission Format
For every author in the dataset
, submission files should contain two columns: Id and value. The Id column identifies the test set author and personality axis, by means of concatenating the
vlogId
string and the axis label (
Extr, Agr, Cons, Emot
or
Open
) together with an underscore (
_
) in between. The value column should contain the predicted value. For example, if for `vlogId`
VLOG14
your team's prediction is that this person has scores Extr = 4.518818, Emot = 2.445342, Agr = 3.005654, Cons = 1.221221, and Open = 4.005012, then the submission file should contain the rows
⋮
VLOG15_Extr,4.518818
VLOG15_Emot,2.445342
VLOG15_Agr,3.005654
VLOG15_Cons,1.221221
VLOG15_Open,4.005012
⋮
The file should contain a header and have the following format:
Id,Expected
VLOG8_Extr,3.518818
VLOG8_Emot,2.946352
VLOG8_Agr,1.205654
VLOG8_Cons,3.221221
VLOG8_Open,4.702312
⋮
etc.
⋮