Evaluation metric
The Root Mean Squared Error (RMSE) will be used for evaluation. The RMSE is defined as the root of the average squared error between the prediction and groundtruth.
![rmse](https://s4.postimg.org/ibb1pbbst/rmse.png)
Outputs associated with missing inputs contribute equally to those without.
Leaderboard and final evaluation
The predictions on 50% of the test data points are used to score the submission according to the RMSE and maintain a public leaderboard. The predictions on the remaining 50% of the test data points will be used, after the competition closes, for the final evaluation. This prevents overfitting on the public test data.
Submission rules
Each team may only submit 10 sets of predictions each day. When the competition closes, each team will select 1 set of predictions to put forward for the final evaluation.