The evaluation format is accuracy. Because there are more wrong answers than correct answers, you should always get above 0.53 accuracy. This is the simplest baseline you should do better than.

Submission Format

For row in the test file, submission files should contain two columns: row and corr. You should provide for each row whether the guess is correct or not.

The file should contain a header and have the following format:

row,corr
31,False
32,False
33,False
88,False
91,False
139,False
140,False
141,False
142,False
188,True