Submissions are evaluated using mean per-question accuracy across all VQA fields. For each video, we compare your predicted numeric code to the ground-truth numeric code for every question column (per the legend), compute accuracy per question over the full test set, and then average these accuracies equally across questions to produce a single score.​

Unknown answers are evaluated as a valid class with value -1.

For any questions which simply do not apply (Eg. what type of vehicle is secondary entity? When secondary entity is not a vehicle) should be filled with -2, these answers will not be scored.

###Submission File

For each video_id in the test set, you must submit one integer-valued prediction per question column, matching the header exactly. The submission must contain a header and follow this format:
```python

video_id,"A)Weather:
Q1.a: What was the time of day during the incident?","A)Weather:
Q1.b: What were the weather conditions during the incident?",...
269,0,1,2,0,-1,7,-1,2,1,0,-2,0,2,2,0,0,0,4,0,0,0,5,7,-1,0
425,1,0,4,2,4,1,0,2,1,0,0,1,2,2,0,0,4,0,6,3,0,0,10,3,0
```

Exact column names, ordering, and a sample_submission are provided in the “Data” tab.