We introduce EgoSchema, a diagnostic benchmark for assessing very long-form video-language understanding capabilities of modern multimodal systems. Derived from Ego4D, EgoSchema consists of over 5000 human curated multiple choice question answer pairs, spanning over 250 hours of real video data, covering a very broad range of natural human activity and behavior. For each question, EgoSchema requires the correct answer to be selected between five given options based on a three-minute-long video clip. More details at our [website](https://egoschema.github.io/) and [Github page](https://github.com/egoschema/egoschema).

**Note**:  EgoSchema is intended for a 0-shot evaluation benchmark, hence the entire correct answer file will not be made public.

## :dizzy: Dataset Highlights 

:exclamation: EgoSchema is 10x to 100x more difficult longer temporal reasoning than almost all other video datasets.
:exclamation: Largest OSS Video-Language models with 7B+ parameters achieve QA accuracy of <33% (Random choice is 20%). Humans achieve ~76%.
:exclamation: Even web-scaled trained closed source models with 100B+ parameters achieve <40% accuracy, highlighting the massive latent gap in model capabilities for long-term video understanding.