AI Benchmarks

Massive Multitask Language Understanding — a multiple-choice benchmark covering 57 academic subjects from STEM to humanities.

A curated subset of SWE-bench with human-verified task instances for evaluating AI systems on real-world software engineering tasks from GitHub issues.

A dataset of 12,500 competition mathematics problems testing mathematical reasoning across difficulty levels 1-5.

A benchmark of 164 hand-written Python programming problems with unit tests, evaluating code generation from docstrings.

Graduate-level Google-Proof Q&A Diamond subset — extremely difficult questions in physics, chemistry, and biology that even domain experts struggle with.

Abstraction and Reasoning Corpus — a benchmark of visual pattern recognition tasks designed to test fluid intelligence and novel reasoning.

A benchmark for multimodal agents on real-world computer tasks across operating systems, testing GUI interaction and task completion.

Second iteration of the ARC benchmark with harder tasks, designed to remain challenging as AI capabilities improve.

A benchmark evaluating AI agents on complex terminal-based tasks requiring multi-step reasoning and system administration skills.

A composite intelligence index from Artificial Analysis that aggregates performance across multiple benchmarks to provide a single capability score.

Second version of the Terminal-Bench benchmark with expanded task coverage and difficulty.

American Invitational Mathematics Examination 2025 — a prestigious math competition used as a benchmark for advanced mathematical reasoning.