Benchmarks for what frontier AI hasn't solved

Terminal agent evaluation led by Stanford University and Laude Institute. v2.1 fixes 28 tasks from 2.0 and introduces continuous validation.

A benchmark for evaluating AI models on complex, real-world coding tasks that require multi-step reasoning, tool use, and autonomous problem-solving.

A benchmark co-created with Snorkel’s financial expert network, to test agents on financial reasoning questions through tool-calling and planning.

A procedurally-generated and expert-verified benchmark for evaluating mathematical reasoning and compositional capabilities in LLMs.

A benchmark of expert-verified financial QA created from financial reports for evaluating AI agents on tool-calling and reasoning capabilities.

A procedurally-generated and expert-verified benchmark for evaluating mathematical and spatial reasoning capabilities of LLMs through graph reasoning problems.

An expert-verified frontier benchmark with multi-turn conversations, focused on agentic reasoning and tool use in commercial underwriting settings.

A benchmark designed to evaluate linguistic reasoning and instruction-following capabilities in language models through the gameplay of Wordle.

A procedurally-generated benchmark for evaluating allocentric and egocentric spatial reasoning capabilities in LLMs.