The latest from Srikar Kodati
Blog
Why Frontier Agents Fail Real Engineering Work: Two Terminal-Bench 3.0 Task Deep Dives
Terminal-Bench 3.0 (formerly Frontier-Bench) recently launched, built to track what AI agents can and can’t do across real computer work. Terminal-Bench 2.1 has been saturating, with top agents reaching 84%; on Terminal-Bench 3.0, the best model, Claude Opus 5, achieves just 43.5%. Terminal-Bench 3.0 raises the bar with 74 authentic, verifiable tasks across 7 domains, designed to expose meaningful gaps…
Blog
Inside Frontier-Bench: two Snorkel-built tasks that frontier agents still can’t crack
Frontier-Bench launched this week – the successor to Terminal-Bench, built to track what AI agents can and can’t do across real computer work. Terminal-Bench 2.1 has been saturating, with top agents clearing 75-84%; on Frontier-Bench’s launch set of 74 tasks across 7 domains, the best mode Opus 5 achieving 43.3%. Snorkel AI contributed as a task author and data partner,…
