Terminal-Bench Science: Contribute your scientific workflows as tasks for AI Agents
The Terminal-Bench team is extending Terminal-Bench to complex scientific workflow tasks in the natural sciences.
✦
TLDR
Terminal-Bench Science is now open for contributions — looking for scientists to turn real research workflows into benchmark tasks that shape the next generation of AI agents.
What is Terminal-Bench Science?
Terminal-Bench Science is a benchmark for evaluating AI agents on real computational workflows from scientific research. It builds on Terminal-Bench, which has been adopted by frontier labs including Anthropic, OpenAI, and Google DeepMind and has helped drive progress in AI agents on software engineering tasks by defining what those labs measure and optimize for. Terminal-Bench Science brings the same approach to the natural sciences.
Why do we need Terminal-Bench Science?
Most existing “AI for Science” benchmarks test textbook knowledge, not real workflows. Terminal-Bench Science closes this gap with real computational workflow tasks from research labs, evaluated in containerized environments with programmatic verification. The goal is to give scientists a direct voice in shaping AI progress: domain experts contribute scientific workflows as benchmark tasks, frontier labs evaluate and improve their AI agents against them, and the improved AI agents with stronger scientific capabilities flow back as better tools for researchers.
Domain Coverage
Terminal-Bench Science is targeting 100+ benchmark tasks across the life sciences, physical sciences, and earth sciences, but is also open to tasks from the mathematical sciences and other domains with computational workflows.
| Domain | Areas |
|---|---|
| Life Sciences | Biology, Medicine, Neuroscience |
| Physical Sciences | Physics, Chemistry, Astronomy, Materials Science |
| Earth Sciences | Atmospheric Science, Geoscience, Water Science |
| Mathematical Sciences | Applied Mathematics, Statistics, Autoformalization |
| Other | Interdisciplinary Sciences, Computational Sciences, Engineering Sciences, etc. |
Contribute
WHY CONTRIBUTE
- Make AI better at your science: Frontier labs optimize for what benchmarks measure. Your tasks directly incentivize them to improve their AI systems on the scientific problems in your domain.
- Gain experience in agentic evaluation: Get hands-on with evaluating frontier AI agents — learn how to design rigorous benchmarks and see firsthand where today’s best models succeed and fail on real scientific work.
- Become a co-author: Contributors with merged tasks receive co-authorship on the Terminal-Bench Science paper.
What Terminal-Bench team looks for
The Terminal-Bench team looking for complex, real-world computational workflows from practicing scientists across the natural sciences that meet the following three key criteria:
- Scientifically grounded. Tasks should reflect computational workflows from real research in the natural sciences — ideally drawn from your own work or replicating published results in your domain of expertise.
- Objectively verifiable. Solutions must be programmatically checkable with deterministic pytest-based evaluation. We are not looking for open-ended tasks like hypothesis generation or literature review.
- Genuinely difficult. We target tasks that today’s best AI agents cannot yet reliably solve. Hard tasks expose real gaps and push capabilities forward — we’re aiming for a 10–20% solve rate at release.
Tasks follow the Harbor Task Format. Check out example tasks for reference.
How to contribute
The Terminal-Bench team follows a curated contribution process to maintain quality:
- Connect — Join the Discord, introduce yourself in #tb-science, and pitch your task idea in #tb-science-task-ideas for early feedback. Follow #tb-science-announcements for updates and weekly meetings (Mondays, 9am PT).
- Propose — When you’re ready, submit your idea via the Task Proposal Form. Proposals are posted on our Task Proposal Board and in #tb-science-task-proposals. An LLM judge evaluates it against our Task Proposal Rubric, and human reviewers use that to approve your proposal and guide you toward implementation.
- Build — Once approved, build the task in the Harbor Task Format and submit a Pull Request following our Contributing Guide. Your implementation is evaluated against our Task Implementation Rubric, and human reviewers also assess difficulty, scientific quality, and overall fit. We work with you iteratively until it’s ready to merge.
Once merged, the Terminal-Bench team runs frontier AI agents against all merged tasks to calibrate difficulty. Tasks that pass are included in the official Terminal-Bench Science release on the Terminal-Bench Benchmarks and Terminal-Bench Leaderboards.
Deadline
Tasks must be submitted and merged by August 17, 2026. Starting early is highly recommended — most tasks require a few rounds of feedback and iteration before they’re ready to merge.
Resources
Join the Discord and reach out to @stevendi11 on Discord or stevendi@stanford.edu to get involved. Key channels: #tb-science for general discussion, #tb-science-announcements for project updates, #tb-science-task-ideas for quick early feedback on ideas, and #tb-science-task-proposals for submitted proposals, automated reviews, and reviewer feedback. Plus, you can join the weekly meeting every Monday at 9am PT.
Acknowledgements
Terminal-Bench Science is an open academic collaboration hosted by Stanford University and the Laude Institute. As part of the Terminal-Bench franchise, it is built by the Terminal-Bench & Harbor Framework team, and scientific contributors. We thank Snorkel AI for support via the Open Benchmarks Grants program, and Laude Institute and 2077AI for API credits that power benchmark evaluations.