our research lab

From cutting-edge research to enterprise and frontier impact

Our research team advances the science of data-centric AI in partnership with leading enterprises and frontier labs. We translate these breakthroughs into production, powering the next generation of AI systems across industries, research, and government.

Browse our research library

Deep research roots

Born out of the Stanford AI lab in 2019 and in collaboration with leading research institutions, Snorkel-affiliated researchers have published more than 170 peer-reviewed research papers on weak supervision, AI data development techniques, foundation models, and more — with special recognition at events such as NeurlPS, ICML, and ICLR. Our researchers are closely affiliated with academic institutions including Stanford University, University of Washington, Brown University, and the University of Wisconsin-Madison

Featured benchmarks

Exclusive to Snorkel, these benchmarks are meticulously designed and validated by subject matter experts to probe frontier AI models on demanding, specialized tasks.

These are just a few of our featured benchmarks—new ones are added regularly, so check back often to see the latest from our research team.

Leaderboards

Challenging benchmarks for models and agents

Snorkel benchmarks are built with human expertise to test models on realistic tasks ranging from coding, to financial analysis, healthcare, and much more. For example, our SnorkelUnderwrite benchmark includes multi-turn agentic tasks germane to the insurance industry.

See the latest scores on our SnorkelUnderwrite leaderboard.

Learn more

Rubrics

Aligning human expertise and automated evaluation

We investigate how to scalably develop rubrics that are both comprehensive of the desired agentic capabilities and reliably assessed by both human experts and AI judges.

Learn more about our findings.

Learn more

RL ENvironments

Environments give agents a fully realized simulation

As tool-calling and more open-ended application requirements break simple test frameworks, agent validation must be done with techniques that reproduce real-world variability. For example, our contributions to Terminal-Bench (tbench.ai) include containerized simulation environments.

Read more in our blog post.

Learn more

Browse blog posts and 100+ peer reviewed academic papers

Research Paper

Automating Benchmark Design

Blog

Parsing Isn’t Neutral: Why Evaluation Choices Matter

Blog

The science of rubric design

Research Paper

Shrinking the Generation-Verification Gap with Weak Verifiers

Blog

Data quality and rubrics: how to build trust in your models

Research Paper

Theoretical Physics Benchmark (TPBench)- a Dataset and Study of AI Reasoning Capabilities in Theoretical Physics

Research Paper

WONDERBREAD: A Benchmark for Evaluating Multimodal Foundation Models on Business Process Management Tasks

Research Paper

The ALCHEmist: Automated Labeling 500x CHEaper Than LLM Data Annotators

Research Paper

Skill-It! A Data-Driven Skills Framework for Understanding and Training Language Models

Blog

NEW

Benchtalks #1: Alex Shaw (Terminal-Bench, Harbor) – Building the Benchmark Factory

Blog

NEW

Building FinQA: An Open RL Environment for Financial Reasoning Agents

Blog

How Tool Discipline Let a 4B Model Outsmart a 235B Giant on Financial Tasks

1 2 3 ... 25

Backed by a $3M commitment, the Open Benchmarks Grants program — in partnership with Hugging Face, Prime Intellect, Together AI and Factory HQ and Harbor— funds open-source datasets, benchmarks, and evaluation artifacts that shape how frontier AI systems are built and evaluated.

Applications are rolling — starting March 1st.

Apply for a grant

Learn more

In collaboration with: