Resource library
Evaluating Coding Agent Capabilities with Terminal-Bench: Snorkel’s Role in Building the Next Generation Benchmark
Closing the Evaluation Gap in Agentic AI
Announcing a $3M commitment to launch Open Benchmarks Grants
Benchtalks #1: Alex Shaw (Terminal-Bench, Harbor) – Building the Benchmark Factory
Building FinQA: An Open RL Environment for Financial Reasoning Agents
As enterprises look toward deploying LLM-powered, business-critical applications, they’re learning to use strategies beyond prompting.
Recent developments in AI tools have made email surveillance for banks better than ever. See how foundation models and Snorkel Flow can help.
In this webinar, Snorkel AI CEO and co-founder Alex Ratner will take live audience questions and provide an overview of how solving the “last mile” problem is increasingly all about the data.
Users and organizations are generating ever-increasing amounts of private data from a wide range of sources. Incorporating private context is important to personalize open-domain tasks such as question-answering, fact-checking, and personal assistants. State-of-the-art systems for these tasks explicitly retrieve information that is relevant to an input question from a background corpus before producing an answer. While today’s retrieval systems assume…
Getting better performance from foundation models (with less data)
Even with the rapid advancements to AI made possible by LLMs and Foundation Models, data remains the key to unlocking real value for enterprise AI.
GenAI may be the most transformative technology of the past decade but data is where enterprises are able to realize real value from AI today.
The paper explores the use of pseudolabels, which are heuristic labels for unlabeled data, to enhance the performance of vision-language models like CLIP via prompt tuning. The authors investigate different learning paradigms and prompt modalities and find that iterative prompt-training strategies leveraging CLIP-based pseudolabels lead to significant improvements in CLIP’s image classification performance.
The paper introduces Alfred, a system for programmatic weak supervision (PWS) that creates training data for machine learning by prompting. It enables users to encode their subject matter expertise via natural language prompts for language and vision-language models.
