Resource library
Evaluating Coding Agent Capabilities with Terminal-Bench: Snorkel’s Role in Building the Next Generation Benchmark
Closing the Evaluation Gap in Agentic AI
Announcing a $3M commitment to launch Open Benchmarks Grants
Benchtalks #1: Alex Shaw (Terminal-Bench, Harbor) – Building the Benchmark Factory
Building FinQA: An Open RL Environment for Financial Reasoning Agents
We aim to help our customers get GenAI into production. In our 2024.R3 release, we’ve delivered some exciting GenAI evaluation results.
Discover new NLP features in Snorkel Flow\’s 2024.R3 release, including named entity recognition for PDFs + advanced sequence tagging tools.
Discover the latest enterprise readiness features for Snorkel Flow. Configure safeguards for data compliance and security.
Existing ML benchmarks lack the depth and diversity of annotations needed for evaluating models on business process management (BPM) tasks. BPM is the practice of documenting, measuring, improving, and automating enterprise workflows. However, research has focused almost exclusively on one task– full end-to-end automation using agents based on multimodal foundation models (FMs) like GPT-4. This focus on automation ignores the…
Using AI to supercharge online retail Wayfair is a Boston-based e-commerce company specializing in home goods and furniture, serving ~22M customers and partnering with ~20K suppliers. To ensure that relevant products appear in customer searches (e.g., “blue outdoor pillows”), they rely on product tags (e.g., “blue” “outdoor” and “pillow”). With over 10,000 product tags across 40 million products, creating and…
These case studies details how to apply data-centric workflows to build high-quality, governable, and adaptable AI applications.
Discover practical solutions for creating data to fine-tune LLMs and learn how to overcome the key challenges in LLM customization.
Using AI to support audit relevance and improve operational efficiency This global “big four” consulting company strives to provide its diverse team of experts with the most current and relevant accounting, auditing, and industry information. Over the last 170 years, the company has learned that to anticipate shifts in regulations and proactively help its clients adapt, it must stay up…
Extracting information from 10-Ks documents for KYC Financial institutions are obligated by government policies to carry out customer due diligence as part of customer onboarding. For example, the U.S. Department of the Treasury Financial Crimes Enforcement Network (FinCEN) requires covered financial institutions to identify and verify the identity of beneficial owners of legal entity customers as one of the measures…
