Resource library
Evaluating Coding Agent Capabilities with Terminal-Bench: Snorkel’s Role in Building the Next Generation Benchmark
Closing the Evaluation Gap in Agentic AI
Announcing a $3M commitment to launch Open Benchmarks Grants
Benchtalks #1: Alex Shaw (Terminal-Bench, Harbor) – Building the Benchmark Factory
Building FinQA: An Open RL Environment for Financial Reasoning Agents
Advancements in artificial intelligence promise efficiency gains for financial institutions. AI-powered applications can revolutionize an organization’s risk management, fraud detection, compliance monitoring, and other processes. Financial services companies have smart data scientists and good infrastructure needed for deploying AI. But their ability to rapidly develop and deploy AI applications is hampered by several unique challenges.
Takeaways from MLSys Seminars with Chip HuyenIn November, I had the opportunity to come back to Stanford to participate in MLSys Seminars, a series about Machine Learning Systems. It was great to see the growing interest of the academic community in building practical AI applications. Here is a recording of the talk.The talk was originally about the principles of good…
This work demonstrates how organizational resources, in the form of aggregate statistics, knowledge bases, and existing services can be used to connect new and existing data modalities.
Electrophysiological signals exhibit both periodic and aperiodic properties. Periodic oscillations have been linked to numerous physiological, cognitive, behavioral and disease states. Emerging evidence demonstrates that the aperiodic component has putative physiological interpretations and that it dynamically changes with age, task demands and cognitive states. Electrophysiological neural activity is typically analyzed using canonically defined frequency bands, without consideration of the aperiodic…
Introducing FlyingSquid, a weak supervision framework that runs orders of magnitude faster than previous weak supervision approaches and requires fewer assumptions
We love meeting people in the data science and machine learning community. Here are a few upcoming events where you can meet Snorkelers.
This paper proposes cross-modal data programming (XMDP) for machine learning (ML) in medicine.
This paper provides a series of results studying how performance scales with changes in source coverage, source accuracy, and the Lipschitzness of label distributions in the embedding space, and compare this rate to standard weak supervision.
Presenting Trove, a framework for weakly supervised entity classification using medical ontologies and expert-generated rules.
