Resource library

If CLIP Could Talk: Understanding Vision-Language Model Representations Through Their Preferred Concept Descriptions

"Recent works often assume that VisionLanguage Model (VLM) representations are based on visual attributes like shape. However, it is unclear to what extent VLMs prioritize this information to represent concepts. We propose Extract and Explore (EX2), a novel approach to characterize important textual features for VLMs. EX2 uses reinforcement learning to align a large language model with VLM preferences and generates descriptions that incorporate the important features for the VLM. Then, we inspect the descriptions to identify the features that contribute to VLM representations. We find that spurious descriptions have a major role in VLM representations despite providing no helpful...

Research Paper

If CLIP Could Talk: Understanding Vision-Language Model Representations Through Their Preferred Concept Descriptions

“Recent works often assume that VisionLanguage Model (VLM) representations are based on visual attributes like shape. However, it is unclear to what extent VLMs prioritize this information to represent concepts. We propose Extract and Explore (EX2), a novel approach to characterize important textual features for VLMs. EX2 uses reinforcement learning to align a large language model with VLM preferences and…

Sep 18, 2024 •

R. Esfandiarpoor, et al.

Learn more about If CLIP Could Talk: Understanding Vision-Language Model Representations Through Their Preferred Concept Descriptions

Found in the Middle: Calibrating Positional Attention Bias Improves Long Context Utilization

Large language models (LLMs), even when specifically trained to process long input contexts, struggle to capture relevant information located in the middle of their input. This phenomenon has been known as the lost-in-themiddle problem. In this work, we make three contributions. First, we set out to understand the factors that cause this phenomenon. In doing so, we establish a connection between lost-in-the-middle to LLMs’ intrinsic attention bias: LLMs exhibit an U-shaped attention bias where the tokens at the beginning and at the end of its input receive higher attention, regardless of their relevance. Second, we mitigate this positional bias through...

Research Paper

Found in the Middle: Calibrating Positional Attention Bias Improves Long Context Utilization

Large language models (LLMs), even when specifically trained to process long input contexts, struggle to capture relevant information located in the middle of their input. This phenomenon has been known as the lost-in-themiddle problem. In this work, we make three contributions. First, we set out to understand the factors that cause this phenomenon. In doing so, we establish a connection…

Sep 18, 2024 •

C. Hsieh, et al.

Learn more about Found in the Middle: Calibrating Positional Attention Bias Improves Long Context Utilization

Exploring the Potential of Large Language Models in Neurology, Using Neurologic Localization as an Example

Research Paper

Exploring the Potential of Large Language Models in Neurology, Using Neurologic Localization as an Example

Sep 18, 2024 •

CC. Chiang, et al.

Learn more about Exploring the Potential of Large Language Models in Neurology, Using Neurologic Localization as an Example

Evaluating Language Model Context Windows: A “Working Memory” Test and Inference-time Correction

Large language models are prominently used in real-world applications, often tasked with reasoning over large volumes of documents. An exciting development in this space is models boasting extended context capabilities, with some accommodating over 2 million tokens. Such long context model capabilities remain uncertain in production systems, motivating the need to benchmark their performance on real world use cases. We address this challenge by proposing SWiM, an evaluation framework that addresses the limitations of standard tests. Testing the framework on eight long context models, we find that even strong models such as GPT-4 and Claude 3 Opus degrade in performance...

Research Paper

Evaluating Language Model Context Windows: A “Working Memory” Test and Inference-time Correction

Large language models are prominently used in real-world applications, often tasked with reasoning over large volumes of documents. An exciting development in this space is models boasting extended context capabilities, with some accommodating over 2 million tokens. Such long context model capabilities remain uncertain in production systems, motivating the need to benchmark their performance on real world use cases. We…

Sep 18, 2024 •

A. Dsouza, et al.

Learn more about Evaluating Language Model Context Windows: A “Working Memory” Test and Inference-time Correction

Characterizing the limitations of using diagnosis codes in the context of machine learning for healthcare

Importance: Diagnostic codes are commonly used as inputs for clinical prediction models, to create labels for prediction tasks, and to identify cohorts for multicenter network studies. However, the coverage rates of diagnostic codes and their variability across institutions are underexplored. Objective: Primary objective was to describe lab- and diagnosis-based labels for 7 selected outcomes at three institutions. Secondary objectives were to describe agreement, sensitivity, and specificity of diagnosis-based labels against lab-based labels. Methods: This study included three cohorts: SickKidsPeds from The Hospital for Sick Children, and StanfordPeds and StanfordAdults from Stanford Medicine. We included seven clinical outcomes with lab-based definitions:...

Research Paper

Characterizing the limitations of using diagnosis codes in the context of machine learning for healthcare

Importance: Diagnostic codes are commonly used as inputs for clinical prediction models, to create labels for prediction tasks, and to identify cohorts for multicenter network studies. However, the coverage rates of diagnostic codes and their variability across institutions are underexplored. Objective: Primary objective was to describe lab- and diagnosis-based labels for 7 selected outcomes at three institutions. Secondary objectives were…

Sep 18, 2024 •

LL Guo, et al.

Learn more about Characterizing the limitations of using diagnosis codes in the context of machine learning for healthcare

A Systematic Review of Testing and Evaluation of Healthcare Applications of Large Language Models (LLMs)

Studies rarely use real patient care data for LLM evaluation. Administrative tasks such as generating provider billing codes and writing prescriptions are understudied. Natural Language Processing (NLP)/Natural Language Understanding (NLU) tasks like summarization, conversational dialogue, and translation are infrequently explored. Accuracy is the predominant dimension of evaluation, while fairness, bias and toxicity assessments are neglected. Evaluations in specialized fields, such as nuclear medicine and medical genetics are rare. Current LLM assessments in healthcare remain shallow and fragmented. To draw concrete insights on their performance, evaluations need to use real patient care data across a broad range of healthcare and NLP/NLU...

Research Paper

A Systematic Review of Testing and Evaluation of Healthcare Applications of Large Language Models (LLMs)

Studies rarely use real patient care data for LLM evaluation. Administrative tasks such as generating provider billing codes and writing prescriptions are understudied. Natural Language Processing (NLP)/Natural Language Understanding (NLU) tasks like summarization, conversational dialogue, and translation are infrequently explored. Accuracy is the predominant dimension of evaluation, while fairness, bias and toxicity assessments are neglected. Evaluations in specialized fields, such…

Sep 18, 2024 •

S. Bedi, et al.

Learn more about A Systematic Review of Testing and Evaluation of Healthcare Applications of Large Language Models (LLMs)

A Multi-Center Study on the Adaptability of a Shared Foundation Model for Electronic Health Records

Background: Foundation models hold promise for transforming artificial intelligence (AI) in healthcare by providing modular components that are easily adaptable to downstream healthcare tasks, making AI development more scalable and cost-effective. Foundation models for structured electronic health records (EHR), trained on coded medical records from millions of patients, demonstrated benefits including increased performance with fewer training labels, and improved robustness to distribution shifts. However, questions remain on the feasibility of sharing these models across different hospitals and their performance for local task adaptation. Objective: This multi-center study examined the adaptability of a recently released structured EHR foundation model (FMSM), trained...

Research Paper

A Multi-Center Study on the Adaptability of a Shared Foundation Model for Electronic Health Records

Background: Foundation models hold promise for transforming artificial intelligence (AI) in healthcare by providing modular components that are easily adaptable to downstream healthcare tasks, making AI development more scalable and cost-effective. Foundation models for structured electronic health records (EHR), trained on coded medical records from millions of patients, demonstrated benefits including increased performance with fewer training labels, and improved robustness…

Sep 18, 2024 •

LL Guo, et al.

Learn more about A Multi-Center Study on the Adaptability of a Shared Foundation Model for Electronic Health Records

Blog

How a global financial services company built a specialized AI copilot accurate enough for production

Learn how Snorkel, Databricks, and AWS enabled the team to build and deploy small, specialized, and highly accurate models which met their AI production requirements and strategic goals.

Sep 09, 2024 •

Team Snorkel

Learn more about How a global financial services company built a specialized AI copilot accurate enough for production