2 citations · 6 across the 15 of their papers we have counts for
11 papers · 1 filter
Measuring Distribution Shift in User Prompts and Its Effects on LLM Performance
Parker Seegmiller, Sarah Masud Preum
LLMs are increasingly deployed in dynamic, real-world settings, where the distribution of user prompts can shift substantially over time as new tasks, prompts, and users are introd…
Medical Triage as Pairwise Ranking: A Benchmark for Urgency in Patient Portal Messages
Joseph Gatto, Parker Seegmiller, Timothy Burdick +4
Medical triage is the task of allocating medical resources and prioritizing patients based on medical need. This paper introduces the first large-scale public dataset for studying…
How Much Would a Clinician Edit This Draft? Evaluating LLM Alignment for Patient Message Response Drafting
Parker Seegmiller, Joseph Gatto, Sarah E. Greer +4
Large language models (LLMs) show promise in drafting responses to patient portal messages, yet their integration into clinical workflows raises various concerns, including whether…
Follow-up Question Generation For Enhanced Patient-Provider Conversations
Joseph Gatto, Parker Seegmiller, Timothy Burdick +3
Follow-up question generation is an essential feature of dialogue systems as it can reduce conversational ambiguity and enhance modeling complex interactions. Conversational contex…
Depth : Improving Evaluation of Cross-Domain Text Classification by Measuring Semantic Generalizability
Parker Seegmiller, Joseph Gatto, Sarah Masud Preum
Recent evaluations of cross-domain text classification models aim to measure the ability of a model to obtain domain-invariant performance in a target domain given labeled samples…
Do LLMs Find Human Answers To Fact-Driven Questions Perplexing? A Case Study on Reddit
Parker Seegmiller, Joseph Gatto, Omar Sharif +2
Large language models (LLMs) have been shown to be proficient in correctly answering questions in the context of online discourse. However, the study of using LLMs to model human-l…
Large Language Models for Document-Level Event-Argument Data Augmentation for Challenging Role Types
Joseph Gatto, Parker Seegmiller, Omar Sharif +1
Event Argument Extraction (EAE) is an extremely difficult information extraction problem -- with significant limitations in few-shot cross-domain (FSCD) settings. A common solution…
Statistical Depth for Ranking and Characterizing Transformer-Based Text Embeddings
Parker Seegmiller, Sarah Masud Preum
The popularity of transformer-based text embeddings calls for better statistical tools for measuring distributions of such embeddings. One such tool would be a method for ranking t…
Text Encoders Lack Knowledge: Leveraging Generative LLMs for Domain-Specific Semantic Textual Similarity
Joseph Gatto, Omar Sharif, Parker Seegmiller +2
Amidst the sharp rise in the evaluation of large language models (LLMs) on various tasks, we find that semantic textual similarity (STS) has been under-explored. In this study, we…
The Scope of In-Context Learning for the Extraction of Medical Temporal Constraints
Parker Seegmiller, Joseph Gatto, Madhusudan Basak +4
Medications often impose temporal constraints on everyday patient activity. Violations of such medical temporal constraints (MTCs) lead to a lack of treatment adherence, in additio…
HealthE: Classifying Entities in Online Textual Health Advice
Joseph Gatto, Parker Seegmiller, Garrett Johnston +1
The processing of entities in natural language is essential to many medical NLP systems. Unfortunately, existing datasets vastly under-represent the entities required to model publ…