4 citations · 13 across the 26 of their papers we have counts for
27 papers
Scaling Therapy Supervision: Assessing Psychotherapy Fidelity Using Large Language Models
Arvind Pillai, Alexis Roach, Jason Holden +11
Psychotherapy fidelity assessment measures how faithfully a therapist adheres to a treatment protocol, a critical determinant of positive patient outcomes. However, fidelity assess…
Measuring Distribution Shift in User Prompts and Its Effects on LLM Performance
Parker Seegmiller, Sarah Masud Preum
LLMs are increasingly deployed in dynamic, real-world settings, where the distribution of user prompts can shift substantially over time as new tasks, prompts, and users are introd…
Medical Triage as Pairwise Ranking: A Benchmark for Urgency in Patient Portal Messages
Joseph Gatto, Parker Seegmiller, Timothy Burdick +4
Medical triage is the task of allocating medical resources and prioritizing patients based on medical need. This paper introduces the first large-scale public dataset for studying…
How Much Would a Clinician Edit This Draft? Evaluating LLM Alignment for Patient Message Response Drafting
Parker Seegmiller, Joseph Gatto, Sarah E. Greer +4
Large language models (LLMs) show promise in drafting responses to patient portal messages, yet their integration into clinical workflows raises various concerns, including whether…
Can LLMs Solve My Grandma's Riddle? Evaluating Multilingual Large Language Models on Reasoning Traditional Bangla Tricky Riddles
Nurul Labib Sayeedi, Md. Faiyaz Abdullah Sayeedi, Khushnur Binte Jahangir +2
Large Language Models (LLMs) show impressive performance on many NLP benchmarks, yet their ability to reason in figurative, culturally grounded, and low-resource settings remains u…
Socially Constructed Treatment Plans: Analyzing Online Peer Interactions to Understand How Patients Navigate Complex Medical Conditions
Madhusudan Basak, Omar Sharif, Jessica Hulsey +4
When faced with complex and uncertain medical conditions (e.g., cancer, mental health conditions, recovery from substance dependency), millions of patients seek online peer support…
Follow-up Question Generation For Enhanced Patient-Provider Conversations
Joseph Gatto, Parker Seegmiller, Timothy Burdick +3
Follow-up question generation is an essential feature of dialogue systems as it can reduce conversational ambiguity and enhance modeling complex interactions. Conversational contex…
REGen: A Reliable Evaluation Framework for Generative Event Argument Extraction
Omar Sharif, Joseph Gatto, Madhusudan Basak +1
Event argument extraction identifies arguments for predefined event roles in text. Existing work evaluates this task with exact match (EM), where predicted arguments must align exa…
In-Context Learning for Preserving Patient Privacy: A Framework for Synthesizing Realistic Patient Portal Messages
Joseph Gatto, Parker Seegmiller, Timothy E. Burdick +1
Since the COVID-19 pandemic, clinicians have seen a large and sustained influx in patient portal messages, significantly contributing to clinician burnout. To the best of our knowl…
Explicit, Implicit, and Scattered: Revisiting Event Extraction to Capture Complex Arguments
Omar Sharif, Joseph Gatto, Madhusudan Basak +1
Prior works formulate the extraction of event-specific arguments as a span extraction problem, where event arguments are explicit -- i.e. assumed to be contiguous spans of text in…
A Thematic Framework for Analyzing Large-scale Self-reported Social Media Data on Opioid Use Disorder Treatment Using Buprenorphine Product
Madhusudan Basak, Omar Sharif, Sarah E. Lord +7
Background: One of the key FDA-approved medications for Opioid Use Disorder (OUD) is buprenorphine. Despite its popularity, individuals often report various information needs regar…
Depth : Improving Evaluation of Cross-Domain Text Classification by Measuring Semantic Generalizability
Parker Seegmiller, Joseph Gatto, Sarah Masud Preum
Recent evaluations of cross-domain text classification models aim to measure the ability of a model to obtain domain-invariant performance in a target domain given labeled samples…
Do LLMs Find Human Answers To Fact-Driven Questions Perplexing? A Case Study on Reddit
Parker Seegmiller, Joseph Gatto, Omar Sharif +2
Large language models (LLMs) have been shown to be proficient in correctly answering questions in the context of online discourse. However, the study of using LLMs to model human-l…
Scope of Large Language Models for Mining Emerging Opinions in Online Health Discourse
Joseph Gatto, Madhusudan Basak, Yash Srivastava +2
In this paper, we develop an LLM-powered framework for the curation and evaluation of emerging opinion mining in online health communities. We formulate emerging opinion mining as…
Deciphering Hate: Identifying Hateful Memes and Their Targets
Eftekhar Hossain, Omar Sharif, Mohammed Moshiul Hoque +1
Internet memes have become a powerful means for individuals to express emotions, thoughts, and perspectives on social media. While often considered as a source of humor and enterta…
Large Language Models for Document-Level Event-Argument Data Augmentation for Challenging Role Types
Joseph Gatto, Parker Seegmiller, Omar Sharif +1
Event Argument Extraction (EAE) is an extremely difficult information extraction problem -- with significant limitations in few-shot cross-domain (FSCD) settings. A common solution…
Align before Attend: Aligning Visual and Textual Features for Multimodal Hateful Content Detection
Eftekhar Hossain, Omar Sharif, Mohammed Moshiul Hoque +1
Multimodal hateful content detection is a challenging task that requires complex reasoning across visual and textual modalities. Therefore, creating a meaningful multimodal represe…
Chain-of-Thought Embeddings for Stance Detection on Social Media
Joseph Gatto, Omar Sharif, Sarah Masud Preum
Stance detection on social media is challenging for Large Language Models (LLMs), as emerging slang and colloquial language in online conversations often contain deeply implicit st…
Statistical Depth for Ranking and Characterizing Transformer-Based Text Embeddings
Parker Seegmiller, Sarah Masud Preum
The popularity of transformer-based text embeddings calls for better statistical tools for measuring distributions of such embeddings. One such tool would be a method for ranking t…
Not Enough Labeled Data? Just Add Semantics: A Data-Efficient Method for Inferring Online Health Texts
Joseph Gatto, Sarah M. Preum
User-generated texts available on the web and social platforms are often long and semantically challenging, making them difficult to annotate. Obtaining human annotation becomes in…
Text Encoders Lack Knowledge: Leveraging Generative LLMs for Domain-Specific Semantic Textual Similarity
Joseph Gatto, Omar Sharif, Parker Seegmiller +2
Amidst the sharp rise in the evaluation of large language models (LLMs) on various tasks, we find that semantic textual similarity (STS) has been under-explored. In this study, we…
Characterizing Information Seeking Events in Health-Related Social Discourse
Omar Sharif, Madhusudan Basak, Tanzia Parvin +5
Social media sites have become a popular platform for individuals to seek and share health information. Despite the progress in natural language processing for social media mining,…
The Scope of In-Context Learning for the Extraction of Medical Temporal Constraints
Parker Seegmiller, Joseph Gatto, Madhusudan Basak +4
Medications often impose temporal constraints on everyday patient activity. Violations of such medical temporal constraints (MTCs) lead to a lack of treatment adherence, in additio…
ActSafe: Predicting Violations of Medical Temporal Constraints for Medication Adherence
Parker Seegmiller, Joseph Gatto, Abdullah Mamun +4
Prescription medications often impose temporal constraints on regular health behaviors (RHBs) of patients, e.g., eating before taking medication. Violations of such medical tempora…