7 papers
OmniVox: Zero-Shot Emotion Recognition with Omni-LLMs
John Murzaku, Owen Rambow
The use of omni-LLMs (large language models that accept any modality as input), particularly for multimodal cognitive state tasks involving speech, is understudied. We present Omni…
ECLAIR: Enhanced Clarification for Interactive Responses in an Enterprise AI Assistant
John Murzaku, Zifan Liu, Vaishnavi Muppala +3
Large language models (LLMs) have shown remarkable progress in understanding and generating natural language across various applications. However, they often struggle with resolvin…
ECLAIR: Enhanced Clarification for Interactive Responses
John Murzaku, Zifan Liu, Md Mehrab Tanjim +3
We present ECLAIR (Enhanced CLArification for Interactive Responses), a novel unified and end-to-end framework for interactive disambiguation in enterprise AI assistants. ECLAIR ge…
Zero-Shot Belief: A Hard Problem for LLMs
John Murzaku, Owen Rambow
We present two LLM-based approaches to zero-shot source-and-target belief prediction on FactBank: a unified system that identifies events, sources, and belief labels in a single pa…
Synthetic Audio Helps for Cognitive State Tasks
Adil Soubki, John Murzaku, Peter Zeng +1
The NLP community has broadly focused on text-only approaches of cognitive state tasks, but audio can provide vital missing cues through prosody. We posit that text-to-speech model…
Multimodal Belief Prediction
John Murzaku, Adil Soubki, Owen Rambow
Recognizing a speaker's level of commitment to a belief is a difficult task; humans do not only interpret the meaning of the words in context, but also understand cues from intonat…
Views Are My Own, but Also Yours: Benchmarking Theory of Mind Using Common Ground
Adil Soubki, John Murzaku, Arash Yousefi Jordehi +4
Evaluating the theory of mind (ToM) capabilities of language models (LMs) has recently received a great deal of attention. However, many existing benchmarks rely on synthetic data,…