5 papers · 1 filter
Are Large Reasoning Models Interruptible?
Tsung-Han Wu, Mihran Miroyan, David M. Chan +3
Real-world applications of Large Reasoning Models (LRMs) often require reasoning about changing prompts or environments. In this work, we challenge the frozen world assumption and…
Search Arena: Analyzing Search-Augmented LLMs
Mihran Miroyan, Tsung-Han Wu, Logan King +8
Search-augmented language models combine web search with Large Language Models (LLMs) to improve response groundedness and freshness. However, analyzing these systems remains chall…
Puzzled by Puzzles: When Vision-Language Models Can't Take a Hint
Heekyung Lee, Jiaxin Ge, Tsung-Han Wu +3
Rebus puzzles, visual riddles that encode language through imagery, spatial arrangement, and symbolic substitution, pose a unique challenge to current vision-language models (VLMs)…
CLAIR-A: Leveraging Large Language Models to Judge Audio Captions
Tsung-Han Wu, Joseph E. Gonzalez, Trevor Darrell +1
The Automated Audio Captioning (AAC) task asks models to generate natural language descriptions of an audio input. Evaluating these machine-generated audio captions is a complex ta…
VibeCheck: Discover and Quantify Qualitative Differences in Large Language Models
Lisa Dunlap, Krishna Mandal, Trevor Darrell +2
Large language models (LLMs) often exhibit subtle yet distinctive characteristics in their outputs that users intuitively recognize, but struggle to quantify. These "vibes" -- such…