7 papers · 1 filter
Uncovering Ordinal-Matching Bias in Audio-Visual LLMs
Jihoo Jung, Youngjoon Jang, Hyebin Cho +2
This work aims to improve how audio-visual large language models (AVLLMs) associate speech with the correct visible speaker in multi-speaker scenes. We find that current AVLLMs fre…
OmniSmartHome: A Multimodal Reasoning Benchmark for Smart-Home Agents
Jihoo Jung, Suho Yoo, Jeongsoo Choi +5
Smart-home assistants are expected to handle diverse, realistic requests that arise in daily life. In such interactions, users often rely on the surrounding multimodal context-poin…
OmniKVQuant: KV Cache Quantization for Omni-LLMs
Suho Yoo, Hyunjong Ok, Jongmin Choi +2
As Omni-modal large language models (Omni-LLMs) take in audio, video and text together, their KV cache memory cost grows. KV cache quantization is the de facto approach in text-onl…
Tracing Audio Grounding and Answer Selection in Audio LLMs
Hyebin Cho, Suho Yoo, Jihoo Jung +1
Audio Large Language Models (Audio LLMs) have advanced in audio understanding, yet they can still predict the answer by reasoning from textual cues or linguistic priors rather than…
Out of Sight, Still in Mind: Token Compression for Omni-LLMs
Suho Yoo, Youngjoon Jang, Hyebin Cho +1
The goal of this paper is to reduce the input token cost of Omni-modal large language models (Omni-LLMs) at inference time. Omni-LLMs reason jointly over audio, video and text, but…
Who Wins the Conflict? Mechanistic Interpretability of Text Bias in Audio LLMs
Hyebin Cho, Suho Yoo, Jaehyuk Jang +2
While Audio Large Language Models (Audio LLMs) excel at multimodal understanding, they suffer from text dominance, a bias where models favor text over acoustic evidence, potentiall…
On the Nature of Attention Sink that Shapes Decoding Strategy in Omni-LLMs
Suho Yoo, Youngjoon Jang, Joon Son Chung
The goal of this paper is to strengthen the reasoning of Omnimodal Large Language Models (Omni-LLMs) at inference time, without additional training. These models jointly process vi…