13 papers
Enhanced Vision-Language Models for Diverse Sensor Understanding: Cost-Efficient Optimization and Benchmarking
Sangyun Chung, Youngjoon Yu, Se Yeon Kim +2
Large-scale Vision-Language Models (VLMs) have achieved notable progress in aligning visual inputs with text. However, their ability to deeply understand the unique physical proper…
GenRecal: Generation after Recalibration from Large to Small Vision-Language Models
Byung-Kwan Lee, Ryo Hachiuma, Yong Man Ro +2
Recent advancements in vision-language models (VLMs) have leveraged large language models (LLMs) to achieve performance on par with closed-source systems like GPT-4V. However, depl…
ReFoCUS: Reinforcement-guided Frame Optimization for Contextual Understanding
Hosu Lee, Junho Kim, Hyunjun Kim +1
Recent progress in Large Multi-modal Models (LMMs) has enabled effective vision-language reasoning, yet the ability to video understanding remains constrained by suboptimal frame s…
Diffusion Large Language Models for Visual Speech Recognition
Jeong Hun Yeo, Chae Won Kim, Hyeongseop Rha +1
Existing Visual Speech Recognition (VSR) systems commonly rely on left-to-right autoregressive decoding, which can force premature decisions on visually ambiguous tokens before suf…
STRIDE: When to Speak Meets Sequence Denoising for Streaming Video Understanding
Junho Kim, Hosu Lee, James M. Rehg +2
Recent progress in video large language models (Video-LLMs) has enabled strong offline reasoning over long and complex videos. However, real-world deployments increasingly require…
MAD: Modality-Adaptive Decoding for Mitigating Cross-Modal Hallucinations in Multimodal Large Language Models
Sangyun Chung, Se Yeon Kim, Youngchae Chee +1
Multimodal Large Language Models (MLLMs) suffer from cross-modal hallucinations, where one modality inappropriately influences generation about another, leading to fabricated outpu…