10 papers
LOCUS: Local Visual Cue Search for Enhancing Fine-Grained Perception in Multimodal Large Language Models
Zhou Tao, Fang Zhang, Zewen Ding +5
Multimodal Large Language Models (MLLMs) remain unreliable on fine-grained visual perception, even when high-resolution inputs preserve the necessary local details. We identify thi…
When Thinking Hurts: Mitigating Visual Forgetting in Video Reasoning via Frame Repetition
Xiaokun Sun, Yubo Wang, Haoyu Cao +1
Recently, Multimodal Large Language Models (MLLMs) have demonstrated significant potential in complex visual tasks through the integration of Chain-of-Thought (CoT) reasoning. Howe…
DiG: Differential Grounding for Enhancing Fine-Grained Perception in Multimodal Large Language Model
Zhou Tao, Shida Wang, Yongxiang Hua +2
Multimodal Large Language Models have achieved impressive performance on a variety of vision-language tasks, yet their fine-grained visual perception and precise spatial reasoning…
VITA-E: Natural Embodied Interaction with Concurrent Seeing, Hearing, Speaking, and Acting
Xiaoyu Liu, Chaoyou Fu, Chi Yan +15
Current Vision-Language-Action (VLA) models are often constrained by a rigid, static interaction paradigm, which lacks the ability to see, hear, speak, and act concurrently as well…
Input Domain Aware MoE: Decoupling Routing Decisions from Task Optimization in Mixture of Experts
Yongxiang Hua, Haoyu Cao, Zhou Tao +4
Sparse Mixture of Experts (sMoE) has become a pivotal approach for scaling large vision-language models, offering substantial capacity while maintaining computational efficiency th…
CROP: Integrating Topological and Spatial Structures via Cross-View Prefixes for Molecular LLMs
Jianting Tang, Yubo Wang, Haoyu Cao +1
Recent advances in molecular science have been propelled significantly by large language models (LLMs). However, their effectiveness is limited when relying solely on molecular seq…