18 papers
BabyVision: Visual Reasoning Beyond Language
Liang Chen, Weichu Xie, Yiyan Liang +27
While humans develop core visual skills long before acquiring language, contemporary Multimodal LLMs (MLLMs) still rely heavily on linguistic priors to compensate for their fragile…
From Context to Skills: Can Language Models Learn from Context Skillfully?
Shuzheng Si, Haozhe Zhao, Yu Lei +10
Many real-world tasks require language models (LMs) to reason over complex contexts that exceed their parametric knowledge. This calls for context learning, where LMs directly lear…
MENTOR: Efficient Multimodal-Conditioned Tuning for Autoregressive Vision Generation Models
Haozhe Zhao, Zefan Cai, Shuzheng Si +5
Recent text-to-image models produce high-quality results but still struggle with precise visual control, balancing multimodal inputs, and requiring extensive training for complex m…
Looking Beyond Text: Reducing Language bias in Large Vision-Language Models via Multimodal Dual-Attention and Soft-Image Guidance
Haozhe Zhao, Shuzheng Si, Liang Chen +4
Large vision-language models (LVLMs) have achieved impressive results in various vision-language tasks. However, despite showing promising performance, LVLMs suffer from hallucinat…
FaithLens: Detecting and Explaining Faithfulness Hallucination
Shuzheng Si, Qingyi Wang, Haozhe Zhao +8
Recognizing whether outputs from large language models (LLMs) contain faithfulness hallucination is crucial for real-world applications, e.g., retrieval-augmented generation and su…
A Goal Without a Plan Is Just a Wish: Efficient and Effective Global Planner Training for Long-Horizon Agent Tasks
Shuzheng Si, Haozhe Zhao, Kangyang Luo +5
Agents based on large language models (LLMs) struggle with brainless trial-and-error and generating hallucinatory actions due to a lack of global planning in long-horizon tasks. In…