5 papers
See Less, See Right: Bi-directional Perceptual Shaping For Multimodal Reasoning
Shuoshuo Zhang, Yizhen Zhang, Jingjing Fu +4
Large vision-language models (VLMs) often benefit from intermediate visual cues, either injected via external tools or generated as latent visual tokens during reasoning, but these…
PixelCraft: A Multi-Agent System for High-Fidelity Visual Reasoning on Structured Images
Shuoshuo Zhang, Zijian Li, Yizhen Zhang +6
Structured images (e.g., charts and geometric diagrams) remain challenging for multimodal large language models (MLLMs), as perceptual slips can cascade into erroneous conclusions.…
OMGM: Orchestrate Multiple Granularities and Modalities for Efficient Multimodal Retrieval
Wei Yang, Jingjing Fu, Rui Wang +3
Vision-language retrieval-augmented generation (RAG) has become an effective approach for tackling Knowledge-Based Visual Question Answering (KB-VQA), which requires external knowl…
Chain of Functions: A Programmatic Pipeline for Fine-Grained Chart Reasoning Data
Zijian Li, Jingjing Fu, Lei Song +3
Visual reasoning is crucial for multimodal large language models (MLLMs) to address complex chart queries, yet high-quality rationale data remains scarce. Existing methods leverage…
PIKE-RAG: sPecIalized KnowledgE and Rationale Augmented Generation
Jinyu Wang, Jingjing Fu, Rui Wang +2
Despite notable advancements in Retrieval-Augmented Generation (RAG) systems that expand large language model (LLM) capabilities through external retrieval, these systems often str…