5 papers
ProMSA:Progressive Multimodal Search Agents for Knowledge-Based Visual Question Answering
ZhengXian Wu, Hangrui Xu, Kai Shi +8
Knowledge-based Visual Question Answering (KB-VQA) requires models to combine image understanding with external knowledge. Most prior methods use a fixed retrieve-then-generate pip…
When Models Judge Themselves: Unsupervised Self-Evolution for Multimodal Reasoning
Zhengxian Wu, Kai Shi, Chuanrui Zhang +10
Recent progress in multimodal large language models has led to strong performance on reasoning tasks, but these improvements largely rely on high-quality annotated data or teacher-…
DaMo: Data Mixing Optimizer in Fine-tuning Multimodal LLMs for Mobile Phone Agents
Kai Shi, Jun Yang, Ni Yang +6
Mobile Phone Agents (MPAs) have emerged as a promising research direction due to their broad applicability across diverse scenarios. While Multimodal Large Language Models (MLLMs)…
Efficient Agent: Optimizing Planning Capability for Multimodal Retrieval Augmented Generation
Yuechen Wang, Yuming Qiao, Dan Meng +4
Multimodal Retrieval-Augmented Generation (mRAG) has emerged as a promising solution to address the temporal limitations of Multimodal Large Language Models (MLLMs) in real-world s…
ReviewInstruct: A Review-Driven Multi-Turn Conversations Generation Method for Large Language Models
Jiangxu Wu, Cong Wang, TianHuang Su +10
The effectiveness of large language models (LLMs) in conversational AI is hindered by their reliance on single-turn supervised fine-tuning (SFT) data, which limits contextual coher…