5 papers
VisNec: Measuring and Leveraging Visual Necessity for Multimodal Instruction Tuning
Mingkang Dong, Hongyi Cai, Jie Li +4
The effectiveness of multimodal instruction tuning depends not only on dataset scale, but critically on whether training samples genuinely require visual reasoning. However, existi…
Once-For-All: A Train-Once and Select-Anytime Framework for Multimodal Instruction Tuning
Mingkang Dong, Hongyi Cai, Xiwen Lei +3
Multimodal instruction tuning is the de facto recipe for adapting vision language models (VLMs), yet instruction data are highly redundant, making data selection critical for train…
Bridging the Semantic-Action Gap in Visual Token Pruning for Efficient VLA Inference
Ziyan Liu, Yeqiu Chen, Hongyi Cai +4
Vision-Language-Action (VLA) models have shown great potential for embodied AI by integrating visual perception, language understanding, and action execution. In real-time deployme…
Evo-1: Lightweight Vision-Language-Action Model with Preserved Semantic Alignment
Tao Lin, Yilei Zhong, Yuxin Du +11
Vision-Language-Action (VLA) models have emerged as a powerful framework that unifies perception, language, and control, enabling robots to perform diverse tasks through multimodal…
MergeIT: From Selection to Merging for Efficient Instruction Tuning
Hongyi Cai, Yuqian Fu, Hongming Fu +1
Instruction tuning is crucial for optimizing Large Language Models (LLMs), yet mainstream data selection methods heavily rely on LLMs as instruction quality scorers, leading to hig…