6 papers
Devol-ONE: One Autoregressive Mixture of Transformers to Unify Vision-Language-Action and Latent World Modeling
Hongyi Cai, Yi Herng Ong, Tingshiuan C. Wu +5
Vision Language Action (VLA) models condition actions directly on current visual and language context, without an explicit account of how the scene evolves under candidate actions.…
OnceSelect: Reusable Data Selection for Efficient Multimodal Instruction Tuning
Mingkang Dong, Hongyi Cai, Xiwen Lei +8
Multimodal instruction tuning is widely used to adapt multimodal large language models (MLLMs), yet the large-scale image-text datasets it relies on are often highly redundant. Exi…
VisNec: Measuring and Leveraging Visual Necessity for Multimodal Instruction Tuning
Mingkang Dong, Hongyi Cai, Jie Li +4
The effectiveness of multimodal instruction tuning depends not only on dataset scale, but critically on whether training samples genuinely require visual reasoning. However, existi…
Evo-1: Lightweight Vision-Language-Action Model with Preserved Semantic Alignment
Tao Lin, Yilei Zhong, Yuxin Du +11
Vision-Language-Action (VLA) models have emerged as a powerful framework that unifies perception, language, and control, enabling robots to perform diverse tasks through multimodal…
Bridging the Semantic-Action Gap in Visual Token Pruning for Efficient VLA Inference
Ziyan Liu, Yeqiu Chen, Hongyi Cai +4
Vision-Language-Action (VLA) models have shown great potential for embodied AI by integrating visual perception, language understanding, and action execution. In real-time deployme…
MergeIT: From Selection to Merging for Efficient Instruction Tuning
Hongyi Cai, Yuqian Fu, Hongming Fu +1
Instruction tuning is crucial for optimizing Large Language Models (LLMs), yet mainstream data selection methods heavily rely on LLMs as instruction quality scorers, leading to hig…