4 papers
MLLM-DataEngine: Closing the Loop of Multimodal Instruction Tuning Data Generation
Zhiyuan Zhao, Bin Wang, Linke Ouyang +5
In this paper, we propose MLLM-DataEngine, a novel closed-loop system that bridges data generation, model training, and evaluation. Within each loop iteration, the MLLM-DataEngine…
A Hamiltonian-Inspired Local-Operator Ansatz for Slimming Large Language Models
Ying Lu, Peng-Fei Zhou, Qi-Xuan Fang +3
Dense linear maps carry much of the parameter and computational burden of modern neural networks, yet their dense form leaves the organization of learned couplings implicit. Quantu…
Strategic Over-Parameterization for Generalizable Low-Rank Adaptation
Jing Gao, Zhong-Yi Lu, Pan Zhang +1
Adapting large language models (LLMs) to downstream tasks via full fine-tuning is increasingly impractical due to its computational and memory demands. Parameter-efficient fine-tun…
InternLM-XComposer2.5-OmniLive: A Comprehensive Multimodal System for Long-term Streaming Video and Audio Interactions
Pan Zhang, Xiaoyi Dong, Yuhang Cao +26
Creating AI systems that can interact with environments over long periods, similar to human cognition, has been a longstanding research goal. Recent advancements in multimodal larg…