4 papers
Offline-Online Curriculum RL for Multimodal Reasoning
Wendi Deng, Hang Du, Guoshun Nan +11
Multimodal large language models exhibit capabilities on reasoning tasks, yet often produce flawed intermediate steps while yielding correct final answers. This behavior undermines…
Covert Visual Prompt Injection against Commercial Multimodal Large Language Models
Meiwen Ding, Song Xia, Chenqi Kong +1
Although multimodal large language models (MLLMs) are increasingly deployed in real-world applications, their instruction-following behavior leaves them vulnerable to prompt inject…
Advancing Expert Specialization for Better MoE
Hongcan Guo, Haolang Lu, Guoshun Nan +8
Mixture-of-Experts (MoE) models enable efficient scaling of large language models (LLMs) by activating only a subset of experts per input. However, we observe that the commonly use…
Two Is Better Than One: Rotations Scale LoRAs
Hongcan Guo, Guoshun Nan, Yuan Yang +9
Scaling Low-Rank Adaptation (LoRA)-based Mixture-of-Experts (MoE) facilitates large language models (LLMs) to efficiently adapt to diverse tasks. However, traditional gating mechan…