8 papers
One Token per Multimodal Evidence: Latent Memory for Resource-Constrained QA
Zhi Zheng, Ziqiao Meng, Hao Luan +2
External memory effectively grounds large language models (LLMs) and vision-language models (VLMs)-based question answering (QA) in relevant multimodal evidence. However, existing…
Rethinking the Divergence Regularization in LLM RL
Jiarui Yao, Xiangxin Zhou, Penghui Qi +3
Reinforcement learning (RL) has become a key component of post-training large language models (LLMs). In practice, LLM RL is often off-policy because of training-inference mismatch…
From Backward Spreading to Forward Replay: Revisiting Target Construction in LLM Parameter Editing
Wei Liu, Hongkai Liu, Zhiying Deng +2
LLM parameter editing methods commonly rely on computing an ideal target hidden-state at a target layer (referred as anchor point) and distributing the target vector to multiple pr…
Can Vision-Language Models Solve the Shell Game?
Tiedong Liu, Wee Sun Lee
Visual entity tracking is an innate cognitive ability in humans, yet it remains a critical bottleneck for Vision-Language Models (VLMs). This deficit is often obscured in existing…
GEM: A Gym for Agentic LLMs
Zichen Liu, Anya Sims, Keyu Duan +16
The training paradigm for large language models (LLMs) is moving from static datasets to experience-based learning, where agents acquire skills via interacting with complex environ…
Are We Evaluating the Edit Locality of LLM Model Editing Properly?
Wei Liu, Haomei Xu, Hongkai Liu +5
Model editing has recently emerged as a popular paradigm for efficiently updating knowledge in LLMs. A central desideratum of updating knowledge is to balance editing efficacy, i.e…