8 papers
Agentic ESOpt: Fine-Tuning Long-Horizon LLM Agents with Minimal GPU Requirements
Zhi Zheng, Rongsheng Chen, Yunpeng Ba +3
Reinforcement Learning (RL) has been promising in single-turn LLM fine-tuning. However, long-horizon agentic reasoning introduces increasingly branching interactions and sparse rew…
One Token per Multimodal Evidence: Latent Memory for Resource-Constrained QA
Zhi Zheng, Ziqiao Meng, Hao Luan +2
External memory effectively grounds large language models (LLMs) and vision-language models (VLMs)-based question answering (QA) in relevant multimodal evidence. However, existing…
Rethinking the Divergence Regularization in LLM RL
Jiarui Yao, Xiangxin Zhou, Penghui Qi +3
Reinforcement learning (RL) has become a key component of post-training large language models (LLMs). In practice, LLM RL is often off-policy because of training-inference mismatch…
From Backward Spreading to Forward Replay: Revisiting Target Construction in LLM Parameter Editing
Wei Liu, Hongkai Liu, Zhiying Deng +2
LLM parameter editing methods commonly rely on computing an ideal target hidden-state at a target layer (referred as anchor point) and distributing the target vector to multiple pr…
LinTree: Improving LLM Reasoning with Explicitly Structured Search Histories
Liwei Kang, Yee Whye Teh, Wee Sun Lee
Large language models (LLMs) often solve reasoning problems by generating intermediate traces that explore and revise partial solutions. From a search perspective, these traces can…
Are We Evaluating the Edit Locality of LLM Model Editing Properly?
Wei Liu, Haomei Xu, Hongkai Liu +5
Model editing has recently emerged as a popular paradigm for efficiently updating knowledge in LLMs. A central desideratum of updating knowledge is to balance editing efficacy, i.e…
Is Model Editing Built on Sand? Revealing Its Illusory Success and Fragile Foundation
Wei Liu, Haomei Xu, Bingqing Liu +6
Large language models (LLMs) inevitably encode outdated or incorrect knowledge. Updating, deleting, and forgetting such knowledge is important for alignment, safety, and other issu…
Extending Epistemic Uncertainty Beyond Parameters Would Assist in Designing Reliable LLMs
T. Duy Nguyen-Hien, Desi R. Ivanova, Yee Whye Teh +1
Although large language models (LLMs) are highly interactive and extendable, current approaches to ensure reliability in deployments remain mostly limited to rejecting outputs with…