From the 1 of 4 linked papers with an AI index.
4 papers
Speculate with Memory: Lossless Acceleration for LLM Agents
Yu Li, Qinyuan Ye, Prafulla Kumar Choubey +2
The paper proposes adding online memory systems to speculative execution for large language model agents, enabling the speculator to learn from past trajectories and improve predic…
The Illusion of Certainty: Decoupling Capability and Calibration in On-Policy Distillation
Jiaxin Zhang, Xiangyu Peng, Qinglin Chen +3
On-policy distillation (OPD) is an increasingly important paradigm for post-training language models. However, we identify a pervasive Scaling Law of Miscalibration: while OPD effe…
Function Induction and Task Generalization: An Interpretability Study with Off-by-One Addition
Qinyuan Ye, Robin Jia, Xiang Ren
Large language models demonstrate the intriguing ability to perform unseen tasks via in-context learning. However, it remains unclear what mechanisms inside the model drive such ta…
Stress-Testing Long-Context Language Models with Lifelong ICL and Task Haystack
Xiaoyue Xu, Qinyuan Ye, Xiang Ren
We introduce Lifelong ICL, a problem setting that challenges long-context language models (LMs) to learn a sequence of language tasks through in-context learning (ICL). We further…