18 papers
Unlocking Parallelism in Autoregressive Language Models via Speculative Decoding with Progressive Tree Drafting
Zipeng Gao, Zhi Zheng, Qingrong Xia +5
Speculative decoding has significantly accelerated Large Language Model (LLM) inference by alleviating memory-bound bottlenecks. However, traditional speculative decoding typically…
DynaDebate: Breaking Homogeneity in Multi-Agent Debate with Dynamic Path Generation
Zhenghao Li, Zhi Zheng, Wei Chen +4
Recent years have witnessed the rapid development of Large Language Model-based Multi-Agent Systems (MAS), which excel at collaborative decision-making and complex problem-solving.…
Entropy-KL Divergence-based Token Masking: A Novel Approach for Selective Fine-tuning of Large Language Models
Qi Liu, Mingdi Sun, Yongyi He +5
Supervised fine-tuning (SFT) followed by reinforcement learning (RL) has become a standard post-training paradigm for large language models. This paradigm provides a cold-start for…
Defending LLM-based Multi-Agent Systems Against Cooperative Attacks with Sentence-Level Rectification
Yaoyang Luo, Zhi Zheng, Ziwei Zhao +5
Recent years have witnessed the rapid development of Large Language Model-based Multi-Agent Systems (MAS), which excel at collaborative decision-making and complex problem-solving.…
PERMA: Benchmarking Personalized Memory Agents via Event-Driven Preference and Realistic Task Environments
Shuochen Liu, Junyi Zhu, Long Shu +11
Empowering large language models with long-term memory is crucial for building agents that adapt to users' evolving needs. Existing evaluations of this capability typically interle…
More Edits, More Stable: Understanding the Lifelong Normalization in Sequential Model Editing
Xin Ma, Wei Chen, Qi Liu +4
Lifelong Model Editing aims to continuously update evolving facts in Large Language Models while preserving unrelated knowledge and general capabilities, yet it remains plagued by…