11 papers
Unlocking Parallelism in Autoregressive Language Models via Speculative Decoding with Progressive Tree Drafting
Zipeng Gao, Zhi Zheng, Qingrong Xia +5
Speculative decoding has significantly accelerated Large Language Model (LLM) inference by alleviating memory-bound bottlenecks. However, traditional speculative decoding typically…
From Blueprint to Reality: Modeling and Applying Putnam's Social Capital Theory with LLM-based Multi-agent Simulations
Shiyi Ling, Zhi Zheng, Hui Zheng +3
Putnam's Social Capital Theory is a foundational framework for collective action and community prosperity. However, traditional empirical methods face practical limits on control a…
AtomMem: Building Simple and Effective Memory System for LLM Agents via Atomic Facts
Yanyu Yao, Shangze Li, Zhi Zheng +4
Large language models (LLMs) demonstrate strong reasoning and generation abilities, but their fixed context windows limit long-term information accumulation and reuse across multi-…
DynaDebate: Breaking Homogeneity in Multi-Agent Debate with Dynamic Path Generation
Zhenghao Li, Zhi Zheng, Wei Chen +4
Recent years have witnessed the rapid development of Large Language Model-based Multi-Agent Systems (MAS), which excel at collaborative decision-making and complex problem-solving.…
SmellBench: Towards Fine-Grained Evaluation of Code Agents on Refactoring Tasks
Fake Lin, Binbin Hu, Xi Zhu +6
Code Agents have achieved remarkable advances in recent years, exhibiting strong capabilities across a wide range of software engineering tasks. However, their misuse often produce…
Entropy-KL Divergence-based Token Masking: A Novel Approach for Selective Fine-tuning of Large Language Models
Qi Liu, Mingdi Sun, Yongyi He +5
Supervised fine-tuning (SFT) followed by reinforcement learning (RL) has become a standard post-training paradigm for large language models. This paradigm provides a cold-start for…