6 papers
Speculative Decoding in Decentralized LLM Inference: Turning Communication Latency into Computation Throughput
Jingwei Song, Wanyi Chen, Xinyuan Song +7
Speculative decoding accelerates large language model (LLM) inference by using a lightweight draft model to propose tokens that are later verified by a stronger target model. While…
Multi-Agent Collaborative Reward Design for Enhancing Reasoning in Reinforcement Learning
Pei Yang, Ke Zhang, Ji Wang +5
We present CRM (Multi-Agent Collaborative Reward Model), a framework that replaces a single black-box reward model with a coordinated team of specialist evaluators to improve robus…
Lattica: A Decentralized Cross-NAT Communication Framework for Scalable AI Inference and Training
Ween Yang, Jason Liu, Suli Wang +4
The rapid expansion of distributed Artificial Intelligence (AI) workloads beyond centralized data centers creates a demand for new communication substrates. These substrates must o…
SEDM: Scalable Self-Evolving Distributed Memory for Agents
Haoran Xu, Jiacong Hu, Ke Zhang +6
Long-term multi-agent systems inevitably generate vast amounts of trajectories and historical interactions, which makes efficient memory management essential for both performance a…
VeriLLM: A Lightweight Framework for Publicly Verifiable Decentralized Inference
Ke Wang, Zishuo Zhao, Xinyuan Song +7
Decentralized inference provides a scalable and resilient paradigm for serving large language models (LLMs), enabling fragmented global resource utilization and reducing reliance o…
Symphony: A Decentralized Multi-Agent Framework for Scalable Collective Intelligence
Ji Wang, Kashing Chen, Xinyuan Song +4
Most existing Large Language Model (LLM)-based agent frameworks rely on centralized orchestration, incurring high deployment costs, rigid communication topologies, and limited adap…