Staircase Streaming for Low-Latency Multi-Agent Inference

Junlin Wang, Jue Wang, Zhen +5

Recent advances in large language models (LLMs) opened up new directions for leveraging the collective expertise of multiple LLMs. These methods, such as Mixture-of-Agents, typical…