Streaming, Fast and Slow: Cognitive Load-Aware Streaming for Efficient LLM Serving

Chang Xiao, Brenda Yang

Generative conversational interfaces powered by large language models (LLMs) typically stream output token-by-token at a rate determined by computational budget, often neglecting a…