AdaDecode: Accelerating LLM Decoding with Adaptive Layer Parallelism

Zhepei Wei, Wei-Lin Chen, Xinyu Zhu +1

Large language models (LLMs) are increasingly used for long-content generation (e.g., long Chain-of-Thought reasoning) where decoding efficiency becomes a critical bottleneck: Auto…