Training-Inference Consistent Segmented Execution for Long-Context LLMs

Xianpeng Shang, Jiang Li, Zehua Duo +2

Transformer-based large language models face severe scalability challenges in long-context generation due to the computational and memory costs of full-context attention. Under pra…