TPLA: Tensor Parallel Latent Attention for Efficient Disaggregated Prefill and Decode Inference

Xiaojuan Tang, Fanxu Meng, Pingzhi Tang +4

Multi-Head Latent Attention (MLA), introduced in DeepSeek-V2, compresses key-value states into a low-rank latent vector, caching only this vector to reduce memory. In tensor parall…