5 papers
KVServe: Service-Aware KV Cache Compression for Communication-Efficient Disaggregated LLM Serving
Zedong Liu, Xinyang Ma, Dejun Luo +9
LLMs are widely adopted in production, pushing inference systems to their limits. Disaggregated LLM serving (e.g., PD separation and KV state disaggregation) improves scalability a…
FIS-DiT: Breaking the Few-Step Video Inference Barrier via Training-Free Frame Interleaved Sparsity
Jian Tang, Jiawei Fan, Qingbin Liu +1
While the overall inference latency of Video Diffusion Transformers (DiTs) can be substantially reduced through model distillation, per-step inference latency remains a critical bo…
TACO: Efficient Communication Compression of Intermediate Tensors for Scalable Tensor-Parallel LLM Training
Man Liu, Xingchen Liu, Xingjian Tian +8
Handling communication overhead in large-scale tensor-parallel training remains a critical challenge due to the dense, near-zero distributions of intermediate tensors, which exacer…
ENEC: A Lossless AI Model Compression Method Enabling Fast Inference on Ascend NPUs
Jinwu Yang, Jiaan Wu, Zedong Liu +17
The rapid scaling of Large Language Models presents significant challenges for their deployment and inference, particularly on resource-constrained specialized AI hardware accelera…
TSUE: A Two-Stage Data Update Method for an Erasure Coded Cluster File System
Zheng Wei, Jing Xing, Yida Gu +4
Compared to replication-based storage systems, erasure-coded storage incurs significantly higher overhead during data updates. To address this issue, various parity logging methods…