6 papers
SERE: Similarity-based Expert Re-routing for Efficient Batch Decoding in MoE Models
Juntong Wu, Jialiang Cheng, Fuyu Lv +2
Mixture-of-Experts (MoE) architectures employ sparse activation to deliver faster training and inference with higher accuracy than dense LLMs. However, in production serving, MoE m…
Quant VideoGen: Auto-Regressive Long Video Generation via 2-Bit KV-Cache Quantization
Haocheng Xi, Shuo Yang, Yilong Zhao +13
Despite rapid progress in autoregressive video diffusion, an emerging system algorithm bottleneck limits both deployability and generation capability: KV cache memory. In autoregre…
Diving into 3D Parallelism with Heterogeneous Spot Instance GPUs: Design and Implications
Yuxiao Wang, Yuedong Xu, Qingyang Duan +4
The rapid growth of large language models (LLMs) and the continuous release of new GPU products have significantly increased the demand for distributed training across heterogeneou…
PrivDFS: Private Inference via Distributed Feature Sharing against Data Reconstruction Attacks
Zihan Liu, Jiayi Wen, Junru Wu +4
In this paper, we introduce PrivDFS, a distributed feature-sharing framework for input-private inference in image classification. A single holistic intermediate representation in s…
It Takes a Good Model to Train a Good Model: Generalized Gaussian Priors for Optimized LLMs
Jun Wu, Patrick Huang, Jiangtao Wen +1
Despite rapid progress in large language models (LLMs), the statistical structure of their weights, activations, and gradients-and its implications for initialization, training dyn…
BackSlash: Rate Constrained Optimized Training of Large Language Models
Jun Wu, Jiangtao Wen, Yuxing Han
The rapid advancement of large-language models (LLMs) has driven extensive research into parameter compression after training has been completed, yet compression during the trainin…