5 papers · 1 filter
Quant VideoGen: Auto-Regressive Long Video Generation via 2-Bit KV-Cache Quantization
Haocheng Xi, Shuo Yang, Yilong Zhao +13
Despite rapid progress in autoregressive video diffusion, an emerging system algorithm bottleneck limits both deployability and generation capability: KV cache memory. In autoregre…
It Takes a Good Model to Train a Good Model: Generalized Gaussian Priors for Optimized LLMs
Jun Wu, Patrick Huang, Jiangtao Wen +1
Despite rapid progress in large language models (LLMs), the statistical structure of their weights, activations, and gradients-and its implications for initialization, training dyn…
SERE: Similarity-based Expert Re-routing for Efficient Batch Decoding in MoE Models
Juntong Wu, Jialiang Cheng, Fuyu Lv +2
Mixture-of-Experts (MoE) architectures employ sparse activation to deliver faster training and inference with higher accuracy than dense LLMs. However, in production serving, MoE m…
PrivDFS: Private Inference via Distributed Feature Sharing against Data Reconstruction Attacks
Zihan Liu, Jiayi Wen, Junru Wu +4
In this paper, we introduce PrivDFS, a distributed feature-sharing framework for input-private inference in image classification. A single holistic intermediate representation in s…
BackSlash: Rate Constrained Optimized Training of Large Language Models
Jun Wu, Jiangtao Wen, Yuxing Han
The rapid advancement of large-language models (LLMs) has driven extensive research into parameter compression after training has been completed, yet compression during the trainin…