4 papers
Variance-Adaptive Muon: Accelerating LLM Pretraining with NSR-Modulated and Variance-Scaled Momentum
Jingru Li, Yibo Fan, Huan Li
Large Language Models (LLMs) achieve competitive performance across diverse natural language processing (NLP) tasks, yet pretraining is computationally demanding, making optimizer…
GRACE: Designing Generative Face Video Codec via Agile Hardware-Centric Workflow
Rui Wan, Qi Zheng, Ruoyu Zhang +6
The Animation-based Generative Codec (AGC) is an emerging paradigm for talking-face video compression. However, deploying its intricate decoder on resource and power-constrained ed…
DiTFastAttnV2: Head-wise Attention Compression for Multi-Modality Diffusion Transformers
Hanling Zhang, Rundong Su, Zhihang Yuan +5
Text-to-image generation models, especially Multimodal Diffusion Transformers (MMDiT), have shown remarkable progress in generating high-quality images. However, these models often…
Unicorn: Unified Neural Image Compression with One Number Reconstruction
Qi Zheng, Haozhi Wang, Zihao Liu +8
Prevalent lossy image compression schemes can be divided into: 1) explicit image compression (EIC), including traditional standards and neural end-to-end algorithms; 2) implicit im…