4 papers
TAP: A Token-Adaptive Predictor Framework for Training-Free Diffusion Acceleration
Haowei Zhu, Tingxuan Huang, Xing Wang +7
Diffusion models achieve strong generative performance but remain slow at inference due to the need for repeated full-model denoising passes. We present Token-Adaptive Predictor (T…
Mixture-of-Channels: Exploiting Sparse FFNs for Efficient LLMs Pre-Training and Inference
Tong Wu, Yutong He, Bin Wang +1
Large language models (LLMs) have demonstrated remarkable success across diverse artificial intelligence tasks, driven by scaling laws that correlate model size and training data w…
Clapping: Removing Per-sample Storage for Pipeline Parallel Distributed Optimization with Communication Compression
Boao Kong, Xu Huang, Yuqi Xu +3
Pipeline-parallel distributed optimization is essential for large-scale machine learning but is challenged by significant communication overhead from transmitting high-dimensional…
Distributed Bilevel Optimization with Communication Compression
Yutong He, Jie Hu, Xinmeng Huang +3
Stochastic bilevel optimization tackles challenges involving nested optimization structures. Its fast-growing scale nowadays necessitates efficient distributed algorithms. In conve…