6 papers
HiLo-Token: Input-Adaptive High-Low Frequency Token Compression for Efficient Image Editing
Haoran You, Yotam Nitzan, Lingzhi Zhang +7
Creative image editing tools, such as Photoshop's Remove or Generative Fill buttons, are central to everyday customer use and account for a major share of traffic in Photoshop and…
DTop-p MoE: Sparsity-Controlled Dynamic Top-p MoE for Foundation Model Pre-training
Can Jin, Hongwu Peng, Mingcan Xiang +7
Sparse Mixture-of-Experts architectures are essential for scaling model capacity efficiently, yet the standard Top- routing imposes a rigid sparsity pattern that ignores the int…
Complete-muE: Optimal Hyperparameter Transfer and Scaling for MoE Models
Hongwu Peng, Ohiremen Dibua, Yuanjun Xiong +3
We propose Complete-muE, a framework which targets hyperparameter transfer across dense FFN and any Mixture-of-Experts (MoE) setups in transformer blocks. Existing tools such as $Î…
DOLLAR: Few-Step Video Generation via Distillation and Latent Reward Optimization
Zihan Ding, Chi Jin, Difan Liu +6
Diffusion probabilistic models have shown significant progress in video generation; however, their computational efficiency is limited by the large number of sampling steps require…
Generating, Fast and Slow: Scalable Parallel Video Generation with Video Interface Networks
Bhishma Dedhia, David Bourgin, Krishna Kumar Singh +5
Diffusion Transformers (DiTs) can generate short photorealistic videos, yet directly training and sampling longer videos with full attention across the video remains computationall…
Layer- and Timestep-Adaptive Differentiable Token Compression Ratios for Efficient Diffusion Transformers
Haoran You, Connelly Barnes, Yuqian Zhou +10
Diffusion Transformers (DiTs) have achieved state-of-the-art (SOTA) image generation quality but suffer from high latency and memory inefficiency, making them difficult to deploy o…