7 papers · 1 filter
OnlineCache: Learning Dynamic Caching Policies with Error Correction for Efficient Diffusion Inference
Zhikang Xie, Xichen Ye, Yifan Wu +5
Diffusion models have revolutionized generative tasks but incur high latency due to iterative denoising. While cache-based strategies accelerate inference by reusing intermediate f…
Distill on a Diet: Efficient Knowledge Distillation via Learnable Data Pruning
Yifan Wu, Yiqi Wang, Xichen Ye +5
Knowledge Distillation (KD) is widely used to obtain compact models for efficient inference in resource-constrained environments. Yet the computational overhead of the distillation…
Holistic Scaling Laws for Optimal Mixture-of-Experts Architecture Optimization
Weilin Wan, Jingtao Han, Weizhong Zhang +1
Scaling laws for Large Language Models govern macroscopic resource allocation, yet translating them into precise Mixture-of-Experts (MoE) architectural configurations remains an op…
Towards Robust Influence Functions with Flat Validation Minima
Xichen Ye, Yifan Wu, Weizhong Zhang +2
The Influence Function (IF) is a widely used technique for assessing the impact of individual training samples on model predictions. However, existing IF methods often fail to prov…
Explore and Establish Synergistic Effects Between Weight Pruning and Coreset Selection in Neural Network Training
Weilin Wan, Fan Yi, Weizhong Zhang +2
Modern deep neural networks rely heavily on massive model weights and training samples, incurring substantial computational costs. Weight pruning and coreset selection are two emer…
Computational Budget Should Be Considered in Data Selection
Weilin Wan, Weizhong Zhang, Cheng Jin
Data selection improves computational efficiency by choosing informative subsets of training samples. However, existing methods ignore the compute budget, treating data selection a…