15 citations · 28 across the 10 of their papers we have counts for
4 papers · 1 filter
Breaking the Compression Ceiling: Data-Free Pipeline for Ultra-Efficient Delta Compression
Xiaohui Wang, Peng Ye, Chenyu Huang +5
With the rise of the fine-tuned-pretrained paradigm, storing numerous fine-tuned models for multi-tasking creates significant storage overhead. Delta compression alleviates this by…
DeRS: Towards Extremely Efficient Upcycled Mixture-of-Experts Models
Yongqi Huang, Peng Ye, Chenyu Huang +5
Upcycled Mixture-of-Experts (MoE) models have shown great potential in various tasks by converting the original Feed-Forward Network (FFN) layers in pre-trained dense models into M…
EMR-Merging: Tuning-Free High-Performance Model Merging
Chenyu Huang, Peng Ye, Tao Chen +3
The success of pretrain-finetune paradigm brings about the release of numerous model weights. In this case, merging models finetuned on different tasks to enable a single model wit…
-DARTS: Beta-Decay Regularization for Differentiable Architecture Search
Peng Ye, Baopu Li, Yikang Li +3
Neural Architecture Search~(NAS) has attracted increasingly more attention in recent years because of its capability to design deep neural networks automatically. Among them, diffe…