79 citations · 140 across the 12 of their papers we have counts for
14 papers
EMQ: Evolving Training-free Proxies for Automated Mixed Precision Quantization
Peijie Dong, Lujun Li, Zimian Wei +3
Mixed-Precision Quantization~(MQ) can achieve a competitive accuracy-complexity trade-off for models. Conventional training-based search methods require time-consuming candidate tr…
NORM: Knowledge Distillation via N-to-One Representation Matching
Xiaolong Liu, Lujun Li, Chao Li +1
Existing feature distillation methods commonly adopt the One-to-one Representation Matching between any pre-selected teacher-student layer pair. In this paper, we present N-to-One…
Catch-Up Distillation: You Only Need to Train Once for Accelerating Sampling
Shitong Shao, Xu Dai, Lujun Li +3
Diffusion Probability Models (DPMs) have made impressive advancements in various machine learning domains. However, achieving high-quality synthetic samples typically involves perf…
DisWOT: Student Architecture Search for Distillation WithOut Training
Peijie Dong, Lujun Li, Zimian Wei
Knowledge distillation (KD) is an effective training strategy to improve the lightweight student models under the guidance of cumbersome teachers. However, the large architecture d…
Progressive Meta-Pooling Learning for Lightweight Image Classification Model
Peijie Dong, Xin Niu, Zhiliang Tian +5
Practical networks for edge devices adopt shallow depth and small convolutional kernels to save memory and computational cost, which leads to a restricted receptive field. Conventi…
RD-NAS: Enhancing One-shot Supernet Ranking Ability via Ranking Distillation from Zero-cost Proxies
Peijie Dong, Xin Niu, Lujun Li +5
Neural architecture search (NAS) has made tremendous progress in the automatic design of effective neural network structures but suffers from a heavy computational burden. One-shot…