11 citations · 40 across the 11 of their papers we have counts for
11 papers
ParZC: Parametric Zero-Cost Proxies for Efficient NAS
Peijie Dong, Lujun Li, Xinglin Pan +4
Recent advancements in Zero-shot Neural Architecture Search (NAS) highlight the efficacy of zero-cost proxies in various NAS benchmarks. Several studies propose the automated desig…
Auto-Prox: Training-Free Vision Transformer Architecture Search via Automatic Proxy Discovery
Zimian Wei, Lujun Li, Peijie Dong +6
The substantial success of Vision Transformer (ViT) in computer vision tasks is largely attributed to the architecture design. This underscores the necessity of efficient architect…
TVT: Training-Free Vision Transformer Search on Tiny Datasets
Zimian Wei, Hengyue Pan, Lujun Li +4
Training-free Vision Transformer (ViT) architecture search is presented to search for a better ViT with zero-cost proxies. While ViTs achieve significant distillation gains from CN…
EMQ: Evolving Training-free Proxies for Automated Mixed Precision Quantization
Peijie Dong, Lujun Li, Zimian Wei +3
Mixed-Precision Quantization~(MQ) can achieve a competitive accuracy-complexity trade-off for models. Conventional training-based search methods require time-consuming candidate tr…
DisWOT: Student Architecture Search for Distillation WithOut Training
Peijie Dong, Lujun Li, Zimian Wei
Knowledge distillation (KD) is an effective training strategy to improve the lightweight student models under the guidance of cumbersome teachers. However, the large architecture d…
Progressive Meta-Pooling Learning for Lightweight Image Classification Model
Peijie Dong, Xin Niu, Zhiliang Tian +5
Practical networks for edge devices adopt shallow depth and small convolutional kernels to save memory and computational cost, which leads to a restricted receptive field. Conventi…
RD-NAS: Enhancing One-shot Supernet Ranking Ability via Ranking Distillation from Zero-cost Proxies
Peijie Dong, Xin Niu, Lujun Li +5
Neural architecture search (NAS) has made tremendous progress in the automatic design of effective neural network structures but suffers from a heavy computational burden. One-shot…
OVO: One-shot Vision Transformer Search with Online distillation
Zimian Wei, Hengyue Pan, Xin Niu +1
Pure transformers have shown great potential for vision tasks recently. However, their accuracy in small or medium datasets is not satisfactory. Although some existing methods intr…
DMFormer: Closing the Gap Between CNN and Vision Transformers
Zimian Wei, Hengyue Pan, Lujun Li +4
Vision transformers have shown excellent performance in computer vision tasks. As the computation cost of their self-attention mechanism is expensive, recent works tried to replace…
Prior-Guided One-shot Neural Architecture Search
Peijie Dong, Xin Niu, Lujun Li +5
Neural architecture search methods seek optimal candidates with efficient weight-sharing supernet training. However, recent studies indicate poor ranking consistency about the perf…
Multi-trial Neural Architecture Search with Lottery Tickets
Zimian Wei, Hengyue Pan, Lujun Li +4
Neural architecture search (NAS) has brought significant progress in recent image recognition tasks. Most existing NAS methods apply restricted search spaces, which limits the uppe…