14 citations · 39 across the 11 of their papers we have counts for
7 papers · 1 filter
TVT: Training-Free Vision Transformer Search on Tiny Datasets
Zimian Wei, Hengyue Pan, Lujun Li +4
Training-free Vision Transformer (ViT) architecture search is presented to search for a better ViT with zero-cost proxies. While ViTs achieve significant distillation gains from CN…
EMQ: Evolving Training-free Proxies for Automated Mixed Precision Quantization
Peijie Dong, Lujun Li, Zimian Wei +3
Mixed-Precision Quantization~(MQ) can achieve a competitive accuracy-complexity trade-off for models. Conventional training-based search methods require time-consuming candidate tr…
Progressive Meta-Pooling Learning for Lightweight Image Classification Model
Peijie Dong, Xin Niu, Zhiliang Tian +5
Practical networks for edge devices adopt shallow depth and small convolutional kernels to save memory and computational cost, which leads to a restricted receptive field. Conventi…
RD-NAS: Enhancing One-shot Supernet Ranking Ability via Ranking Distillation from Zero-cost Proxies
Peijie Dong, Xin Niu, Lujun Li +5
Neural architecture search (NAS) has made tremendous progress in the automatic design of effective neural network structures but suffers from a heavy computational burden. One-shot…
OVO: One-shot Vision Transformer Search with Online distillation
Zimian Wei, Hengyue Pan, Xin Niu +1
Pure transformers have shown great potential for vision tasks recently. However, their accuracy in small or medium datasets is not satisfactory. Although some existing methods intr…
DMFormer: Closing the Gap Between CNN and Vision Transformers
Zimian Wei, Hengyue Pan, Lujun Li +4
Vision transformers have shown excellent performance in computer vision tasks. As the computation cost of their self-attention mechanism is expensive, recent works tried to replace…
Prior-Guided One-shot Neural Architecture Search
Peijie Dong, Xin Niu, Lujun Li +5
Neural architecture search methods seek optimal candidates with efficient weight-sharing supernet training. However, recent studies indicate poor ranking consistency about the perf…