116 citations · 231 across the 8 of their papers we have counts for
8 papers
Prioritized Subnet Sampling for Resource-Adaptive Supernet Training
Bohong Chen, Mingbao Lin, Rongrong Ji +1
A resource-adaptive supernet adjusts its subnets for inference to fit the dynamically available resources. In this paper, we propose prioritized subnet sampling to train a resource…
DPT: Deformable Patch-based Transformer for Visual Recognition
Zhiyang Chen, Yousong Zhu, Chaoyang Zhao +4
Transformer has achieved great success in computer vision, while how to split patches in an image remains a problem. Existing methods usually use a fixed-size patch embedding which…
Training Compact CNNs for Image Classification using Dynamic-coded Filter Fusion
Mingbao Lin, Bohong Chen, Fei Chao +1
The mainstream approach for filter pruning is usually either to force a hard-coded importance estimation upon a computation-heavy pretrained model to select "important" filters, or…
PanGu-: Large-scale Autoregressive Pretrained Chinese Language Models with Auto-parallel Computation
Wei Zeng, Xiaozhe Ren, Teng Su +35
Large-scale Pretrained Language Models (PLMs) have become the new paradigm for Natural Language Processing (NLP). PLMs with hundreds of billions parameters such as GPT-3 have demon…
Contrastive Neural Architecture Search with Neural Architecture Comparators
Yaofo Chen, Yong Guo, Qi Chen +4
One of the key steps in Neural Architecture Search (NAS) is to estimate the performance of candidate architectures. Existing methods either directly use the validation performance…
Adaptive Class Suppression Loss for Long-Tail Object Detection
Tong Wang, Yousong Zhu, Chaoyang Zhao +3
To address the problem of long-tail distribution for the large vocabulary object detection task, existing methods usually divide the whole categories into several groups and treat…
The distance between the weights of the neural network is meaningful
Liqun Yang, Yijun Yang, Yao Wang +2
In the application of neural networks, we need to select a suitable model based on the problem complexity and the dataset scale. To analyze the network's capacity, quantifying the…
Learning long-term dependencies for action recognition with a biologically-inspired deep network
Yemin Shi, Yonghong Tian, Yaowei Wang +1
Despite a lot of research efforts devoted in recent years, how to efficiently learn long-term dependencies from sequences still remains a pretty challenging task. As one of the key…