5k citations · 8.4k across the 35 of their papers we have counts for
9 papers · 1 filter
Regularized Contrastive Learning of Semantic Search
Mingxi Tan, Alexis Rolland, Andong Tian
Semantic search is an important task which objective is to find the relevant index from a database for query. It requires a retrieval model that can properly learn the semantics of…
Combined Scaling for Zero-shot Transfer Learning
Hieu Pham, Zihang Dai, Golnaz Ghiasi +9
We present a combined scaling method - named BASIC - that achieves 85.7% top-1 accuracy on the ImageNet ILSVRC-2012 validation set without learning from any labeled ImageNet exampl…
Rethinking Co-design of Neural Architectures and Hardware Accelerators
Yanqi Zhou, Xuanyi Dong, Berkin Akin +7
Neural architectures and hardware accelerators have been two driving forces for the progress in deep learning. Previous works typically attempt to optimize hardware given a fixed m…
PyGlove: Symbolic Programming for Automated Machine Learning
Daiyi Peng, Xuanyi Dong, Esteban Real +7
Neural networks are sensitive to hyper-parameter and architecture choices. Automated Machine Learning (AutoML) is a promising paradigm for automating these choices. Current ML soft…
Training EfficientNets at Supercomputer Scale: 83% ImageNet Top-1 Accuracy in One Hour
Arissa Wongpanich, Hieu Pham, James Demmel +4
EfficientNets are a family of state-of-the-art image classification models based on efficiently scaled convolutional neural networks. Currently, EfficientNets can take on the order…
Go Wide, Then Narrow: Efficient Training of Deep Thin Networks
Denny Zhou, Mao Ye, Chen Chen +6
For deploying a deep learning model into production, it needs to be both accurate and compact to meet the latency and memory constraints. This usually results in a network that is…
Smooth Adversarial Training
Cihang Xie, Mingxing Tan, Boqing Gong +2
It is commonly believed that networks cannot be both accurate and robust, that gaining robustness means losing accuracy. It is also generally believed that, unless making networks…
When Ensembling Smaller Models is More Efficient than Single Large Models
Dan Kondratyuk, Mingxing Tan, Matthew Brown +1
Ensembling is a simple and popular technique for boosting evaluation performance by training multiple models (e.g., with different initializations) and aggregating their prediction…
EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks
Mingxing Tan, Quoc V. Le
Convolutional Neural Networks (ConvNets) are commonly developed at a fixed resource budget, and then scaled up for better accuracy if more resources are available. In this paper, w…