1.2k citations · 3.3k across the 19 of their papers we have counts for
9 papers · 1 filter
Improving Time Series Encoding with Noise-Aware Self-Supervised Learning and an Efficient Encoder
Duy A. Nguyen, Trang H. Tran, Huy Hieu Pham +2
In this work, we investigate the time series representation learning problem using self-supervised techniques. Contrastive learning is well-known in this area as it is a powerful m…
Symbolic Discovery of Optimization Algorithms
Xiangning Chen, Chen Liang, Da Huang +9
We present a method to formulate algorithm discovery as program search, and apply it to discover optimization algorithms for deep neural network training. We leverage efficient sea…
MixupE: Understanding and Improving Mixup from Directional Derivative Perspective
Yingtian Zou, Vikas Verma, Sarthak Mittal +6
Mixup is a popular data augmentation technique for training deep neural networks where additional samples are generated by linearly interpolating pairs of inputs and their labels.…
Combined Scaling for Zero-shot Transfer Learning
Hieu Pham, Zihang Dai, Golnaz Ghiasi +9
We present a combined scaling method - named BASIC - that achieves 85.7% top-1 accuracy on the ImageNet ILSVRC-2012 validation set without learning from any labeled ImageNet exampl…
AutoDropout: Learning Dropout Patterns to Regularize Deep Networks
Hieu Pham, Quoc V. Le
Neural networks are often over-parameterized and hence benefit from aggressive regularization. Conventional regularization methods, such as Dropout or weight decay, do not leverage…
Training EfficientNets at Supercomputer Scale: 83% ImageNet Top-1 Accuracy in One Hour
Arissa Wongpanich, Hieu Pham, James Demmel +4
EfficientNets are a family of state-of-the-art image classification models based on efficiently scaled convolutional neural networks. Currently, EfficientNets can take on the order…
Towards Domain-Agnostic Contrastive Learning
Vikas Verma, Minh-Thang Luong, Kenji Kawaguchi +2
Despite recent success, most contrastive self-supervised learning methods are domain-specific, relying heavily on data augmentation techniques that require knowledge about a partic…
Efficient Neural Architecture Search via Parameter Sharing
Hieu Pham, Melody Y. Guan, Barret Zoph +2
We propose Efficient Neural Architecture Search (ENAS), a fast and inexpensive approach for automatic model design. In ENAS, a controller learns to discover neural network architec…
Device Placement Optimization with Reinforcement Learning
Azalia Mirhoseini, Hieu Pham, Quoc V. Le +7
The past few years have witnessed a growth in size and computational requirements for training and inference with neural networks. Currently, a common approach to address these req…