87 citations · 197 across the 12 of their papers we have counts for
12 papers
Brainformers: Trading Simplicity for Efficiency
Yanqi Zhou, Nan Du, Yanping Huang +12
Transformers are central to recent successes in natural language processing and computer vision. Transformers have a mostly uniform backbone where layers alternate between feed-for…
LayerNAS: Neural Architecture Search in Polynomial Complexity
Yicheng Fan, Dana Alon, Jingyue Shen +7
Neural Architecture Search (NAS) has become a popular method for discovering effective model architectures, especially for target hardware. As such, NAS methods that find optimal a…
PyGlove: Efficiently Exchanging ML Ideas as Code
Daiyi Peng, Xuanyi Dong, Esteban Real +2
The increasing complexity and scale of machine learning (ML) has led to the need for more efficient collaboration among multiple teams. For example, when a research team invents a…
DeepFusion: Lidar-Camera Deep Fusion for Multi-Modal 3D Object Detection
Yingwei Li, Adams Wei Yu, Tianjian Meng +10
Lidars and cameras are critical sensors that provide complementary information for 3D detection in autonomous driving. While prevalent multi-modal methods simply decorate raw lidar…
Differentiable Architecture Search for Reinforcement Learning
Yingjie Miao, Xingyou Song, John D. Co-Reyes +4
In this paper, we investigate the fundamental question: To what extent are gradient-based neural architecture search (NAS) techniques applicable to RL? Using the original DARTS as…
Rethinking Co-design of Neural Architectures and Hardware Accelerators
Yanqi Zhou, Xuanyi Dong, Berkin Akin +7
Neural architectures and hardware accelerators have been two driving forces for the progress in deep learning. Previous works typically attempt to optimize hardware given a fixed m…
PyGlove: Symbolic Programming for Automated Machine Learning
Daiyi Peng, Xuanyi Dong, Esteban Real +7
Neural networks are sensitive to hyper-parameter and architecture choices. Automated Machine Learning (AutoML) is a promising paradigm for automating these choices. Current ML soft…
ES-ENAS: Efficient Evolutionary Optimization for Large Hybrid Search Spaces
Xingyou Song, Krzysztof Choromanski, Jack Parker-Holder +8
In this paper, we approach the problem of optimizing blackbox functions over large hybrid search spaces consisting of both combinatorial and continuous parameters. We demonstrate t…
Evolving Reinforcement Learning Algorithms
John D. Co-Reyes, Yingjie Miao, Daiyi Peng +5
We propose a method for meta-learning reinforcement learning algorithms by searching over the space of computational graphs which compute the loss function for a value-based model-…
Towards NNGP-guided Neural Architecture Search
Daniel S. Park, Jaehoon Lee, Daiyi Peng +2
The predictions of wide Bayesian neural networks are described by a Gaussian process, known as the Neural Network Gaussian Process (NNGP). Analytic forms for NNGP kernels are known…
AutoHAS: Efficient Hyperparameter and Architecture Search
Xuanyi Dong, Mingxing Tan, Adams Wei Yu +3
Efficient hyperparameter or architecture search methods have shown remarkable results, but each of them is only applicable to searching for either hyperparameters (HPs) or architec…
Domain Adaptive Transfer Learning with Specialist Models
Jiquan Ngiam, Daiyi Peng, Vijay Vasudevan +3
Transfer learning is a widely used method to build high performing computer vision models. In this paper, we study the efficacy of transfer learning by examining how the choice of…