263 citations · 1.1k across the 15 of their papers we have counts for
4 papers · 1 filter
Primer: Searching for Efficient Transformers for Language Modeling
David R. So, Wojciech Mańke, Hanxiao Liu +3
Large Transformer models have been central to recent advances in natural language processing. The training and inference costs of these models, however, have grown rapidly and beco…
Pay Attention to MLPs
Hanxiao Liu, Zihang Dai, David R. So +1
Transformers have become one of the most important architectural innovations in deep learning and have enabled many breakthroughs over the past few years. Here we propose a simple…
Carbon Emissions and Large Neural Network Training
David Patterson, Joseph Gonzalez, Quoc Le +6
The computation demand for machine learning (ML) has grown rapidly recently, which comes with a number of costs. Estimating the energy cost helps measure its environmental impact a…
MUFASA: Multimodal Fusion Architecture Search for Electronic Health Records
Zhen Xu, David R. So, Andrew M. Dai
One important challenge of applying deep learning to electronic health records (EHR) is the complexity of their multimodal structure. EHR usually contains a mixture of structured (…