263 citations · 1.1k across the 15 of their papers we have counts for
15 papers
Brainformers: Trading Simplicity for Efficiency
Yanqi Zhou, Nan Du, Yanping Huang +12
Transformers are central to recent successes in natural language processing and computer vision. Transformers have a mostly uniform backbone where layers alternate between feed-for…
PaLM 2 Technical Report
Rohan Anil, Andrew M. Dai, Orhan Firat +125
We introduce PaLM 2, a new state-of-the-art language model that has better multilingual and reasoning capabilities and is more compute-efficient than its predecessor PaLM. PaLM 2 i…
Unified Functional Hashing in Automatic Machine Learning
Ryan Gillard, Stephen Jonany, Yingjie Miao +7
The field of Automatic Machine Learning (AutoML) has recently attained impressive results, including the discovery of state-of-the-art machine learning solutions, such as neural im…
EvoPrompting: Language Models for Code-Level Neural Architecture Search
Angelica Chen, David M. Dohan, David R. So
Given the recent impressive accomplishments of language models (LMs) for code generation, we explore the use of LMs as adaptive mutation and crossover operators for an evolutionary…
Transcending Scaling Laws with 0.1% Extra Compute
Yi Tay, Jason Wei, Hyung Won Chung +13
Scaling language models improves performance but comes with significant computational costs. This paper proposes UL2R, a method that substantially improves existing language models…
The Carbon Footprint of Machine Learning Training Will Plateau, Then Shrink
David Patterson, Joseph Gonzalez, Urs Hölzle +7
Machine Learning (ML) workloads have rapidly grown in importance, but raised concerns about their carbon footprint. Four best practices can reduce ML training energy by up to 100x…