3.9k citations · 4k across the 3 of their papers we have counts for
3 papers
cs.AI2023★ 113 cited
The Flan Collection: Designing Data and Methods for Effective Instruction Tuning
Shayne Longpre, Le Hou, Tu Vu +8
We study the design decisions of publicly available instruction tuning methods, and break down the development of Flan 2022 (Chung et al., 2022). Through careful ablation studies o…
cs.LG2022★ 33 cited
A Review of Sparse Expert Models in Deep Learning
William Fedus, Jeff Dean, Barret Zoph
Sparse expert models are a thirty-year old concept re-emerging as a popular architecture in deep learning. This class of architecture encompasses Mixture-of-Experts, Switch Transfo…
cs.LG2016★ 3.9k cited
Neural Architecture Search with Reinforcement Learning
Barret Zoph, Quoc V. Le
Neural networks are powerful and flexible models that work well for many difficult learning tasks in image, speech and natural language understanding. Despite their success, neural…