activity
20192026
most citedExploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer

3.7k citations · 3.7k across the 8 of their papers we have counts for

collaborators
Showing 2021Show all

2 papers · 1 filter

cs.LG2021

Merging Models with Fisher-Weighted Averaging

Michael Matena, Colin Raffel

Averaging the parameters of models that have the same architecture and initialization can provide a means of combining their respective capabilities. In this paper, we take the per…

cs.LG2021

Do Transformer Modifications Transfer Across Implementations and Applications?

Sharan Narang, Hyung Won Chung, Yi Tay +13

The research community has proposed copious modifications to the Transformer architecture since it was introduced over three years ago, relatively few of which have seen widespread…