143 citations · 204 across the 5 of their papers we have counts for
3 papers · 1 filter
Neurons in Large Language Models: Dead, N-gram, Positional
Elena Voita, Javier Ferrando, Christoforos Nalmpantis
We analyze a family of large language models in such a lightweight manner that can be done on a single GPU. Specifically, we focus on the OPT family of models ranging from 125m to…
Augmented Language Models: a Survey
Grégoire Mialon, Roberto Dessì, Maria Lomeli +10
This survey reviews works in which language models (LMs) are augmented with reasoning skills and the ability to use tools. The former is defined as decomposing a potentially comple…
PEER: A Collaborative Language Model
Timo Schick, Jane Dwivedi-Yu, Zhengbao Jiang +7
Textual content is often the output of a collaborative writing process: We start with an initial draft, ask for suggestions, and repeatedly make changes. Agnostic of this process,…