45 citations · 157 across the 16 of their papers we have counts for
Showing cs.LGShow all
2 papers · 1 filter
cs.LG2023★ 2 cited
Functional Interpolation for Relative Positions Improves Long Context Transformers
Shanda Li, Chong You, Guru Guruganesh +7
Preventing the performance decay of Transformers on inputs longer than those used for training has been an important challenge in extending the context length of these models. Thou…
cs.LG2020★ 28 cited
ETC: Encoding Long and Structured Inputs in Transformers
Joshua Ainslie, Santiago Ontanon, Chris Alberti +7
Transformer models have advanced the state of the art in many Natural Language Processing (NLP) tasks. In this paper, we present a new Transformer architecture, Extended Transforme…