2.3k citations · 4.2k across the 44 of their papers we have counts for
6 papers · 1 filter
Focused Transformer: Contrastive Training for Context Scaling
Szymon Tworkowski, Konrad Staniszewski, Mikołaj Pacek +3
Large language models have an exceptional capability to incorporate new information in a contextual manner. However, the full potential of such an approach is often restrained due…
Lexinvariant Language Models
Qian Huang, Eric Zelikman, Sarah Li Chen +3
Token embeddings, a mapping from discrete lexical symbols to continuous vectors, are at the heart of any language model (LM). However, lexical symbol meanings can also be determine…
PaLM 2 Technical Report
Rohan Anil, Andrew M. Dai, Orhan Firat +125
We introduce PaLM 2, a new state-of-the-art language model that has better multilingual and reasoning capabilities and is more compute-efficient than its predecessor PaLM. PaLM 2 i…
Language Model Cascades
David Dohan, Winnie Xu, Aitor Lewkowycz +9
Prompted models have demonstrated impressive few-shot learning abilities. Repeated interactions at test-time with a single model, or the composition of multiple models together, fu…
Exploring Length Generalization in Large Language Models
Cem Anil, Yuhuai Wu, Anders Andreassen +7
The ability to extrapolate from short problem instances to longer ones is an important form of out-of-distribution generalization in reasoning tasks, and is crucial when learning f…
Solving Quantitative Reasoning Problems with Language Models
Aitor Lewkowycz, Anders Andreassen, David Dohan +11
Language models have achieved remarkable performance on a wide range of tasks that require natural language understanding. Nevertheless, state-of-the-art models have generally stru…