281 citations · 451 across the 25 of their papers we have counts for
3 papers · 1 filter
Solving Quantitative Reasoning Problems with Language Models
Aitor Lewkowycz, Anders Andreassen, David Dohan +11
Language models have achieved remarkable performance on a wide range of tasks that require natural language understanding. Nevertheless, state-of-the-art models have generally stru…
Block-Recurrent Transformers
DeLesley Hutchins, Imanol Schlag, Yuhuai Wu +2
We introduce the Block-Recurrent Transformer, which applies a transformer layer in a recurrent fashion along a sequence, and has linear complexity with respect to sequence length.…
A Modern Self-Referential Weight Matrix That Learns to Modify Itself
Kazuki Irie, Imanol Schlag, Róbert Csordás +1
The weight matrix (WM) of a neural network (NN) is its program. The programs of many traditional NNs are learned through gradient descent in some error function, then remain fixed.…