5 papers
NoProp: Training Neural Networks without Full Back-propagation or Full Forward-propagation
Qinyu Li, Yee Whye Teh, Razvan Pascanu
The canonical deep learning approach for learning requires computing a gradient term at each block by back-propagating the error signal from the output towards each learnable param…
L3Ms -- Lagrange Large Language Models
Guneet S. Dhillon, Xingjian Shi, Yee Whye Teh +1
Supervised fine-tuning (SFT) and alignment of large language models (LLMs) are key steps in providing a good user experience. However, the concept of an appropriate alignment is in…
Prompting Strategies for Enabling Large Language Models to Infer Causation from Correlation
Eleni Sgouritsa, Virginia Aglietti, Yee Whye Teh +3
The reasoning abilities of Large Language Models (LLMs) are attracting increasing attention. In this work, we focus on causal reasoning and address the task of establishing causal…
Non-Stationary Learning of Neural Networks with Automatic Soft Parameter Reset
Alexandre Galashov, Michalis K. Titsias, András György +4
Neural networks are traditionally trained under the assumption that data come from a stationary distribution. However, settings which violate this assumption are becoming more popu…
RecurrentGemma: Moving Past Transformers for Efficient Open Language Models
Aleksandar Botev, Soham De, Samuel L Smith +59
We introduce RecurrentGemma, a family of open language models which uses Google's novel Griffin architecture. Griffin combines linear recurrences with local attention to achieve ex…