8 citations · 17 across the 8 of their papers we have counts for
9 papers
NoProp: Training Neural Networks without Full Back-propagation or Full Forward-propagation
Qinyu Li, Yee Whye Teh, Razvan Pascanu
The canonical deep learning approach for learning requires computing a gradient term at each block by back-propagating the error signal from the output towards each learnable param…
Prompting Strategies for Enabling Large Language Models to Infer Causation from Correlation
Eleni Sgouritsa, Virginia Aglietti, Yee Whye Teh +3
The reasoning abilities of Large Language Models (LLMs) are attracting increasing attention. In this work, we focus on causal reasoning and address the task of establishing causal…
Non-Stationary Learning of Neural Networks with Automatic Soft Parameter Reset
Alexandre Galashov, Michalis K. Titsias, András György +4
Neural networks are traditionally trained under the assumption that data come from a stationary distribution. However, settings which violate this assumption are becoming more popu…
L3Ms -- Lagrange Large Language Models
Guneet S. Dhillon, Xingjian Shi, Yee Whye Teh +1
Supervised fine-tuning (SFT) and alignment of large language models (LLMs) are key steps in providing a good user experience. However, the concept of an appropriate alignment is in…
RecurrentGemma: Moving Past Transformers for Efficient Open Language Models
Aleksandar Botev, Soham De, Samuel L Smith +59
We introduce RecurrentGemma, a family of open language models which uses Google's novel Griffin architecture. Griffin combines linear recurrences with local attention to achieve ex…
Revisiting Dynamic Evaluation: Online Adaptation for Large Language Models
Amal Rannen-Triki, Jorg Bornschein, Razvan Pascanu +5
We consider the problem of online fine tuning the parameters of a language model at test time, also known as dynamic evaluation. While it is generally known that this approach impr…
Griffin: Mixing Gated Linear Recurrences with Local Attention for Efficient Language Models
Soham De, Samuel L. Smith, Anushan Fernando +14
Recurrent neural networks (RNNs) have fast inference and scale efficiently on long sequences, but they are difficult to train and hard to scale. We propose Hawk, an RNN with gated…
A characterization of product-form exchangeable feature probability functions
Marco Battiston, Stefano Favaro, Daniel M. Roy +1
We characterize the class of exchangeable feature allocations assigning probability to a feature allocation of individuals, display…
Modeling population structure under hierarchical Dirichlet processes
M. De Iorio, L. T. Elliott, S. Favaro +2
We propose a Bayesian nonparametric model to infer population admixture, extending the Hierarchical Dirichlet Process to allow for correlation between loci due to Linkage Disequili…