15 citations · 34 across the 5 of their papers we have counts for
5 papers
MaD Physics: Evaluating information seeking under constraints in physical environments
Moksh Jain, Mehdi Bennani, Johannes Bausch +4
Scientific discovery is fundamentally a resource-constrained process that requires navigating complex trade-offs between the quality and quantity of measurements due to physical an…
Randomized Positional Encodings Boost Length Generalization of Transformers
Anian Ruoss, Grégoire Delétang, Tim Genewein +5
Transformers have impressive generalization capabilities on tasks with a fixed context length. However, they fail to generalize to sequences of arbitrary length, even for seemingly…
A Generalist Neural Algorithmic Learner
Borja Ibarz, Vitaly Kurin, George Papamakarios +12
The cornerstone of neural algorithmic reasoning is the ability to solve algorithmic tasks, especially in a way that generalises out of distribution. While recent years have seen a…
A Theoretical Analysis of Catastrophic Forgetting through the NTK Overlap Matrix
Thang Doan, Mehdi Bennani, Bogdan Mazoure +2
Continual learning (CL) is a setting in which an agent has to learn from an incoming stream of data during its entire lifetime. Although major advances have been made in the field,…
Generalisation Guarantees for Continual Learning with Orthogonal Gradient Descent
Mehdi Abbana Bennani, Thang Doan, Masashi Sugiyama
In Continual Learning settings, deep neural networks are prone to Catastrophic Forgetting. Orthogonal Gradient Descent was proposed to tackle the challenge. However, no theoretical…