3 citations · 5 across the 8 of their papers we have counts for
9 papers
On the Infinite Width and Depth Limits of Predictive Coding Networks
Francesco Innocenti, El Mehdi Achour, Rafal Bogacz
Predictive coding (PC) is a biologically plausible alternative to standard backpropagation (BP) that minimises an energy function with respect to network activities before updating…
Avoidance of non-strict saddle points by blow-up
El Mehdi Achour, Umberto L. Hryniewicz, Michael Westdickenberg
It is an old idea to use gradient flows or time-discretized variants thereof as methods for solving minimization problems. In some applications, for example in machine learning con…
A Simple Generalisation of the Implicit Dynamics of In-Context Learning
Francesco Innocenti, El Mehdi Achour
In-context learning (ICL) refers to the ability of a model to learn new tasks from examples in its input without any parameter updates. In contrast to previous theories of ICL rely…
The Riemannian Geometry Associated to Gradient Flows of Linear Convolutional Networks
El Mehdi Achour, Kathlén Kohn, Holger Rauhut
We study geometric properties of the gradient flow for learning deep linear convolutional networks. For linear fully connected networks, it has been shown recently that the corresp…
PC: Scaling Predictive Coding to 100+ Layer Networks
Francesco Innocenti, El Mehdi Achour, Christopher L. Buckley
The biological implausibility of backpropagation (BP) has motivated many alternative, brain-inspired algorithms that attempt to rely only on local information, such as predictive c…
Only Strict Saddles in the Energy Landscape of Predictive Coding Networks?
Francesco Innocenti, El Mehdi Achour, Ryan Singh +1
Predictive coding (PC) is an energy-based learning algorithm that performs iterative inference over network activities before updating weights. Recent work suggests that PC can con…
A general approximation lower bound in norm, with applications to feed-forward neural networks
El Mehdi Achour, Armand Foucault, Sébastien Gerchinovitz +1
We study the fundamental limits to the expressive power of neural networks. Given two sets , of real-valued functions, we first prove a general lower bound on how well funct…
Existence, Stability and Scalability of Orthogonal Convolutional Neural Networks
El Mehdi Achour, François Malgouyres, Franck Mamalet
Imposing orthogonality on the layers of neural networks is known to facilitate the learning by limiting the exploding/vanishing of the gradient; decorrelate the features; improve t…
The loss landscape of deep linear neural networks: a second-order analysis
El Mehdi Achour, François Malgouyres, Sébastien Gerchinovitz
We study the optimization landscape of deep linear neural networks with the square loss. It is known that, under weak assumptions, there are no spurious local minima and no local m…