42 citations · 123 across the 15 of their papers we have counts for
15 papers
Inferring effective interactions and task-related brain states from large-scale neural activity with Restricted Boltzmann Machines
Nicolas Béreux, Giovanni Catania, Aurélien Decelle +3
Large-scale electrophysiological recordings now enable the simultaneous monitoring of thousands of neurons across multiple brain regions, revealing structured variability in popula…
Neural subspaces, minimax entropy, and mean-field theory for networks of neurons
Luca Di Carlo, Francesca Mignacco, Christopher W. Lynn +1
Recent advances in experimental techniques enable the simultaneous recording of activity from thousands of neurons in the brain, presenting both an opportunity and a challenge: to…
A statistical physics framework for optimal learning
Francesca Mignacco, Francesco Mori
Learning is a complex dynamical process shaped by a range of interconnected decisions. Careful design of hyperparameter schedules for artificial neural networks or efficient alloca…
Analytic theory of dropout regularization
Francesco Mori, Francesca Mignacco
Dropout is a regularization technique widely used in training artificial neural networks to mitigate overfitting. It consists of dynamically deactivating subsets of the network dur…
Extended mean-field theories for networks of real neurons
Luca Di Carlo, Francesca Mignacco, Christopher W. Lynn +1
If the behavior of a system with many degrees of freedom can be captured by a small number of collective variables, then plausibly there is an underlying mean-field theory. We show…
Optimal Protocols for Continual Learning via Statistical Physics and Control Theory
Francesco Mori, Stefano Sarao Mannelli, Francesca Mignacco
Artificial neural networks often struggle with catastrophic forgetting when learning multiple tasks sequentially, as training on new tasks degrades the performance on previously le…
Dissecting the Interplay of Attention Paths in a Statistical Mechanics Theory of Transformers
Lorenzo Tiberi, Francesca Mignacco, Kazuki Irie +1
Despite the remarkable empirical performance of Transformers, their theoretical understanding remains elusive. Here, we consider a deep multi-head self-attention network, that is c…
Nonlinear classification of neural manifolds with contextual information
Francesca Mignacco, Chi-Ning Chou, SueYeon Chung
Understanding how neural systems efficiently process information through distributed representations is a fundamental challenge at the interface of neuroscience and machine learnin…
High-Dimensional Non-Convex Landscapes and Gradient Descent Dynamics
Tony Bonnaire, Davide Ghio, Kamesh Krishnamurthy +3
In these lecture notes we present different methods and concepts developed in statistical physics to analyze gradient descent dynamics in high-dimensional non-convex landscapes. Ou…
Forward Learning with Top-Down Feedback: Empirical and Analytical Characterization
Ravi Srinivasan, Francesca Mignacco, Martino Sorbaro +4
"Forward-only" algorithms, which train neural networks while avoiding a backward pass, have recently gained attention as a way of solving the biologically unrealistic aspects of ba…
Learning curves for the multi-class teacher-student perceptron
Elisabetta Cornacchia, Francesca Mignacco, Rodrigo Veiga +3
One of the most classical results in high-dimensional learning theory provides a closed-form expression for the generalisation error of binary classification with the single-layer…
The effective noise of Stochastic Gradient Descent
Francesca Mignacco, Pierfrancesco Urbani
Stochastic Gradient Descent (SGD) is the workhorse algorithm of deep learning technology. At each step of the training phase, a mini batch of samples is drawn from the training dat…
Stochasticity helps to navigate rough landscapes: comparing gradient-descent-based algorithms in the phase retrieval problem
Francesca Mignacco, Pierfrancesco Urbani, Lenka Zdeborová
In this paper we investigate how gradient-based algorithms such as gradient descent, (multi-pass) stochastic gradient descent, its persistent variant, and the Langevin algorithm na…
Dynamical mean-field theory for stochastic gradient descent in Gaussian mixture classification
Francesca Mignacco, Florent Krzakala, Pierfrancesco Urbani +1
We analyze in a closed form the learning dynamics of stochastic gradient descent (SGD) for a single-layer neural network classifying a high-dimensional Gaussian mixture where each…
The role of regularization in classification of high-dimensional noisy Gaussian mixture
Francesca Mignacco, Florent Krzakala, Yue M. Lu +1
We consider a high-dimensional mixture of two Gaussians in the noisy regime where even an oracle knowing the centers of the clusters misclassifies a small but finite fraction of th…