149 citations · 545 across the 20 of their papers we have counts for
22 papers
A Dynamical Theory of LoRA in Continual Learning
Théo Marchetta, Filippo Alessandroni, Alessandro Breccia +2
Despite the widespread use of Low-Rank Adaptation (LoRA), little is known about its dynamics in continual learning and the mechanisms by which low-rank updates affect catastrophic…
An Analytical Theory of Auxiliary Learning
Federico Milanesio, Alessandro Ingrosso, Matteo Osella
Auxiliary learning is an optimization paradigm in which a neural network's performance on a target task is improved by jointly training it on additional tasks. However, the mechani…
Coarse-Graining Hidden Representations: Unsupervised Neuron Selection via Mapping Entropy
Margherita Mele, Andrea Castagna, Roberto Menichetti +2
Overparameterized neural networks carry far more hidden units than a task nominally requires, raising the question of which neurons are essential and whether that distinction is le…
The dimensionality of the Hopfield model
Cristopher Erazo, Santiago Acevedo, Alessandro Ingrosso
We use the Binary Intrinsic Dimension (BID), a geometrical measure designed for binary data, to analyze the Hopfield model, a paradigmatic spin system from statistical mechanics, m…
Generalization performance of narrow one-hidden layer networks in the teacher-student setting
Rodrigo Pérez Ortiz, Gibbs Nwemadji, Jean Barbier +4
Understanding the generalization properties of neural networks on simple input-output distributions is key to explaining their performance on real datasets. The classical teacher-s…
Density of states in neural networks: an in-depth exploration of learning in parameter space
Margherita Mele, Roberto Menichetti, Alessandro Ingrosso +1
Learning in neural networks critically hinges on the intricate geometry of the loss landscape associated with a given task. Traditionally, most research has focused on finding spec…
Statistical mechanics of transfer learning in fully-connected networks in the proportional limit
Alessandro Ingrosso, Rosalba Pacelli, Pietro Rotondo +1
Transfer learning (TL) is a well-established machine learning technique to boost the generalization performance on a specific (target) task using information gained from a related…
Feature learning in finite-width Bayesian deep linear networks with multiple outputs and convolutional layers
Federico Bassetti, Marco Gherardi, Alessandro Ingrosso +2
Deep linear networks have been extensively studied, as they provide simplified models of deep learning. However, little is known in the case of finite-width architectures with mult…
Machine learning at the mesoscale: a computation-dissipation bottleneck
Alessandro Ingrosso, Emanuele Panizon
The cost of information processing in physical systems calls for a trade-off between performance and energetic expenditure. Here we formulate and study a computation-dissipation bo…
Neural networks trained with SGD learn distributions of increasing complexity
Maria Refinetti, Alessandro Ingrosso, Sebastian Goldt
The ability of deep neural networks to generalise well even when they interpolate their training data has been explained using various "simplicity biases". These theories postulate…
Data-driven emergence of convolutional structure in neural networks
Alessandro Ingrosso, Sebastian Goldt
Exploiting data invariances is crucial for efficient learning in both artificial and biological neural circuits. Understanding how neural networks can discover appropriate represen…
Input correlations impede suppression of chaos and learning in balanced rate networks
Rainer Engelken, Alessandro Ingrosso, Ramin Khajeh +2
Neural circuits exhibit complex activity patterns, both spontaneously and evoked by external stimuli. Information encoding and learning in neural circuits depend on how well time-v…
Epidemic mitigation by statistical inference from contact tracing data
Antoine Baker, Indaco Biazzo, Alfredo Braunstein +10
Contact-tracing is an essential tool in order to mitigate the impact of pandemic such as the COVID-19. In order to achieve efficient and scalable contact-tracing in real time, digi…
Optimal Learning with Excitatory and Inhibitory synapses
Alessandro Ingrosso
Characterizing the relation between weight structure and input/output statistics is fundamental for understanding the computational capabilities of neural circuits. In this work, I…
Training dynamically balanced excitatory-inhibitory networks
Alessandro Ingrosso, L. F. Abbott
The construction of biologically plausible models of neural circuits is crucial for understanding the computational properties of the nervous system. Constructing functional networ…
From statistical inference to a differential learning rule for stochastic neural networks
Luca Saglietti, Federica Gerace, Alessandro Ingrosso +2
Stochastic neural networks are a prototypical computational device able to build a probabilistic representation of an ensemble of external stimuli. Building on the relationship bet…
Network reconstruction from infection cascades
Alfredo Braunstein, Alessandro Ingrosso, Anna Paola Muntoni
Accessing the network through which a propagation dynamics diffuse is essential for understanding and controlling it. In a few cases, such information is available through direct e…
Unreasonable Effectiveness of Learning Neural Networks: From Accessible States and Robust Ensembles to Basic Algorithmic Schemes
Carlo Baldassi, Christian Borgs, Jennifer Chayes +4
In artificial neural networks, learning from data is a computationally demanding task in which a large number of connection weights are iteratively tuned through stochastic-gradien…
Discovering Neuronal Cell Types and Their Gene Expression Profiles Using a Spatial Point Process Mixture Model
Furong Huang, Animashree Anandkumar, Christian Borgs +6
Cataloging the neuronal cell types that comprise circuitry of individual brain regions is a major goal of modern neuroscience and the BRAIN initiative. Single-cell RNA sequencing c…
Local entropy as a measure for sampling solutions in Constraint Satisfaction Problems
Carlo Baldassi, Alessandro Ingrosso, Carlo Lucibello +2
We introduce a novel Entropy-driven Monte Carlo (EdMC) strategy to efficiently sample solutions of random Constraint Satisfaction Problems (CSPs). First, we extend a recent result…
Subdominant Dense Clusters Allow for Simple Learning and High Computational Performance in Neural Networks with Discrete Synapses
Carlo Baldassi, Alessandro Ingrosso, Carlo Lucibello +2
We show that discrete synaptic weights can be efficiently used for learning in large scale neural systems, and lead to unanticipated computational performance. We focus on the repr…
The zero-patient problem with noisy observations
Fabrizio Altarelli, Alfredo Braunstein, Luca Dall'Asta +2
A Belief Propagation approach has been recently proposed for the zero-patient problem in a SIR epidemics. The zero-patient problem consists in finding the initial source of an epid…