4 papers
Escape dynamics and implicit bias of one-pass SGD in overparameterized quadratic networks
Dario Bocchi, Theotime Regimbeau, Carlo Lucibello +2
We analyze the one-pass stochastic gradient descent dynamics of a two-layer neural network with quadratic activations in a teacher--student framework. In the high-dimensional regim…
Bias-inducing geometries: an exactly solvable data model with fairness implications
Stefano Sarao Mannelli, Federica Gerace, Negar Rostamzadeh +1
Machine learning (ML) may be oblivious to human bias but it is not immune to its perpetuation. Marginalisation and iniquitous group representation are often traceable in the very d…
Tilting the Odds at the Lottery: the Interplay of Overparameterisation and Curricula in Neural Networks
Stefano Sarao Mannelli, Yaraslau Ivashynka, Andrew Saxe +1
A wide range of empirical and theoretical works have shown that overparameterisation can amplify the performance of neural networks. According to the lottery ticket hypothesis, ove…
The phase diagram of compressed sensing with -norm regularization
Damien Barbier, Carlo Lucibello, Luca Saglietti +2
Noiseless compressive sensing is a two-steps setting that allows for undersampling a sparse signal and then reconstructing it without loss of information. The LASSO algorithm, base…