4 papers
Where You Place the Norm Matters: From Prejudiced to Neutral Initializations
Emanuele Francazi, Francesco Pinto, Aurelien Lucchi +1
Normalization layers were introduced to stabilize and accelerate training, yet their influence is critical already at initialization, where they shape signal propagation and output…
When Bias Meets Trainability: Connecting Theories of Initialization
Alberto Bassi, Marco Baity-Jesi, Aurelien Lucchi +2
The statistical properties of deep neural networks (DNNs) at initialization play an important role to comprehend their trainability and the intrinsic architectural biases they poss…
Class Imbalance in Anomaly Detection: Learning from an Exactly Solvable Model
F. S. Pezzicoli, V. Ros, F. P. Landes +1
Class imbalance (CI) is a longstanding problem in machine learning, slowing down training and reducing performances. Although empirical remedies exist, it is often unclear which on…
Initial Guessing Bias: How Untrained Networks Favor Some Classes
Emanuele Francazi, Aurelien Lucchi, Marco Baity-Jesi
Understanding and controlling biasing effects in neural networks is crucial for ensuring accurate and fair model performance. In the context of classification problems, we provide…