2 papers
cs.LG2026
Correlation flow governs learning at criticality
Andrea Combette, Nelly Pustelnik, Antoine Venaille
The initialisation of deep neural networks determines whether information and gradients can propagate across depth, yet a unified theory connecting these properties to learning dyn…
cs.LG2026
A new initialisation to Control Gradients in Sinusoidal Neural network
Andrea Combette, Antoine Venaille, Nelly Pustelnik
Proper initialisation strategy is of primary importance to mitigate gradient explosion or vanishing when training neural networks. Yet, the impact of initialisation parameters stil…