4 papers
Training Instabilities Induce Flatness Bias in Gradient Descent
Lawrence Wang, Stephen J. Roberts
Classical analyses of gradient descent (GD) define a stability threshold based on the largest eigenvalue of the loss Hessian, often termed sharpness. When the learning rate lies be…
Pruning Cannot Hurt Robustness: Certified Trade-offs in Reinforcement Learning
James Pedley, Benjamin Etheridge, Stephen J. Roberts +1
Reinforcement learning (RL) policies deployed in real-world environments must remain reliable under adversarial perturbations. At the same time, modern deep RL agents are heavily o…
Beyond Accuracy: EcoL2 Metric for Sustainable Neural PDE Solvers
Taniya Kapoor, Abhishek Chandra, Anastasios Stamou +1
Real-world systems, from aerospace to railway engineering, are modeled with partial differential equations (PDEs) describing the physics of the system. Estimating robust solutions…
Can Stability be Detrimental? Better Generalization through Gradient Descent Instabilities
Lawrence Wang, Stephen J. Roberts
Traditional analyses of gradient descent optimization show that, when the largest eigenvalue of the loss Hessian - often referred to as the sharpness - is below a critical learning…