5 citations · 6 across the 3 of their papers we have counts for
3 papers · 1 filter
Optimizers Qualitatively Alter Solutions And We Should Leverage This
Razvan Pascanu, Clare Lyle, Ionut-Vlad Modoranu +6
Due to the nonlinear nature of Deep Neural Networks (DNNs), one can not guarantee convergence to a unique global minimum of the loss when using optimizers relying only on local inf…
Normalization and effective learning rates in reinforcement learning
Clare Lyle, Zeyu Zheng, Khimya Khetarpal +4
Normalization layers have recently experienced a renaissance in the deep reinforcement learning and continual learning literature, with several works highlighting diverse benefits…
Disentangling the Causes of Plasticity Loss in Neural Networks
Clare Lyle, Zeyu Zheng, Khimya Khetarpal +4
Underpinning the past decades of work on the design, initialization, and optimization of neural networks is a seemingly innocuous assumption: that the network is trained on a \text…