5 citations · 5 across the 2 of their papers we have counts for
Showing cs.LGShow all
2 papers · 1 filter
cs.LG2022
Orthogonalising gradients to speed up neural network optimisation
Mark Tuddenham, Adam Prügel-Bennett, Jonathan Hare
The optimisation of neural networks can be sped up by orthogonalising the gradients before the optimisation step, ensuring the diversification of the learned representations. We or…
cs.LG2020★ 5 cited
Quasi-Newton's method in the class gradient defined high-curvature subspace
Mark Tuddenham, Adam Prügel-Bennett, Jonathan Hare
Classification problems using deep learning have been shown to have a high-curvature subspace in the loss landscape equal in dimension to the number of classes. Moreover, this subs…