Ridge Rider: Finding Diverse Solutions by Following Eigenvectors of the Hessian

Abstract

Over the last decade, a single algorithm has changed many facets of our lives - Stochastic Gradient Descent (SGD). In the era of ever decreasing loss functions, SGD and its various offspring have become the go-to optimization tool in machine learning and are a key component of the success of deep neural networks (DNNs). While SGD is guaranteed to converge to a local optimum (under loose assumptions), in some cases it may matter which local optimum is found, and this is often context-dependent. Examples frequently arise in machine learning, from shape-versus-texture-features to ensemble methods and zero-shot coordination. In these settings, there are desired solutions which SGD on 'standard' loss functions will not find, since it instead converges to the 'easy' solutions. In this paper, we present a different approach. Rather than following the gradient, which corresponds to a locally greedy direction, we instead follow the eigenvectors of the Hessian, which we call "ridges". By iteratively following and branching amongst the ridges, we effectively span the loss surface to find qualitatively different solutions. We show both theoretically and experimentally that our method, called Ridge Rider (RR), offers a promising direction for a variety of challenging problems.

arXiv commentCamera-ready version, NeurIPS 2020

References in corpus (10)

• TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems
• Shortcut Learning in Deep Neural Networks
• The Loss Surfaces of Multilayer Networks
• Generalisation in humans and deep neural networks
• Illuminating search spaces by mapping elites
• Unsupervised Visual Representation Learning by Context Prediction
• The Hanabi Challenge: A New Frontier for AI Research
• An Investigation into Neural Net Optimization via Hessian Eigenvalue Density
• Video Representation Learning by Dense Predictive Coding
• Effective Diversity in Population Based Reinforcement Learning