Corridor Geometry in Gradient-Based Optimization

Benoit Dherin, Mihaela Rosca

We characterize regions of a loss surface as corridors when the continuous curves of steepest descent -- the solutions of the gradient flow -- become straight lines. We show that c…