On the Impossibility of Global Convergence in Multi-Loss Optimization

Alistair Letcher

Under mild regularity conditions, gradient-based methods converge globally to a critical point in the single-loss setting. This is known to break down for vanilla gradient descent…