Optimization-Induced Dynamics of Lipschitz Continuity in Neural Networks

Róisín Luo, James McDermott, Christian Gagné +2

Lipschitz continuity characterizes the worst-case sensitivity of neural networks to small input perturbations; yet its dynamics (i.e. temporal evolution) during training remains un…