Implicit regularization in AI meets generalized hardness of approximation in optimization -- Sharp results for diagonal linear networks

Johan S. Wind, Vegard Antun, Anders C. Hansen

Understanding the implicit regularization imposed by neural network architectures and gradient based optimization methods is a key challenge in deep learning and AI. In this work w…