On Local Overfitting and Forgetting in Deep Neural Networks

Uri Stern, Tomer Yaacoby, Daphna Weinshall

The infrequent occurrence of overfitting in deep neural networks is perplexing: contrary to theoretical expectations, increasing model size often enhances performance in practice.…