9 papers
On the Implicit Bias of Gradient Descent for Temporal Extrapolation
Edo Cohen-Karlik, Avichai Ben David, Nadav Cohen +1
When using recurrent neural networks (RNNs) it is common practice to apply trained models to sequences longer than those seen in training. This "extrapolating" usage deviates from…
Semantic Segmentation in Art Paintings
Nadav Cohen, Yael Newman, Ariel Shamir
Semantic segmentation is a difficult task even when trained in a supervised manner on photographs. In this paper, we tackle the problem of semantic segmentation of artistic paintin…
Implicit Regularization in Tensor Factorization
Noam Razin, Asaf Maman, Nadav Cohen
Recent efforts to unravel the mystery of implicit regularization in deep learning have led to a theoretical focus on matrix factorization -- matrix completion via linear neural net…
Implicit Regularization in Deep Learning May Not Be Explainable by Norms
Noam Razin, Nadav Cohen
Mathematically characterizing the implicit regularization induced by gradient-based optimization is a longstanding pursuit in the theory of deep learning. A widespread hope is that…
Implicit Regularization in Deep Matrix Factorization
Sanjeev Arora, Nadav Cohen, Wei Hu +1
Efforts to understand the generalization mystery in deep learning have led to the belief that gradient-based optimization induces a form of implicit regularization, a bias towards…
A Convergence Analysis of Gradient Descent for Deep Linear Neural Networks
Sanjeev Arora, Nadav Cohen, Noah Golowich +1
We analyze speed of convergence to global optimum for gradient descent training a deep linear neural network (parameterized as ) by minimizing t…