210 citations · 984 across the 29 of their papers we have counts for
15 papers · 1 filter
What Makes Data Suitable for a Locally Connected Neural Network? A Necessary and Sufficient Condition Based on Quantum Entanglement
Yotam Alexander, Nimrod De La Vega, Noam Razin +1
The question of what makes a data distribution suitable for deep learning is a fundamental open problem. Focusing on locally connected neural networks (a prevalent family of archit…
On the Ability of Graph Neural Networks to Model Interactions Between Vertices
Noam Razin, Tom Verbin, Nadav Cohen
Graph neural networks (GNNs) are widely used for modeling complex interactions between entities represented as vertices of a graph. Despite recent efforts to theoretically analyze…
Learning Low Dimensional State Spaces with Overparameterized Recurrent Neural Nets
Edo Cohen-Karlik, Itamar Menuhin-Gruman, Raja Giryes +2
Overparameterization in deep learning typically refers to settings where a trained neural network (NN) has representational capacity to fit the training data in many ways, some of…
On the Implicit Bias of Gradient Descent for Temporal Extrapolation
Edo Cohen-Karlik, Avichai Ben David, Nadav Cohen +1
When using recurrent neural networks (RNNs) it is common practice to apply trained models to sequences longer than those seen in training. This "extrapolating" usage deviates from…
Implicit Regularization in Hierarchical Tensor Factorization and Deep Convolutional Neural Networks
Noam Razin, Asaf Maman, Nadav Cohen
In the pursuit of explaining implicit regularization in deep learning, prominent focus was given to matrix and tensor factorizations, which correspond to simplified neural networks…
Continuous vs. Discrete Optimization of Deep Neural Networks
Omer Elkabetz, Nadav Cohen
Existing analyses of optimization in deep learning are either continuous, focusing on (variants of) gradient flow, or discrete, directly treating (variants of) gradient descent. Gr…
Implicit Regularization in Tensor Factorization
Noam Razin, Asaf Maman, Nadav Cohen
Recent efforts to unravel the mystery of implicit regularization in deep learning have led to a theoretical focus on matrix factorization -- matrix completion via linear neural net…
Implicit Regularization in Deep Learning May Not Be Explainable by Norms
Noam Razin, Nadav Cohen
Mathematically characterizing the implicit regularization induced by gradient-based optimization is a longstanding pursuit in the theory of deep learning. A widespread hope is that…
Implicit Regularization in Deep Matrix Factorization
Sanjeev Arora, Nadav Cohen, Wei Hu +1
Efforts to understand the generalization mystery in deep learning have led to the belief that gradient-based optimization induces a form of implicit regularization, a bias towards…
A Convergence Analysis of Gradient Descent for Deep Linear Neural Networks
Sanjeev Arora, Nadav Cohen, Noah Golowich +1
We analyze speed of convergence to global optimum for gradient descent training a deep linear neural network (parameterized as ) by minimizing t…
On the Optimization of Deep Networks: Implicit Acceleration by Overparameterization
Sanjeev Arora, Nadav Cohen, Elad Hazan
Conventional wisdom in deep learning states that increasing depth improves expressiveness but complicates optimization. This paper suggests that, sometimes, increasing depth can sp…
Analysis and Design of Convolutional Networks via Hierarchical Tensor Decompositions
Nadav Cohen, Or Sharir, Yoav Levine +3
The driving force behind convolutional networks - the most successful deep learning architecture to date, is their expressive power. Despite its wide acceptance and vast empirical…
Deep Learning and Quantum Entanglement: Fundamental Connections with Implications to Network Design
Yoav Levine, David Yakira, Nadav Cohen +1
Deep convolutional networks have witnessed unprecedented success in various machine learning applications. Formal understanding on what makes these networks so successful is gradua…
Boosting Dilated Convolutional Networks with Mixed Tensor Decompositions
Nadav Cohen, Ronen Tamari, Amnon Shashua
The driving force behind deep networks is their ability to compactly represent rich classes of functions. The primary notion for formally reasoning about this phenomenon is express…
Tensorial Mixture Models
Or Sharir, Ronen Tamari, Nadav Cohen +1
Casting neural networks in generative frameworks is a highly sought-after endeavor these days. Contemporary methods, such as Generative Adversarial Networks, capture some of the ge…