77 citations · 204 across the 9 of their papers we have counts for
10 papers
How Uniform Random Weights Induce Non-uniform Bias: Typical Interpolating Neural Networks Generalize with Narrow Teachers
Gon Buzaglo, Itamar Harel, Mor Shpigel Nacson +3
Background. A main theoretical puzzle is why over-parameterized Neural Networks (NNs) generalize well when trained to zero loss (i.e., so they interpolate the data). Usually, the N…
A Theoretical Analysis of Fine-tuning with Linear Teachers
Gal Shachaf, Alon Brutzkus, Amir Globerson
Fine-tuning is a common practice in deep learning, achieving excellent generalization results on downstream tasks using relatively little training data. Although widely used in pra…
Towards Understanding Learning in Neural Networks with Linear Teachers
Roei Sarussi, Alon Brutzkus, Amir Globerson
Can a neural network minimizing cross-entropy learn linearly separable data? Despite progress in the theory of deep learning, this question remains unsolved. Here we prove that SGD…
An Optimization and Generalization Analysis for Max-Pooling Networks
Alon Brutzkus, Amir Globerson
Max-Pooling operations are a core component of deep learning architectures. In particular, they are part of most convolutional architectures used in machine vision, since pooling i…
On the Optimality of Trees Generated by ID3
Alon Brutzkus, Amit Daniely, Eran Malach
Since its inception in the 1980s, ID3 has become one of the most successful and widely used algorithms for learning decision trees. However, its theoretical properties remain poorl…
ID3 Learns Juntas for Smoothed Product Distributions
Alon Brutzkus, Amit Daniely, Eran Malach
In recent years, there are many attempts to understand popular heuristics. An example of such a heuristic algorithm is the ID3 algorithm for learning decision trees. This algorithm…
Low Latency Privacy Preserving Inference
Alon Brutzkus, Oren Elisha, Ran Gilad-Bachrach
When applying machine learning to sensitive data, one has to find a balance between accuracy, information security, and computational-complexity. Recent studies combined Homomorphi…
Why do Larger Models Generalize Better? A Theoretical Perspective via the XOR Problem
Alon Brutzkus, Amir Globerson
Empirical evidence suggests that neural networks with ReLU activations generalize better with over-parameterization. However, there is currently no theoretical analysis that explai…
SGD Learns Over-parameterized Networks that Provably Generalize on Linearly Separable Data
Alon Brutzkus, Amir Globerson, Eran Malach +1
Neural networks exhibit good generalization behavior in the over-parameterized regime, where the number of network parameters exceeds the number of observations. Nonetheless, curre…
Globally Optimal Gradient Descent for a ConvNet with Gaussian Inputs
Alon Brutzkus, Amir Globerson
Deep learning models are often successfully trained using gradient descent, despite the worst case hardness of the underlying non-convex optimization problem. The key question is t…