125 citations · 237 across the 22 of their papers we have counts for
13 papers · 1 filter
Eigenvalues of the Hessian in Deep Learning: The Origin of Symmetry and Its Breaking
Yossi Arjevani
Hessian spectra at trained models in deep learning exhibit a persistent pattern: eigenvalues organize into distinct clusters, including a large bulk near zero and a few isolated ou…
Geometry and Optimization of Shallow Polynomial Networks
Yossi Arjevani, Joan Bruna, Joe Kileel +2
We study shallow neural networks with monomial activations and output dimension one. The function space for these models can be identified with a set of symmetric tensors with boun…
Symmetry & Critical Points
Yossi Arjevani
Critical points of an invariant function may or may not be symmetric. We prove, however, that if a symmetric critical point exists, those adjacent to it are generically symmetry br…
Hidden Minima in Two-Layer ReLU Networks
Yossi Arjevani
We consider the optimization problem associated with training two-layer ReLU networks with \(d\) inputs under the squared loss, where the labels are generated by a target network.…
Annihilation of Spurious Minima in Two-Layer ReLU Networks
Yossi Arjevani, Michael Field
We study the optimization problem associated with fitting two-layer ReLU neural networks with respect to the squared loss, where labels are generated by a target network. Use is ma…
Equivariant bifurcation, quadratic equivariants, and symmetry breaking for the standard representation of
Yossi Arjevani, Michael Field
Motivated by questions originating from the study of a class of shallow student-teacher neural networks, methods are developed for the analysis of spurious minima in classes of gra…
Analytic Study of Families of Spurious Minima in Two-Layer ReLU Neural Networks: A Tale of Symmetry II
Yossi Arjevani, Michael Field
We study the optimization problem associated with fitting two-layer ReLU neural networks with respect to the squared loss, where labels are generated by a target network. We make u…
Analytic Characterization of the Hessian in Shallow ReLU Models: A Tale of Symmetry
Yossi Arjevani, Michael Field
We consider the optimization problem associated with fitting two-layers ReLU networks with respect to the squared loss, where labels are generated by a target network. We leverage…
Second-Order Information in Non-Convex Stochastic Optimization: Power and Limitations
Yossi Arjevani, Yair Carmon, John C. Duchi +3
We design an algorithm which finds an -approximate stationary point (with ) using stochastic gradient and Hessian-vector products, matching gua…
Symmetry & critical points for a model shallow neural network
Yossi Arjevani, Michael Field
We consider the optimization problem associated with fitting two-layer ReLU networks with hidden neurons, where labels are assumed to be generated by a (teacher) neural network…
On the Complexity of Minimizing Convex Finite Sums Without Using the Indices of the Individual Functions
Yossi Arjevani, Amit Daniely, Stefanie Jegelka +1
Recent advances in randomized incremental methods for minimizing -smooth -strongly convex finite sums have culminated in tight complexity of $\tilde{O}((n+\sqrt{n L/μ})\log(1…
On the Principle of Least Symmetry Breaking in Shallow ReLU Models
Yossi Arjevani, Michael Field
We consider the optimization problem associated with fitting two-layer ReLU networks with respect to the squared loss, where labels are assumed to be generated by a target network.…
Communication Complexity of Distributed Convex Learning and Optimization
Yossi Arjevani, Ohad Shamir
We study the fundamental limits to communication-efficient distributed methods for convex learning and optimization, under different assumptions on the information available to ind…