1 citations · 4 across the 13 of their papers we have counts for
14 papers
Beyond Structural Symmetries: Linear Mode Connectivity via Neuron Identifiability
Vincent Bürgin, Daniel Herbst, Ya-Wei Eileen Lin +1
Many striking phenomena in deep learning, such as linear mode connectivity and the structured behavior of training dynamics, are closely tied to parameter symmetries: transformatio…
Any-Subgroup Equivariant Networks via Symmetry Breaking
Abhinav Goel, Derek Lim, Hannah Lawrence +2
The inclusion of symmetries as an inductive bias, known as equivariance, often improves generalization on geometric data (e.g. grids, sets, and graphs). However, equivariant archit…
Learning to Approximate Uniform Facility Location via Graph Neural Networks
Chendi Qian, Christopher Morris, Stefanie Jegelka +1
Neural networks, particularly message-passing neural networks (MPNNs), are increasingly used as heuristics for hard combinatorial optimization problems. Yet many learning-based met…
Neural Networks With Dense Weights Are Not Universal Approximators
Levi Rauchwerger, Stefanie Jegelka, Ron Levie
We investigate the approximation capabilities of dense neural networks. While universal approximation theorems establish that sufficiently large architectures can approximate arbit…
Survey on Generalization Theory for Graph Neural Networks
Antonis Vasileiou, Stefanie Jegelka, Ron Levie +1
Message-passing graph neural networks (MPNNs) have emerged as the leading approach for machine learning on graphs, attracting significant attention in recent years. While a large s…
Generalization, Expressivity, and Universality of Graph Neural Networks on Attributed Graphs
Levi Rauchwerger, Stefanie Jegelka, Ron Levie
We analyze the universality and generalization of graph neural networks (GNNs) on attributed graphs, i.e., with node attributes. To this end, we propose pseudometrics over the spac…
Learning on LoRAs: GL-Equivariant Processing of Low-Rank Weight Spaces for Large Finetuned Models
Theo Putterman, Derek Lim, Yoav Gelberg +2
Low-rank adaptations (LoRAs) have revolutionized the finetuning of large foundation models, enabling efficient adaptation even with limited computational resources. The resulting p…
Learning Linear Attention in Polynomial Time
Morris Yau, Ekin Akyürek, Jiayuan Mao +3
Previous research has explored the computational expressivity of Transformer models in simulating Boolean circuits or Turing machines. However, the learnability of these simulators…
The Empirical Impact of Neural Parameter Symmetries, or Lack Thereof
Derek Lim, Theo Moe Putterman, Robin Walters +2
Many algorithms and observed phenomena in deep learning appear to be affected by parameter symmetries -- transformations of neural network parameters that do not change the underly…
Future Directions in the Theory of Graph Machine Learning
Christopher Morris, Fabrizio Frasca, Nadav Dym +7
Machine learning on graphs, especially using graph neural networks (GNNs), has seen a surge in interest due to the wide availability of graph data across a broad spectrum of discip…
On the hardness of learning under symmetries
Bobak T. Kiani, Thien Le, Hannah Lawrence +2
We study the problem of learning equivariant neural networks via gradient descent. The incorporation of known symmetries ("equivariance") into neural nets has empirically improved…
Expressive Sign Equivariant Networks for Spectral Geometric Learning
Derek Lim, Joshua Robinson, Stefanie Jegelka +1
Recent work has shown the utility of developing machine learning models that respect the structure and symmetries of eigenvectors. These works promote sign invariance, since for an…
On the Stability of Expressive Positional Encodings for Graphs
Yinan Huang, William Lu, Joshua Robinson +4
Designing effective positional encodings for graphs is key to building powerful graph transformers and enhancing message-passing graph neural networks. Although widespread, using L…
Are Graph Neural Networks Optimal Approximation Algorithms?
Morris Yau, Nikolaos Karalias, Eric Lu +2
In this work we design graph neural network architectures that capture optimal approximation algorithms for a large class of combinatorial optimization problems, using powerful alg…