61 citations · 358 across the 40 of their papers we have counts for
45 papers
Transferable Graph Metanetworks
Yuxin Ma, Adir Dayan, Yam Eitan +2
A weight space network (or metanetwork) takes the weights of another neural network as input and predicts properties of it. Most prior work trains such models on input networks of…
Warm-starting PDE solvers with any-dimensional machine learning
Wilson G. Gregory, George A. Kevrekidis, Ben Blum-Smith +1
Any-dimensional machine learning models, such as graph neural networks (GNNs), can be naturally trained and evaluated on inputs of different sizes and dimensions. Inspired by the G…
pscaling small models: Principled warm starts and hyperparameter transfer
Yuxin Ma, Nan Chen, Mateo Díaz +3
Modern large-scale neural networks are often trained and released in multiple sizes to accommodate diverse inference budgets. To improve efficiency, recent work has explored model…
If you can distinguish, you can express: Galois theory, Stone--Weierstrass, machine learning, and linguistics
Ben Blum-Smith, Claudia Brugman, Thomas Conners +1
This essay develops a parallel between the Fundamental Theorem of Galois Theory and the Stone--Weierstrass theorem: both can be viewed as assertions that tie the distinguishing pow…
Thinner Latent Spaces: Detecting Dimension and Imposing Invariance with Conformal Autoencoders
George A. Kevrekidis, Zan Ahmad, Mauro Maggioni +2
Conformal Autoencoders are a neural network architecture that imposes orthogonality conditions between the gradients of latent variables to obtain disentangled representations of d…
Tensor learning with orthogonal, Lorentz, and symplectic symmetries
Wilson G. Gregory, Josué Tonelli-Cueto, Nicholas F. Marshall +2
Tensors are a fundamental data structure for many scientific contexts, such as time series analysis, materials science, and physics, among many others. Improving our ability to pro…
Is machine learning good or bad for the natural sciences?
David W. Hogg, Soledad Villar
Machine learning (ML) methods are having a huge impact across all of the sciences. However, ML has a strong ontology - in which only the data exist - and a strong epistemology - in…
A Galois theorem for machine learning: Functions on symmetric matrices and point clouds via lightweight invariant features
Ben Blum-Smith, Ningyuan Huang, Marco Cuturi +1
In this work, we present a mathematical formulation for machine learning of (1) functions on symmetric matrices that are invariant with respect to the action of permutations by con…
Approximately Equivariant Graph Networks
Ningyuan Huang, Ron Levie, Soledad Villar
Graph neural networks (GNNs) are commonly described as being permutation equivariant with respect to node relabeling in the graph. This symmetry of GNNs is often compared to the tr…
Structuring Representation Geometry with Rotationally Equivariant Contrastive Learning
Sharut Gupta, Joshua Robinson, Derek Lim +2
Self-supervised learning converts raw perceptual data such as images to a compact space where simple Euclidean distances measure meaningful variations in data. In this paper, we ex…
Fine-grained Expressivity of Graph Neural Networks
Jan Böker, Ron Levie, Ningyuan Huang +2
Numerous recent works have analyzed the expressive power of message-passing graph neural networks (MPNNs), primarily utilizing combinatorial techniques such as the -dimensional…
Equivariant geometric convolutions for emulation of dynamical systems
Wilson G. Gregory, David W. Hogg, Ben Blum-Smith +3
Machine learning methods are increasingly being employed as surrogate models in place of computationally expensive and slow numerical integrators for a bevy of applications in the…
Towards fully covariant machine learning
Soledad Villar, David W. Hogg, Weichi Yao +2
Any representation of data involves arbitrary investigator choices. Because those choices are external to the data-generating process, each choice leads to an exact symmetry, corre…
Sketch-and-solve approaches to k-means clustering by semidefinite programming
Charles Clum, Dustin G. Mixon, Soledad Villar +1
We introduce a sketch-and-solve approach to speed up the Peng-Wei semidefinite relaxation of k-means clustering. When the data is appropriately separated we identify the k-means op…
A Spectral Analysis of Graph Neural Networks on Dense and Sparse Graphs
Luana Ruiz, Ningyuan Huang, Soledad Villar
In this work we propose a random graph model that can produce graphs at different levels of sparsity. We analyze how sparsity affects the graph spectra, and thus the performance of…
Deep Learning is Provably Robust to Symmetric Label Noise
Carey E. Priebe, Ningyuan Huang, Soledad Villar +2
Deep neural networks (DNNs) are capable of perfectly fitting the training data, including memorizing noisy data. It is commonly believed that memorization hurts generalization. The…
Shuffled linear regression through graduated convex relaxation
Efe Onaran, Soledad Villar
The shuffled linear regression problem aims to recover linear relationships in datasets where the correspondence between input and output is unknown. This problem arises in a wide…
Machine learning and invariant theory
Ben Blum-Smith, Soledad Villar
Inspired by constraints from physical law, equivariant machine learning restricts the learning to a hypothesis class where all the functions are equivariant with respect to some gr…
From Local to Global: Spectral-Inspired Graph Neural Networks
Ningyuan Huang, Soledad Villar, Carey E. Priebe +4
Graph Neural Networks (GNNs) are powerful deep learning methods for Non-Euclidean data. Popular GNNs are message-passing algorithms (MPNNs) that aggregate and combine signals in a…
MarkerMap: nonlinear marker selection for single-cell studies
Nabeel Sarwar, Wilson Gregory, George A Kevrekidis +2
Single-cell RNA-seq data allow the quantification of cell type differences across a growing set of biological contexts. However, pinpointing a small subset of genomic features expl…
Dimensionless machine learning: Imposing exact units equivariance
Soledad Villar, Weichi Yao, David W. Hogg +2
Units equivariance (or units covariance) is the exact symmetry that follows from the requirement that relationships among measured quantities of physics relevance must obey self-co…
Prospective Learning: Principled Extrapolation to the Future
Ashwin De Silva, Rahul Ramesh, Lyle Ungar +40
Learning is a process which can update decision rules, based on past experience, such that future performance improves. Traditionally, machine learning is often evaluated under the…
A Short Tutorial on The Weisfeiler-Lehman Test And Its Variants
Ningyuan Huang, Soledad Villar
Graph neural networks are designed to learn functions on graphs. Typically, the relevant target functions are invariant with respect to actions by permutations. Therefore the desig…
Three proofs of the Benedetto-Fickus theorem
Dustin G. Mixon, Tom Needham, Clayton Shonkwiler +1
In 2003, Benedetto and Fickus introduced a vivid intuition for an objective function called the frame potential, whose global minimizers are fundamental objects known today as unit…