61 citations · 358 across the 40 of their papers we have counts for
13 papers · 1 filter
pscaling small models: Principled warm starts and hyperparameter transfer
Yuxin Ma, Nan Chen, Mateo Díaz +3
Modern large-scale neural networks are often trained and released in multiple sizes to accommodate diverse inference budgets. To improve efficiency, recent work has explored model…
Thinner Latent Spaces: Detecting Dimension and Imposing Invariance with Conformal Autoencoders
George A. Kevrekidis, Zan Ahmad, Mauro Maggioni +2
Conformal Autoencoders are a neural network architecture that imposes orthogonality conditions between the gradients of latent variables to obtain disentangled representations of d…
A Galois theorem for machine learning: Functions on symmetric matrices and point clouds via lightweight invariant features
Ben Blum-Smith, Ningyuan Huang, Marco Cuturi +1
In this work, we present a mathematical formulation for machine learning of (1) functions on symmetric matrices that are invariant with respect to the action of permutations by con…
Structuring Representation Geometry with Rotationally Equivariant Contrastive Learning
Sharut Gupta, Joshua Robinson, Derek Lim +2
Self-supervised learning converts raw perceptual data such as images to a compact space where simple Euclidean distances measure meaningful variations in data. In this paper, we ex…
Fine-grained Expressivity of Graph Neural Networks
Jan Böker, Ron Levie, Ningyuan Huang +2
Numerous recent works have analyzed the expressive power of message-passing graph neural networks (MPNNs), primarily utilizing combinatorial techniques such as the -dimensional…
Equivariant geometric convolutions for emulation of dynamical systems
Wilson G. Gregory, David W. Hogg, Ben Blum-Smith +3
Machine learning methods are increasingly being employed as surrogate models in place of computationally expensive and slow numerical integrators for a bevy of applications in the…
Sketch-and-solve approaches to k-means clustering by semidefinite programming
Charles Clum, Dustin G. Mixon, Soledad Villar +1
We introduce a sketch-and-solve approach to speed up the Peng-Wei semidefinite relaxation of k-means clustering. When the data is appropriately separated we identify the k-means op…
Prospective Learning: Principled Extrapolation to the Future
Ashwin De Silva, Rahul Ramesh, Lyle Ungar +40
Learning is a process which can update decision rules, based on past experience, such that future performance improves. Traditionally, machine learning is often evaluated under the…
A simple equivariant machine learning method for dynamics based on scalars
Weichi Yao, Kate Storey-Fisher, David W. Hogg +1
Physical systems obey strict symmetry principles. We expect that machine learning methods that intrinsically respect these symmetries should have higher prediction accuracy and bet…
Scalars are universal: Equivariant machine learning, structured like classical physics
Soledad Villar, David W. Hogg, Kate Storey-Fisher +2
There has been enormous progress in the last few years in designing neural networks that respect the fundamental symmetries and coordinate freedoms of physical law. Some of these f…
Can Graph Neural Networks Count Substructures?
Zhengdao Chen, Lei Chen, Soledad Villar +1
The ability to detect and count certain substructures in graphs is important for solving many tasks on graph-structured data, especially in the contexts of computational chemistry…
On the equivalence between graph isomorphism testing and function approximation with GNNs
Zhengdao Chen, Soledad Villar, Lei Chen +1
Graph Neural Networks (GNNs) have achieved much success on graph-structured data. In light of this, there have been increasing interests in studying their expressive power. One lin…
SUNLayer: Stable denoising with generative networks
Ruhui Jin, Dustin G. Mixon, Soledad Villar
Deep neural networks are often used to implement powerful generative models for real-world data. Notable applications include image denoising, as well as other classical inverse pr…