2 citations · 5 across the 10 of their papers we have counts for
10 papers · 1 filter
Transformers Learn Two Competing Sorting Algorithms, and Only One Length-Generalizes
Khashayar Gatmiry, Nathan W. Henry, Chloe H. Su +5
We show that two-layer transformers trained to near-zero loss on a sorting task converge to one of two mechanistically distinct solution classes depending on their initialization.…
Train Small, Deploy Large: Zero-Shot GNN Transfer Through Geometric Renormalization
Robert Jankowski, Pedro Almagro-Blanco, Marián Boguñá +2
Graph neural networks (GNNs) can operate on large graphs but become infrastructure-sensitive at the scale of millions of nodes and typically require scalable training techniques fo…
Contrastive Neural Algorithmic Reasoning for Graph Coloring
Thien Le, Tianyu Zhao, Melanie Weber
Graph coloring seeks to assigns colors to a graph's nodes so that adjacent nodes receive different colors, using as few colors as possible. Here, we study approximate -coloring,…
Towards Distillation Guarantees under Algorithmic Alignment for Combinatorial Optimization
Thien Le, Melanie Weber
Distillation transfers knowledge from a large model trained on broad data to a smaller, more efficient model suitable for deployment. In structured prediction settings, prior knowl…
Performance Heterogeneity in Graph Neural Networks: Lessons for Architecture Design and Preprocessing
Lukas Fesser, Melanie Weber
Graph Neural Networks have emerged as the most popular architecture for graph-level learning, including graph classification and regression tasks, which frequently arise in areas s…
Enhancing the Utility of Higher-Order Information in Relational Learning
Raphael Pellegrin, Lukas Fesser, Melanie Weber
Higher-order information is crucial for relational learning in many domains where relationships extend beyond pairwise interactions. Hypergraphs provide a natural framework for mod…
Unitary convolutions for learning on graphs and groups
Bobak T. Kiani, Lukas Fesser, Melanie Weber
Data with geometric structure is ubiquitous in machine learning often arising from fundamental symmetries in a domain, such as permutation-invariance in graphs and translation-inva…
Hardness of Learning Neural Networks under the Manifold Hypothesis
Bobak T. Kiani, Jason Wang, Melanie Weber
The manifold hypothesis presumes that high-dimensional data lies on or near a low-dimensional manifold. While the utility of encoding geometric structure has been demonstrated empi…
On the hardness of learning under symmetries
Bobak T. Kiani, Thien Le, Hannah Lawrence +2
We study the problem of learning equivariant neural networks via gradient descent. The incorporation of known symmetries ("equivariance") into neural nets has empirically improved…
Effective Structural Encodings via Local Curvature Profiles
Lukas Fesser, Melanie Weber
Structural and Positional Encodings can significantly improve the performance of Graph Neural Networks in downstream tasks. Recent literature has begun to systematically investigat…