12 papers
CrossGMN: Graph Metanetworks for Cross-Architecture Weight-Space Transformations
Adir Dayan, Yam Eitan, Haggai Maron
Weight-space networks operate directly on parameters of other neural networks, enabling tasks such as predicting model properties, editing trained models, and generating weights. W…
Transferable Graph Metanetworks
Yuxin Ma, Adir Dayan, Yam Eitan +2
A weight space network (or metanetwork) takes the weights of another neural network as input and predicts properties of it. Most prior work trains such models on input networks of…
Permutation-Equivariant Flow Matching for Alignment-Free Neural Weight Generation
Arkadi Piven, Yam Eitan, Guy Bar-Shalom +5
A trained neural network can be represented by a parameter vector in high dimensions. Learning distributions over these vectors enables the generation of new models across various…
Training Transformers for KV Cache Compressibility
Yoav Gelberg, Yam Eitan, Michael Bronstein +2
Long-context language modeling is increasingly constrained by the Key-Value (KV) cache, whose memory and decode-time access costs scale linearly with the prefix length. This bottle…
On the Expressive Power of Permutation-Equivariant Weight-Space Networks
Adir Dayan, Yam Eitan, Haggai Maron
Weight-space learning studies neural architectures that operate directly on the parameters of other neural networks. Motivated by the growing availability of pretrained models, rec…
On The Expressive Power of GNN Derivatives
Yam Eitan, Moshe Eliasof, Yoav Gelberg +3
Despite significant advances in Graph Neural Networks (GNNs), their limited expressivity remains a fundamental challenge. Research on GNN expressivity has produced many expressive…
FS-KAN: Permutation Equivariant Kolmogorov-Arnold Networks via Function Sharing
Ran Elbaz, Guy Bar-Shalom, Yam Eitan +2
Permutation equivariant neural networks employing parameter-sharing schemes have emerged as powerful models for leveraging a wide range of data symmetries, significantly enhancing…
GradMetaNet: An Equivariant Architecture for Learning on Gradients
Yoav Gelberg, Yam Eitan, Aviv Navon +5
Gradients of neural networks encode valuable information for optimization, editing, and analysis of models. Therefore, practitioners often treat gradients as inputs to task-specifi…
Balancing Efficiency and Expressiveness: Subgraph GNNs with Walk-Based Centrality
Joshua Southern, Yam Eitan, Guy Bar-Shalom +3
Subgraph GNNs have emerged as promising architectures that overcome the expressiveness limitations of Graph Neural Networks (GNNs) by processing bags of subgraphs. Despite their co…
Topological Blindspots: Understanding and Extending Topological Deep Learning Through the Lens of Expressivity
Yam Eitan, Yoav Gelberg, Guy Bar-Shalom +3
Topological deep learning (TDL) is a rapidly growing field that seeks to leverage topological structure in data and facilitate learning from data supported on topological objects,…
A Flexible, Equivariant Framework for Subgraph GNNs via Graph Products and Graph Coarsening
Guy Bar-Shalom, Yam Eitan, Fabrizio Frasca +1
Subgraph GNNs enhance message-passing GNNs expressivity by representing graphs as sets of subgraphs, demonstrating impressive performance across various tasks. However, their scala…
The centered convex body whose marginals have the heaviest tails
Yam Eitan
Given any real numbers , we study the norm ratio (i.e. the ratio between the -norm and the -norm) of marginals of centered convex bodies. We first show that some margi…