5 papers · 1 filter
CrossGMN: Graph Metanetworks for Cross-Architecture Weight-Space Transformations
Adir Dayan, Yam Eitan, Haggai Maron
Weight-space networks operate directly on parameters of other neural networks, enabling tasks such as predicting model properties, editing trained models, and generating weights. W…
Transferable Graph Metanetworks
Yuxin Ma, Adir Dayan, Yam Eitan +2
A weight space network (or metanetwork) takes the weights of another neural network as input and predicts properties of it. Most prior work trains such models on input networks of…
Permutation-Equivariant Flow Matching for Alignment-Free Neural Weight Generation
Arkadi Piven, Yam Eitan, Guy Bar-Shalom +5
A trained neural network can be represented by a parameter vector in high dimensions. Learning distributions over these vectors enables the generation of new models across various…
Training Transformers for KV Cache Compressibility
Yoav Gelberg, Yam Eitan, Michael Bronstein +2
Long-context language modeling is increasingly constrained by the Key-Value (KV) cache, whose memory and decode-time access costs scale linearly with the prefix length. This bottle…
On the Expressive Power of Permutation-Equivariant Weight-Space Networks
Adir Dayan, Yam Eitan, Haggai Maron
Weight-space learning studies neural architectures that operate directly on the parameters of other neural networks. Motivated by the growing availability of pretrained models, rec…