27 citations · 40 across the 32 of their papers we have counts for
32 papers · 1 filter
Graph Metanetworks for Transformer Based Language Models
Yonathan Wolanowsky, Guy Bar-Shalom, Fabrizio Frasca +1
Weight-space learning treats the weights of neural networks as data, processed by other models called metanetworks. A key challenge is that different weights can realize the same f…
CrossGMN: Graph Metanetworks for Cross-Architecture Weight-Space Transformations
Adir Dayan, Yam Eitan, Haggai Maron
Weight-space networks operate directly on parameters of other neural networks, enabling tasks such as predicting model properties, editing trained models, and generating weights. W…
Finite Probes Suffice: Identifiability and Universality for Weight-Space Learning
Soutrik Sarangi, Yonatan Sverdlov, Adir Dayan +2
Learning properties of neural networks has recently attracted growing interest, with existing approaches operating either directly on network parameters or through probe-based repr…
Permutation-Equivariant Flow Matching for Alignment-Free Neural Weight Generation
Arkadi Piven, Yam Eitan, Guy Bar-Shalom +5
A trained neural network can be represented by a parameter vector in high dimensions. Learning distributions over these vectors enables the generation of new models across various…
Training Transformers for KV Cache Compressibility
Yoav Gelberg, Yam Eitan, Michael Bronstein +2
Long-context language modeling is increasingly constrained by the Key-Value (KV) cache, whose memory and decode-time access costs scale linearly with the prefix length. This bottle…
A Survey of Weight Space Learning: Understanding, Representation, and Generation
Xiaolong Han, Zehong Wang, Bo Zhao +8
Neural network weights are typically viewed as the end product of training, while most deep learning research focuses on data, features, and architectures. However, recent advances…
A Graph Meta-Network for Learning on Kolmogorov-Arnold Networks
Guy Bar-Shalom, Ami Tavory, Itay Evron +3
Weight-space models learn directly from the parameters of neural networks, enabling tasks such as predicting their accuracy on new datasets. Naive methods -- like applying MLPs to…
On the Expressive Power of Permutation-Equivariant Weight-Space Networks
Adir Dayan, Yam Eitan, Haggai Maron
Weight-space learning studies neural architectures that operate directly on the parameters of other neural networks. Motivated by the growing availability of pretrained models, rec…
Learning from Historical Activations in Graph Neural Networks
Yaniv Galron, Hadar Sinai, Haggai Maron +1
Graph Neural Networks (GNNs) have demonstrated remarkable success in various domains such as social networks, molecular chemistry, and more. A crucial component of GNNs is the pool…
On The Expressive Power of GNN Derivatives
Yam Eitan, Moshe Eliasof, Yoav Gelberg +3
Despite significant advances in Graph Neural Networks (GNNs), their limited expressivity remains a fundamental challenge. Research on GNN expressivity has produced many expressive…
Beyond Token Probes: Hallucination Detection via Activation Tensors with ACT-ViT
Guy Bar-Shalom, Fabrizio Frasca, Yaniv Galron +2
Detecting hallucinations in Large Language Model-generated text is crucial for their safe deployment. While probing classifiers show promise, they operate on isolated layer-token p…
Neural Message-Passing on Attention Graphs for Hallucination Detection
Fabrizio Frasca, Guy Bar-Shalom, Yftah Ziser +1
Large Language Models (LLMs) often generate incorrect or unsupported content, known as hallucinations. Existing detection methods rely on heuristics or simple models over isolated…
FS-KAN: Permutation Equivariant Kolmogorov-Arnold Networks via Function Sharing
Ran Elbaz, Guy Bar-Shalom, Yam Eitan +2
Permutation equivariant neural networks employing parameter-sharing schemes have emerged as powerful models for leveraging a wide range of data symmetries, significantly enhancing…
GradMetaNet: An Equivariant Architecture for Learning on Gradients
Yoav Gelberg, Yam Eitan, Aviv Navon +5
Gradients of neural networks encode valuable information for optimization, editing, and analysis of models. Therefore, practitioners often treat gradients as inputs to task-specifi…
Understanding and Improving Laplacian Positional Encodings For Temporal GNNs
Yaniv Galron, Fabrizio Frasca, Haggai Maron +2
Temporal graph learning has applications in recommendation systems, traffic forecasting, and social network analysis. Although multiple architectures have been introduced, progress…
It Takes a Graph to Know a Graph: Rewiring for Homophily with a Reference Graph
Harel Mendelman, Haggai Maron, Ronen Talmon
Graph Neural Networks (GNNs) excel at analyzing graph-structured data but struggle on heterophilic graphs, where connected nodes often belong to different classes. While this chall…
Efficient GNN Training Through Structure-Aware Randomized Mini-Batching
Vignesh Balaji, Christos Kozyrakis, Gal Chechik +1
Graph Neural Networks (GNNs) enable learning on realworld graphs and mini-batch training has emerged as the de facto standard for training GNNs because it can scale to very large g…
Beyond Next Token Probabilities: Learnable, Fast Detection of Hallucinations and Data Contamination on LLM Output Distributions
Guy Bar-Shalom, Fabrizio Frasca, Derek Lim +5
The automated detection of hallucinations and training data contamination is pivotal to the safe deployment of Large Language Models (LLMs). These tasks are particularly challengin…
Balancing Efficiency and Expressiveness: Subgraph GNNs with Walk-Based Centrality
Joshua Southern, Yam Eitan, Guy Bar-Shalom +3
Subgraph GNNs have emerged as promising architectures that overcome the expressiveness limitations of Graph Neural Networks (GNNs) by processing bags of subgraphs. Despite their co…
Towards Foundation Models on Graphs: An Analysis on Cross-Dataset Transfer of Pretrained GNNs
Fabrizio Frasca, Fabian Jogl, Moshe Eliasof +4
To develop a preliminary understanding towards Graph Foundation Models, we study the extent to which pretrained Graph Neural Networks can be applied across datasets, an effort requ…
Foldable SuperNets: Scalable Merging of Transformers with Different Initializations and Tasks
Edan Kinderman, Itay Hubara, Haggai Maron +1
Recent methods aim to merge neural networks (NNs) with identical architectures trained on different tasks into a single multi-task model. While most works focus on the simpler setu…
Topological Blindspots: Understanding and Extending Topological Deep Learning Through the Lens of Expressivity
Yam Eitan, Yoav Gelberg, Guy Bar-Shalom +3
Topological deep learning (TDL) is a rapidly growing field that seeks to leverage topological structure in data and facilitate learning from data supported on topological objects,…
A Flexible, Equivariant Framework for Subgraph GNNs via Graph Products and Graph Coarsening
Guy Bar-Shalom, Yam Eitan, Fabrizio Frasca +1
Subgraph GNNs enhance message-passing GNNs expressivity by representing graphs as sets of subgraphs, demonstrating impressive performance across various tasks. However, their scala…
GRANOLA: Adaptive Normalization for Graph Neural Networks
Moshe Eliasof, Beatrice Bevilacqua, Carola-Bibiane Schönlieb +1
In recent years, significant efforts have been made to refine the design of Graph Neural Network (GNN) layers, aiming to overcome diverse challenges, such as limited expressive pow…