4 citations · 9 across the 17 of their papers we have counts for
11 papers · 1 filter
Cost-Aware Learning
Clara Mohri, Amir Globerson, Haim Kaplan +2
We consider the problem of Cost-Aware Learning, where sampling different components of a finite-sum objective incurs different costs. The objective is to reach a target error while…
Lost in Tokenization: Fundamental Trade-offs in Graph Tokenization for Transformers
Maya Bechler-Speicher, Gilad Yehudai, Gil Harari +3
Transformers have become a central architecture for graph learning, but their application to graphs requires first choosing a tokenization: a graph-to-token map that determines whi…
Depth-Width tradeoffs in Algorithmic Reasoning of Graph Tasks with Transformers
Gilad Yehudai, Clayton Sanford, Maya Bechler-Speicher +3
Transformers have revolutionized the field of machine learning. In particular, they can be used to solve complex algorithmic problems, including graph-based tasks. In such algorith…
Towards Invariance to Node Identifiers in Graph Neural Networks
Maya Bechler-Speicher, Moshe Eliasof, Carola-Bibiane Schonlieb +2
Message-Passing Graph Neural Networks (GNNs) are known to have limited expressive power, due to their message passing structure. One mechanism for circumventing this limitation is…
On the Expressivity of Selective State-Space Layers: A Multivariate Polynomial Approach
Edo Cohen-Karlik, Itamar Zimerman, Liane Galanti +3
Recent advances in efficient sequence modeling have introduced selective state-space layers, a key component of the Mamba architecture, which have demonstrated remarkable success i…
On the Utilization of Unique Node Identifiers in Graph Neural Networks
Maya Bechler-Speicher, Moshe Eliasof, Carola-Bibiane Schönlieb +2
Graph Neural Networks have inherent representational limitations due to their message-passing structure. Recent work has suggested that these limitations can be overcome by using u…
Provable Benefits of Complex Parameterizations for Structured State Space Models
Yuval Ran-Milo, Eden Lumbroso, Edo Cohen-Karlik +3
Structured state space models (SSMs), the core engine behind prominent neural networks such as S4 and Mamba, are linear dynamical systems adhering to a specified structure, most no…
DeciMamba: Exploring the Length Extrapolation Potential of Mamba
Assaf Ben-Kish, Itamar Zimerman, Shady Abu-Hussein +4
Long-range sequence processing poses a significant challenge for Transformers due to their quadratic complexity in input length. A promising alternative is Mamba, which demonstrate…
The Interpretable and Effective Graph Neural Additive Networks
Maya Bechler-Speicher, Amir Globerson, Ran Gilad-Bachrach
Graph Neural Networks (GNNs) have emerged as the predominant approach for learning over graph-structured data. However, most GNNs operate as black-box models and require post-hoc e…
Implicit Bias of Policy Gradient in Linear Quadratic Control: Extrapolation to Unseen Initial States
Noam Razin, Yotam Alexander, Edo Cohen-Karlik +3
In modern machine learning, models can often fit training data in numerous ways, some of which perform well on unseen (test) data, while others do not. Remarkably, in such cases gr…
Graph Neural Networks Use Graphs When They Shouldn't
Maya Bechler-Speicher, Ido Amos, Ran Gilad-Bachrach +1
Predictions over graphs play a crucial role in various domains, including social networks and medicine. Graph Neural Networks (GNNs) have emerged as the dominant approach for learn…