4 citations · 9 across the 17 of their papers we have counts for
17 papers
LMs as Task-Specific Knowledge Bases: An Interpretability Analysis
Amit Elhelo, Amir Globerson, Mor Geva
Language models (LMs) capture large amounts of factual knowledge applicable to a wide range of tasks, motivating the view of their parameters as a knowledge base. An important prop…
Cost-Aware Learning
Clara Mohri, Amir Globerson, Haim Kaplan +2
We consider the problem of Cost-Aware Learning, where sampling different components of a finite-sum objective incurs different costs. The objective is to reach a target error while…
Lost in Tokenization: Fundamental Trade-offs in Graph Tokenization for Transformers
Maya Bechler-Speicher, Gilad Yehudai, Gil Harari +3
Transformers have become a central architecture for graph learning, but their application to graphs requires first choosing a tokenization: a graph-to-token map that determines whi…
Controllable User Simulation
Guy Tennenholtz, Ofer Meshi, Amir Globerson +3
Using offline datasets to evaluate conversational agents often fails to cover rare scenarios or to support testing new policies. This has motivated the use of controllable user sim…
Latent Reasoning with Supervised Thinking States
Ido Amos, Avi Caciularu, Mor Geva +4
Reasoning with a chain-of-thought (CoT) enables Large Language Models (LLMs) to solve complex tasks but incurs significant inference costs due to the generation of long rationales.…
Depth-Width tradeoffs in Algorithmic Reasoning of Graph Tasks with Transformers
Gilad Yehudai, Clayton Sanford, Maya Bechler-Speicher +3
Transformers have revolutionized the field of machine learning. In particular, they can be used to solve complex algorithmic problems, including graph-based tasks. In such algorith…
Towards Invariance to Node Identifiers in Graph Neural Networks
Maya Bechler-Speicher, Moshe Eliasof, Carola-Bibiane Schonlieb +2
Message-Passing Graph Neural Networks (GNNs) are known to have limited expressive power, due to their message passing structure. One mechanism for circumventing this limitation is…
On the Expressivity of Selective State-Space Layers: A Multivariate Polynomial Approach
Edo Cohen-Karlik, Itamar Zimerman, Liane Galanti +3
Recent advances in efficient sequence modeling have introduced selective state-space layers, a key component of the Mamba architecture, which have demonstrated remarkable success i…
On the Utilization of Unique Node Identifiers in Graph Neural Networks
Maya Bechler-Speicher, Moshe Eliasof, Carola-Bibiane Schönlieb +2
Graph Neural Networks have inherent representational limitations due to their message-passing structure. Recent work has suggested that these limitations can be overcome by using u…
Provable Benefits of Complex Parameterizations for Structured State Space Models
Yuval Ran-Milo, Eden Lumbroso, Edo Cohen-Karlik +3
Structured state space models (SSMs), the core engine behind prominent neural networks such as S4 and Mamba, are linear dynamical systems adhering to a specified structure, most no…
When Can Transformers Count to n?
Gilad Yehudai, Haim Kaplan, Guy Dar +4
Large language models based on the transformer architecture can solve highly complex tasks, yet their fundamental limitations on simple algorithmic problems remain poorly understoo…
DeciMamba: Exploring the Length Extrapolation Potential of Mamba
Assaf Ben-Kish, Itamar Zimerman, Shady Abu-Hussein +4
Long-range sequence processing poses a significant challenge for Transformers due to their quadratic complexity in input length. A promising alternative is Mamba, which demonstrate…
Hopping Too Late: Exploring the Limitations of Large Language Models on Multi-Hop Queries
Eden Biran, Daniela Gottesman, Sohee Yang +2
Large language models (LLMs) can solve complex multi-step problems, but little is known about how these computations are implemented internally. Motivated by this, we study how LLM…
The Interpretable and Effective Graph Neural Additive Networks
Maya Bechler-Speicher, Amir Globerson, Ran Gilad-Bachrach
Graph Neural Networks (GNNs) have emerged as the predominant approach for learning over graph-structured data. However, most GNNs operate as black-box models and require post-hoc e…
Implicit Bias of Policy Gradient in Linear Quadratic Control: Extrapolation to Unseen Initial States
Noam Razin, Yotam Alexander, Edo Cohen-Karlik +3
In modern machine learning, models can often fit training data in numerous ways, some of which perform well on unseen (test) data, while others do not. Remarkably, in such cases gr…
In-Context Learning Creates Task Vectors
Roee Hendel, Mor Geva, Amir Globerson
In-context learning (ICL) in Large Language Models (LLMs) has emerged as a powerful new learning paradigm. However, its underlying mechanism is still not well understood. In partic…
Graph Neural Networks Use Graphs When They Shouldn't
Maya Bechler-Speicher, Ido Amos, Ran Gilad-Bachrach +1
Predictions over graphs play a crucial role in various domains, including social networks and medicine. Graph Neural Networks (GNNs) have emerged as the dominant approach for learn…