26 citations · 39 across the 14 of their papers we have counts for
7 papers · 1 filter
Mean-Field Parallel Decoding for Discrete Diffusion Language Models
Tamim Zoabi, Ameen Ali, Liran Ringel +1
Discrete diffusion language models enable parallel token generation, offering a pathway to low-latency decoding. However, selecting tokens independently by marginal confidence limi…
Explaining Modern Gated-Linear RNNs via a Unified Implicit Attention Formulation
Itamar Zimerman, Ameen Ali, Lior Wolf
Recent advances in efficient sequence modeling have led to attention-free layers, such as Mamba, RWKV, and various gated RNNs, all featuring sub-quadratic complexity in sequence le…
The Hidden Attention of Mamba Models
Ameen Ali, Itamar Zimerman, Lior Wolf
The Mamba layer offers an efficient selective state space model (SSM) that is highly effective in modeling multiple domains, including NLP, long-range sequence processing, and comp…
Degree-based stratification of nodes in Graph Neural Networks
Ameen Ali, Hakan Cevikalp, Lior Wolf
Despite much research, Graph Neural Networks (GNNs) still do not display the favorable scaling properties of other deep neural networks such as Convolutional Neural Networks and Tr…
Centered Self-Attention Layers
Ameen Ali, Tomer Galanti, Lior Wolf
The self-attention mechanism in transformers and the message-passing mechanism in graph neural networks are repeatedly applied within deep learning architectures. We show that this…
XAI for Transformers: Better Explanations through Conservative Propagation
Ameen Ali, Thomas Schnake, Oliver Eberle +3
Transformers have become an important workhorse of machine learning, with numerous applications. This necessitates the development of reliable methods for increasing their transpar…
Weakly Supervised Recovery of Semantic Attributes
Ameen Ali, Tomer Galanti, Evgeniy Zheltonozhskiy +2
We consider the problem of the extraction of semantic attributes, supervised only with classification labels. For example, when learning to classify images of birds into species, w…