429 citations · 639 across the 30 of their papers we have counts for
8 papers · 1 filter
The Hidden Attention of Mamba Models
Ameen Ali, Itamar Zimerman, Lior Wolf
The Mamba layer offers an efficient selective state space model (SSM) that is highly effective in modeling multiple domains, including NLP, long-range sequence processing, and comp…
Dynamic Layer Tying for Parameter-Efficient Transformers
Tamir David Hay, Lior Wolf
In the pursuit of reducing the number of trainable parameters in deep transformer networks, we employ Reinforcement Learning to dynamically select layers during training and tie th…
Converting Transformers to Polynomial Form for Secure Inference Over Homomorphic Encryption
Itamar Zimerman, Moran Baruch, Nir Drucker +3
Designing privacy-preserving deep learning models is a major challenge within the deep learning community. Homomorphic Encryption (HE) has emerged as one of the most promising appr…
Decision S4: Efficient Sequence-Based RL via State Spaces Layers
Shmuel Bar-David, Itamar Zimerman, Eliya Nachmani +1
Recently, sequence learning methods have been applied to the problem of off-policy Reinforcement Learning, including the seminal work on Decision Transformers, which employs transf…
Centered Self-Attention Layers
Ameen Ali, Tomer Galanti, Lior Wolf
The self-attention mechanism in transformers and the message-passing mechanism in graph neural networks are repeatedly applied within deep learning architectures. We show that this…
Energy Regularized RNNs for Solving Non-Stationary Bandit Problems
Michael Rotman, Lior Wolf
We consider a Multi-Armed Bandit problem in which the rewards are non-stationary and are dependent on past actions and potentially on past contexts. At the heart of our method, we…