6 citations · 21 across the 21 of their papers we have counts for
9 papers · 1 filter
On the Long Range Abilities of Transformers
Itamar Zimerman, Lior Wolf
Despite their dominance in modern DL and, especially, NLP domains, transformer architectures exhibit sub-optimal performance on long-range tasks compared to recent layers that are…
Converting Transformers to Polynomial Form for Secure Inference Over Homomorphic Encryption
Itamar Zimerman, Moran Baruch, Nir Drucker +3
Designing privacy-preserving deep learning models is a major challenge within the deep learning community. Homomorphic Encryption (HE) has emerged as one of the most promising appr…
Multi-Dimensional Hyena for Spatial Inductive Bias
Itamar Zimerman, Lior Wolf
In recent years, Vision Transformers have attracted increasing interest from computer vision researchers. However, the advantage of these transformers over CNNs is only fully manif…
Efficient Skip Connections Realization for Secure Inference on Encrypted Data
Nir Drucker, Itamar Zimerman
Homomorphic Encryption (HE) is a cryptographic tool that allows performing computation under encryption, which is used by many privacy-preserving machine learning solutions, for ex…
2-D SSM: A General Spatial Layer for Visual Transformers
Ethan Baron, Itamar Zimerman, Lior Wolf
A central objective in computer vision is to design models with appropriate 2-D inductive bias. Desiderata for 2D inductive bias include two-dimensional position awareness, dynamic…
Decision S4: Efficient Sequence-Based RL via State Spaces Layers
Shmuel Bar-David, Itamar Zimerman, Eliya Nachmani +1
Recently, sequence learning methods have been applied to the problem of off-policy Reinforcement Learning, including the seminal work on Decision Transformers, which employs transf…
Focus Your Attention (with Adaptive IIR Filters)
Shahar Lutati, Itamar Zimerman, Lior Wolf
We present a new layer in which dynamic (i.e.,input-dependent) Infinite Impulse Response (IIR) filters of order two are used to process the input sequence prior to applying convent…
Algorithms for Reconstructing DDoS Attack Graphs using Probabilistic Packet Marking
Dina Barak-Pelleg, Daniel Berend, Thomas J. Robinson +1
DoS and DDoS attacks are widely used and pose a constant threat. Here we explore Probability Packet Marking (PPM), one of the important methods for reconstructing the attack-graph…
Training Large Scale Polynomial CNNs for E2E Inference over Homomorphic Encryption
Moran Baruch, Nir Drucker, Gilad Ezov +5
Training large-scale CNNs that during inference can be run under Homomorphic Encryption (HE) is challenging due to the need to use only polynomial operations. This limits HE-based…