2.2k citations · 5.3k across the 18 of their papers we have counts for
20 papers
More Value per Key: Asymmetric Sparse Attention for Faster LLM Decoding
Noam Elata, Itay Lamprecht, Mikey Shechter +3
Autoregressive generation in Large Language Models (LLMs) is constrained by the memory and computational demands of attention mechanisms. Sparse attention methods mitigate this cos…
Block Sparse Flash Attention
Daniel Ohayon, Itay Lamprecht, Itay Hubara +3
Modern large language models increasingly require long contexts for reasoning and multi-document tasks, but attention's quadratic complexity creates a severe computational bottlene…
Foldable SuperNets: Scalable Merging of Transformers with Different Initializations and Tasks
Edan Kinderman, Itay Hubara, Haggai Maron +1
Recent methods aim to merge neural networks (NNs) with identical architectures trained on different tasks into a single multi-task model. While most works focus on the simpler setu…
Towards Cheaper Inference in Deep Networks with Lower Bit-Width Accumulators
Yaniv Blumenfeld, Itay Hubara, Daniel Soudry
The majority of the research on the quantization of Deep Neural Networks (DNNs) is focused on reducing the precision of tensors visible by high-level frameworks (e.g., weights, act…
Minimum Variance Unbiased N:M Sparsity for the Neural Gradients
Brian Chmiel, Itay Hubara, Ron Banner +1
In deep learning, fine-grained N:M sparsity reduces the data footprint and bandwidth of a General Matrix multiply (GEMM) up to x2, and doubles throughput by skipping computation of…
Accelerated Sparse Neural Training: A Provable and Efficient Method to Find N:M Transposable Masks
Itay Hubara, Brian Chmiel, Moshe Island +3
Unstructured pruning reduces the memory footprint in deep neural networks (DNNs). Recently, researchers proposed different types of structural pruning intending to reduce also the…