464 citations · 491 across the 5 of their papers we have counts for
5 papers
FastDraft: How to Train Your Draft
Ofir Zafrir, Igor Margulis, Dorin Shteyman +2
Speculative Decoding has gained popularity as an effective technique for accelerating the auto-regressive inference process of Large Language Models. However, Speculative Decoding…
An Efficient Sparse Inference Software Accelerator for Transformer-based Language Models on CPUs
Haihao Shen, Hengyu Meng, Bo Dong +9
In recent years, Transformer-based language models have become the standard approach for natural language processing tasks. However, stringent throughput and latency requirements i…
Fast DistilBERT on CPUs
Haihao Shen, Ofir Zafrir, Bo Dong +7
Transformer-based language models have become the standard approach to solving natural language processing tasks. However, industry adoption usually requires the maximum throughput…
Prune Once for All: Sparse Pre-Trained Language Models
Ofir Zafrir, Ariel Larey, Guy Boudoukh +2
Transformer-based language models are applied to a wide range of applications in natural language processing. However, they are inefficient and difficult to deploy. In recent years…
Q8BERT: Quantized 8Bit BERT
Ofir Zafrir, Guy Boudoukh, Peter Izsak +1
Recently, pre-trained Transformer based language models such as BERT and GPT, have shown great improvement in many Natural Language Processing (NLP) tasks. However, these models co…