26 citations · 39 across the 8 of their papers we have counts for
8 papers
MARLIN: Mixed-Precision Auto-Regressive Parallel Inference on Large Language Models
Elias Frantar, Roberto L. Castro, Jiale Chen +2
As inference on Large Language Models (LLMs) emerges as an important workload in machine learning applications, weight quantization has become a standard technique for efficient GP…
QUIK: Towards End-to-End 4-Bit Inference on Generative Large Language Models
Saleh Ashkboos, Ilia Markov, Elias Frantar +5
Large Language Models (LLMs) from the GPT family have become extremely popular, leading to a race towards reducing their inference costs to allow for efficient local computation. Y…
QMoE: Practical Sub-1-Bit Compression of Trillion-Parameter Models
Elias Frantar, Dan Alistarh
Mixture-of-Experts (MoE) architectures offer a general solution to the high inference costs of large language models (LLMs) via sparse routing, bringing faster and more accurate mo…
Sparse Fine-tuning for Inference Acceleration of Large Language Models
Eldar Kurtic, Denis Kuznedelev, Elias Frantar +2
We consider the problem of accurate sparse fine-tuning of large language models (LLMs), that is, fine-tuning pretrained LLMs on specialized tasks, while inducing sparsity in their…
Scaling Laws for Sparsely-Connected Foundation Models
Elias Frantar, Carlos Riquelme, Neil Houlsby +2
We explore the impact of parameter sparsity on the scaling behavior of Transformers trained on massive datasets (i.e., "foundation models"), in both vision and language domains. In…
Accurate Neural Network Pruning Requires Rethinking Sparse Optimization
Denis Kuznedelev, Eldar Kurtic, Eugenia Iofinova +3
Obtaining versions of deep neural networks that are both highly-accurate and highly-sparse is one of the main challenges in the area of model compression, and several high-performa…