26 citations · 50 across the 20 of their papers we have counts for
9 papers · 1 filter
Pushing the Limits of Large Language Model Quantization via the Linearity Theorem
Vladimir Malinovskii, Andrei Panferov, Ivan Ilin +3
Quantizing large language models has become a standard way to reduce their memory and computational costs. Typically, existing methods focus on breaking down the problem into indiv…
MARLIN: Mixed-Precision Auto-Regressive Parallel Inference on Large Language Models
Elias Frantar, Roberto L. Castro, Jiale Chen +2
As inference on Large Language Models (LLMs) emerges as an important workload in machine learning applications, weight quantization has become a standard technique for efficient GP…
QUIK: Towards End-to-End 4-Bit Inference on Generative Large Language Models
Saleh Ashkboos, Ilia Markov, Elias Frantar +5
Large Language Models (LLMs) from the GPT family have become extremely popular, leading to a race towards reducing their inference costs to allow for efficient local computation. Y…
AsGrad: A Sharp Unified Analysis of Asynchronous-SGD Algorithms
Rustem Islamov, Mher Safaryan, Dan Alistarh
We analyze asynchronous-type algorithms for distributed SGD in the heterogeneous setting, where each worker has its own computation and communication speeds, as well as data distri…
QMoE: Practical Sub-1-Bit Compression of Trillion-Parameter Models
Elias Frantar, Dan Alistarh
Mixture-of-Experts (MoE) architectures offer a general solution to the high inference costs of large language models (LLMs) via sparse routing, bringing faster and more accurate mo…
Scaling Laws for Sparsely-Connected Foundation Models
Elias Frantar, Carlos Riquelme, Neil Houlsby +2
We explore the impact of parameter sparsity on the scaling behavior of Transformers trained on massive datasets (i.e., "foundation models"), in both vision and language domains. In…