49 papers
The Sparsity Whisperer
Linghao Kong, Inimai Subramanian, Micah Adler +3
Pruning reduces the inference cost of large language models, but existing criteria primarily preserve large activations or reconstruct layer outputs. We argue that this overlooks a…
GPTQ-2D: Cubic-Time Two-Sided Adaptive Rounding
Jiale Chen, Torsten Hoefler, Dan Alistarh
The paper introduces GPTQ-2D, an algorithm that rounds a real matrix to integers under a two-sided quadratic metric in cubic time by processing entries anti-diagonal by anti-diagon…
DarwinLM: Evolutionary Structured Pruning of Large Language Models
Shengkun Tang, Oliver Sieberling, Eldar Kurtic +2
The paper introduces DarwinLM, an evolutionary search method for training-aware structured pruning of large language models that integrates lightweight post‑pruning training to fin…
DASH: Faster Shampoo via Batched Block Preconditioning and Efficient Inverse-Root Solvers
Ionut-Vlad Modoranu, Philip Zmushko, Erik Schultheis +2
Shampoo is one of the leading approximate second-order optimizers: a variant of it has won the MLCommons AlgoPerf competition, and it has been shown to produce models with lower ac…
CAGE: Curvature-Aware Gradient Estimation For Accurate Quantization-Aware Training
Soroush Tabesh, Mher Safaryan, Andrei Panferov +2
Despite significant work on low-bit quantization-aware training (QAT), there is still an accuracy gap between such techniques and native training. To address this, we introduce CAG…
LoRDO: Distributed Low-Rank Optimization with Infrequent Communication
Andrej JovanoviÄ, Alex Iacob, Mher Safaryan +6
Distributed training of foundation models via is limited by interconnect bandwidth. While infrequent communication strategies reduce synchronization frequency, they…