307 citations · 693 across the 70 of their papers we have counts for
80 papers
Two For The Price Of One: Curvature-Aware Pruning Preserves Influence Functions Too
Mehdi Makni, Natalia Ponomareva, Rahul Mazumder
Post-training pruning enables efficient deployment, but conventional methods do not explicitly preserve internal-gradient alignment between sparse and dense models. We study novel…
ThinQuant: Scalable Rotation Learning for Weight and Activation Quantization of LLMs
Mehdi Makni, Ryan Lucas, Rahul Mazumder
Learned rotations play an important role in enabling low-bit weight and activation quantization of large language models by smoothing outliers in the activation distribution. State…
Large-Scale Regularized Matching on GPU Clusters
Aida Rahmattalabi, Gregory Dexter, Sanjana Garg +5
Production decision systems such as ad allocation or content matching involve millions of users and thousands of items, reducing to large-scale linear programs with sparse block-di…
ADMM-Q: An Improved Hessian-based Weight Quantizer for Post-Training Quantization of Large Language Models
Ryan Lucas, Mehdi Makni, Xiang Meng +2
Quantization is an effective strategy to reduce the storage and computation footprint of large language models (LLMs). Post-training quantization (PTQ) is a leading approach for co…
MOONSHOT : A Framework for Multi-Objective Pruning of Vision and Large Language Models
Gabriel Afriat, Xiang Meng, Shibal Ibrahim +2
Weight pruning is a common technique for compressing large neural networks. We focus on the challenging post-training one-shot setting, where a pre-trained model is compressed with…
A GPU-accelerated Nonlinear Branch-and-Bound Framework for Sparse Linear Models
Ryan Lucas, Xiang Meng, Rahul Mazumder
We study exact sparse linear regression with an penalty and develop a branch-and-bound (BnB) algorithm explicitly designed for GPU execution. Starting from a perspe…