5 papers · 1 filter
ADMM-Q: An Improved Hessian-based Weight Quantizer for Post-Training Quantization of Large Language Models
Ryan Lucas, Mehdi Makni, Xiang Meng +2
Quantization is an effective strategy to reduce the storage and computation footprint of large language models (LLMs). Post-training quantization (PTQ) is a leading approach for co…
3BASiL: An Algorithmic Framework for Sparse plus Low-Rank Compression of LLMs
Mehdi Makni, Xiang Meng, Rahul Mazumder
Sparse plus Low-Rank decomposition of Large Language Models (LLMs) has emerged as a promising direction in model compression, aiming to decompose pre-t…
ALPS: Improved Optimization for Highly Sparse One-Shot Pruning for Large Language Models
Xiang Meng, Kayhan Behdin, Haoyue Wang +1
The impressive performance of Large Language Models (LLMs) across various natural language processing tasks comes at the cost of vast computational resources and storage requiremen…
TSENOR: Highly-Efficient Algorithm for Finding Transposable N:M Sparse Masks
Xiang Meng, Mehdi Makni, Rahul Mazumder
Network pruning reduces the computational requirements of large neural networks, with N:M sparsity -- retaining only N out of every M consecutive weights -- offering a compelling b…
FALCON: FLOP-Aware Combinatorial Optimization for Neural Network Pruning
Xiang Meng, Wenyu Chen, Riade Benbaki +1
The increasing computational demands of modern neural networks present deployment challenges on resource-constrained devices. Network pruning offers a solution to reduce model size…