4 papers · 1 filter
ThinQuant: Scalable Rotation Learning for Weight and Activation Quantization of LLMs
Mehdi Makni, Ryan Lucas, Rahul Mazumder
Learned rotations play an important role in enabling low-bit weight and activation quantization of large language models by smoothing outliers in the activation distribution. State…
UniPixie: Unified and Probabilistic 3D Physics Learning via Flow Matching
Qilin Huang, Quynh Anh Huynh, Long Le +5
Existing feed-forward networks excel at predicting a single set of physical properties from visual appearance, but this point-estimate paradigm fundamentally fails to capture the r…
ADMM-Q: An Improved Hessian-based Weight Quantizer for Post-Training Quantization of Large Language Models
Ryan Lucas, Mehdi Makni, Xiang Meng +2
Quantization is an effective strategy to reduce the storage and computation footprint of large language models (LLMs). Post-training quantization (PTQ) is a leading approach for co…
A GPU-accelerated Nonlinear Branch-and-Bound Framework for Sparse Linear Models
Ryan Lucas, Xiang Meng, Rahul Mazumder
We study exact sparse linear regression with an penalty and develop a branch-and-bound (BnB) algorithm explicitly designed for GPU execution. Starting from a perspe…