7 papers
ThinQuant: Scalable Rotation Learning for Weight and Activation Quantization of LLMs
Mehdi Makni, Ryan Lucas, Rahul Mazumder
Learned rotations play an important role in enabling low-bit weight and activation quantization of large language models by smoothing outliers in the activation distribution. State…
UniPixie: Unified and Probabilistic 3D Physics Learning via Flow Matching
Qilin Huang, Quynh Anh Huynh, Long Le +5
Existing feed-forward networks excel at predicting a single set of physical properties from visual appearance, but this point-estimate paradigm fundamentally fails to capture the r…
ADMM-Q: An Improved Hessian-based Weight Quantizer for Post-Training Quantization of Large Language Models
Ryan Lucas, Mehdi Makni, Xiang Meng +2
Quantization is an effective strategy to reduce the storage and computation footprint of large language models (LLMs). Post-training quantization (PTQ) is a leading approach for co…
A GPU-accelerated Nonlinear Branch-and-Bound Framework for Sparse Linear Models
Ryan Lucas, Xiang Meng, Rahul Mazumder
We study exact sparse linear regression with an penalty and develop a branch-and-bound (BnB) algorithm explicitly designed for GPU execution. Starting from a perspe…
Reasoning Models Can be Accurately Pruned Via Chain-of-Thought Reconstruction
Ryan Lucas, Kayhan Behdin, Zhipeng Wang +3
Reasoning language models such as DeepSeek-R1 produce long chain-of-thought traces during inference time which make them costly to deploy at scale. We show that using compression t…
Pixie: Fast and Generalizable Supervised Learning of 3D Physics from Pixels
Long Le, Ryan Lucas, Chen Wang +4
Inferring the physical properties of 3D scenes from visual information is a critical yet challenging task for creating interactive and realistic virtual worlds. While humans intuit…
Preserving Deep Representations In One-Shot Pruning: A Hessian-Free Second-Order Optimization Framework
Ryan Lucas, Rahul Mazumder
We present SNOWS, a one-shot post-training pruning framework aimed at reducing the cost of vision network inference without retraining. Current leading one-shot pruning methods min…