From the 1 of 9 linked papers with an AI index.
6 papers · 1 filter
WUSH: Near-Optimal Adaptive Transforms for LLM Quantization
Jiale Chen, Vage Egiazarian, Roberto L. Castro +2
Quantizing LLM weights and activations is a standard approach for efficient deployment, but a few extreme outliers can stretch the dynamic range and amplify low-bit quantization er…
The Geometry of LLM Quantization: GPTQ as Babai's Nearest Plane Algorithm
Jiale Chen, Yalda Shabanzadeh, Elvir CrnÄeviÄ +2
Quantizing the weights of large language models (LLMs) from 16-bit to lower bitwidth is the de facto approach to deploy massive transformers onto more affordable accelerators. Whil…
Quartet: Native FP4 Training Can Be Optimal for Large Language Models
Roberto L. Castro, Andrei Panferov, Soroush Tabesh +5
Training large language models (LLMs) models directly in low-precision offers a way to address computational costs by improving both throughput and energy efficiency. For those pur…
QuEST: Stable Training of LLMs with 1-Bit Weights and Activations
Andrei Panferov, Jiale Chen, Soroush Tabesh +3
One approach to reducing the massive costs of large language models (LLMs) is the use of quantized or sparse representations for training or deployment. While post-training compres…
Scalable Mechanistic Neural Networks for Differential Equations and Machine Learning
Jiale Chen, Dingling Yao, Adeel Pervez +2
We propose Scalable Mechanistic Neural Network (S-MNN), an enhanced neural network framework designed for scientific machine learning applications involving long temporal sequences…
MARLIN: Mixed-Precision Auto-Regressive Parallel Inference on Large Language Models
Elias Frantar, Roberto L. Castro, Jiale Chen +2
As inference on Large Language Models (LLMs) emerges as an important workload in machine learning applications, weight quantization has become a standard technique for efficient GP…