Measuring Maximum Activations in Open Large Language Models

Luxuan Chen, Han Tian, Xinran Chen +9

The dynamic range of activations is a first-order constraint for low-bit quantization, activation scaling, and stable LLM inference. Prior work characterized outlier features and m…