GaugeQuant: Online Learning of Quantization-Optimal Bases from LLM Symmetries

Miguel P. Bento, João Seabra, João F. Seabra

Transformers are known to have internal continuous symmetries that leave outputs invariant, while modifying quantization. GaugeQuant leverages this in-training by introducing a Log…