Colinearity Decay: Training Quantization-Friendly ViTs with Outlier Decay

Jin Tong, Guang Liang, Peilin Sun +1

Low-bit quantization is a practical route for efficiently deploying vision Transformers, yet activation outliers complicate fully quantized deployment. Existing methods either hand…