Interactions Across Blocks in Post-Training Quantization of Large Language Models

Khasmamad Shabanovi, Lukas Wiest, Vladimir Golkov +2

Post-training quantization is widely employed to reduce the computational demands of neural networks. Typically, individual substructures, such as layers or blocks of layers, are q…