2 papers
cs.CL2024
First Activations Matter: Training-Free Methods for Dynamic Activation in Large Language Models
Chi Ma, Mincong Huang, Ying Zhang +5
Dynamic activation (DA) techniques, such as DejaVu and MoEfication, have demonstrated their potential to significantly enhance the inference efficiency of large language models (LL…
cs.LG2024
QQQ: Quality Quattuor-Bit Quantization for Large Language Models
Ying Zhang, Peng Zhang, Mincong Huang +7
Quantization is a proven effective method for compressing large language models. Although popular techniques like W8A8 and W4A16 effectively maintain model performance, they often…