Model Compression and Efficient Inference for Large Language Models: A Survey

Wenxiao Wang, Wei Chen, Yicong Luo +6

Transformer based large language models have achieved tremendous success. However, the significant memory and computational costs incurred during the inference process make it chal…