Progressive Mixed-Precision Decoding for Efficient LLM Inference

Hao Mark Chen, Fuwen Tan, Alexandros Kouris +3

In spite of the great potential of large language models (LLMs) across various tasks, their deployment on resource-constrained devices remains challenging due to their excessive co…