Towards Coarse-to-Fine Evaluation of Inference Efficiency for Large Language Models

Yushuo Chen, Tianyi Tang, Erge Xiang +5

In real world, large language models (LLMs) can serve as the assistant to help users accomplish their jobs, and also support the development of advanced applications. For the wide…