A Hardware Evaluation Framework for Large Language Model Inference

Hengrui Zhang, August Ning, Rohan Prabhakar +1

The past year has witnessed the increasing popularity of Large Language Models (LLMs). Their unprecedented scale and associated high hardware cost have impeded their broader adopti…