PipeMax: Enhancing Offline LLM Inference on Commodity GPU Servers

Hongbin Zhang, Taosheng Wei, Jiazhi Jiang +3

Offline LLM inference seeks to maximize request processing under fixed budgets, making commodity GPU servers a promising choice. However, prior work typically considers offloading…