Forecasting LLM Inference Performance via Hardware-Agnostic Analytical Modeling

Rajeev Patwari, Ashish Sirasao, Devleena Das

Large language models (LLMs) have been increasingly deployed as local agents on personal devices with CPUs, NPUs and integrated GPUs. However, forecasting inference performance on…