GGUF-Metadata Prediction of Single-Sequence llama.cpp Throughput Across Three Systems

Xinyu Qiu, Chuhong Xu, Bo Su +3

We predict single-sequence model throughput from GGUF metadata using roofline-shaped predictors with quantization-specific scale factors fitted on reference models. The scored coho…