Learning the Cost of Reliable Inference

Dimitrios Rontogiannis, Ander Artola Velasco, Manuel Gomez Rodriguez

Benchmarking and routing platforms increasingly act as intermediaries connecting large language model providers with end-users. However, providers on these platforms typically use…