Prediction-Assisted Pricing and Admission for LLM APIs with Stochastic Token Consumption

Patrick Wong

An LLM application often sells or internally allocates several service products: a small or premium model, a short or long token cap, and possibly multiple posted prices. The opera…