Cost-Saving LLM Cascades with Early Abstention

Michael J. Zellinger, Rex Liu, Matt Thomson

LLM cascades deploy small LLMs to answer most queries, limiting the use of large and expensive LLMs to difficult queries. This approach can significantly reduce costs without impac…