Online Pandora's Box for Contextual LLM Cascading

Alexandre Belloni, Yan Chen, Yehua Wei

Motivated by Large Language Model (LLM) cascading, we propose an online contextual Pandora's Box model for adaptively querying and selecting LLM APIs. In each period, a decision-ma…