Pay for Hints, Not Answers: LLM Shepherding for Cost-Efficient Inference

Ziming Dong, Hardik Sharma, Evan O'Toole +2

Large Language Models (LLMs) deliver state-of-the-art performance on complex reasoning tasks, but their inference costs limit deployment at scale. Small Language Models (SLMs) offe…