CacheSpec: Finding the Sweet Spot for Small Models in Large Language Models

Jingquan Chen, Jie Feng, Jinghua Piao +2

Large language models (LLMs) are increasingly used for program-aided reasoning, agentic decision making, and structured task execution, but these settings often incur substantial i…