CacheSpec: Finding the Sweet Spot for Small Models in Large Language Models

Jingquan Chen, Jinghua Piao, Jie Feng +2

Large language models (LLMs) are increasingly used for program-aided reasoning, agentic decision making, and structured task execution, but these settings often incur substantial i…