Cache & Distil: Optimising API Calls to Large Language Models

Guillem Ramírez, Matthias Lindemann, Alexandra Birch +1

Large-scale deployment of generative AI tools often depends on costly API calls to a Large Language Model (LLM) to fulfil user queries. To curtail the frequency of these calls, one…