ODIA: Oriented Distillation for Inline Acceleration of LLM-based Function Calling

Hanlong Zhang, Jingsheng Yang, Hao Li +2

Function Calling is a crucial technique that enables Large Language Models (LLMs) to interact with external systems through APIs. However, the high latency associated with LLM-base…