Uncertainty-Aware Hybrid Inference with On-Device Small and Remote Large Language Models

Seungeun Oh, Jinhyuk Kim, Jihong Park +3

This paper studies a hybrid language model (HLM) architecture that integrates a small language model (SLM) operating on a mobile device with a large language model (LLM) hosted at…