Optimal Model Activation Policies for Inference Networks of Large Language Models

Foivos Charalampakos, Md Ibrahim Ibne Alam, Iordanis Koutsopoulos +1

Recent advances in large language models (LLMs) have rendered them necessary for Natural Language Processing (NLP) tasks, and their high inference cost motivates the study of cost-…