River-LLM: Large Language Model Seamless Exit Based on KV Share

Yingtao Shen, An Zou

Large Language Models (LLMs) have demonstrated exceptional performance across diverse domains but are increasingly constrained by high inference latency. Early Exit has emerged as…