Voltron: Enabling Elastic Multi-Device Execution of LLM Inference for Empowered Edge Intelligence

Chanwoo Cho, Wooseok Kim, Yonglak Son +2

Large language models (LLMs) are widely used in intelligent services due to their remarkable capability in generative tasks. Typically, LLM-based services process the inference req…