Model-Distributed Inference for Large Language Models at the Edge

Davide Macario, Hulya Seferoglu, Erdem Koyuncu

We introduce Model-Distributed Inference for Large-Language Models (MDI-LLM), a novel framework designed to facilitate the deployment of state-of-the-art large-language models (LLM…