DILEMMA: Joint LLM Quantization and Distributed LLM Inference Over Edge Computing Systems

Minoo Hosseinzadeh, Hana Khamfroush

With a recent trend of using Large Language Models (LLMs) for different applications within smart cities, there is a need for pushing these models toward the edge of network while…