SpecFed: Accelerating Federated LLM Inference with Speculative Decoding and Compressed Transmission

Ce Zheng, Xinghan Wang, Jiahong Ning +3

Federated inference enhances LLM performance in edge computing through weighted averaging of distributed model predictions. However, autoregressive LLM inference requires frequent…