SPADE: Speculative Decoding for Precise and Low Cost Distributed Edge Cloud Inference

Divya Jyoti Bajpai, Kishan Kumar Upadhyay, Manjesh Kumar Hanawal

Large Language Models (LLMs) have achieved remarkable success in natural language understanding and generation, but their deployment is constrained by high computational demands. D…