Convergence Theory of Knowledge Distillation in Asynchronous P2P Gossip Learning Network

Lucas Qingyang Fang, Tiyao Liu, Jinhao Jing +4

Decentralized, serverless learning increasingly connects devices running different architectures, where the standard tool, decentralized SGD, is undefined as models with different…