XShare: Collaborative in-Batch Expert Sharing for Faster MoE Inference

Daniil Vankov, Nikita Ivkin, Kyle Ulrich +3

Mixture-of-Experts (MoE) architectures are increasingly used to efficiently scale large language models. However, in production inference, request batching and speculative decoding…