RAPTOR: Role-Aware Private Training for Mixture-of-Experts

Duc Dm, Khai Le-Duc, Nguyen Do +18

Differentially private (DP) fine-tuning methods treat sparse Mixture-of-Experts (MoE) models as a single dense block, ignoring that shared layers see all data while experts only se…