On the Failure of Boundary-Seeking Distillation in Bottlenecked Generative Architectures

Mohamed Amine Kina

Data-free knowledge distillation transfers the knowledge encoded in a teacher model to a student model without access to the original training data. Prior work such as Contrastive…