Zero-shot Knowledge Transfer via Adversarial Belief Matching

Paul Micaelli, Amos Storkey

Performing knowledge transfer from a large teacher network to a smaller student is a popular task in modern deep learning applications. However, due to growing dataset sizes and st…