Slimmable Networks for Contrastive Self-supervised Learning

Shuai Zhao, Linchao Zhu, Xiaohan Wang +1

Self-supervised learning makes significant progress in pre-training large models, but struggles with small models. Mainstream solutions to this problem rely mainly on knowledge dis…