How Well Does Self-Supervised Pre-Training Perform with Streaming Data?

Dapeng Hu, Shipeng Yan, Qizhengqiu Lu +6

Prior works on self-supervised pre-training focus on the joint training scenario, where massive unlabeled data are assumed to be given as input all at once, and only then is a lear…