The Effect of Batch Size on Contrastive Self-Supervised Speech Representation Learning

Nik Vaessen, David A. van Leeuwen

Foundation models in speech are often trained using many GPUs, which implicitly leads to large effective batch sizes. In this paper we study the effect of batch size on pre-trainin…