Efficient Self-supervised Learning with Contextualized Target Representations for Vision, Speech and Language

Alexei Baevski, Arun Babu, Wei-Ning Hsu +1

Current self-supervised learning algorithms are often modality-specific and require large amounts of computational resources. To address these issues, we increase the training effi…