AV-JEPA: Extending LeJEPA to Audio-Visual Self-Supervised Learning

Benjamin Robson, Santeri Mentu, Wenshuai Zhao +1

We present AV-JEPA, an elegant multimodal extension of LeJEPA to audio-visual self-supervised learning. Using an early-fusion Vision Transformer and modality dropout as masking, th…