BERT-JEPA: Reorganizing CLS Embeddings for Language-Invariant Semantics

Taj Gillin, Adam Lalani, Kenneth Zhang +1

Joint Embedding Predictive Architectures (JEPA) are a novel self supervised training technique that have shown recent promise across domains. We introduce BERT-JEPA (BEPA), a train…