LLM-JEPA: Large Language Models Meet Joint Embedding Predictive Architectures

Hai Huang, Yann LeCun, Randall Balestriero

Large Language Model (LLM) pretraining, finetuning, and evaluation rely on input-space reconstruction and generative capabilities. Yet, it has been observed in vision that embeddin…