Better Representations via Adversarial Training in Pre-Training: A Theoretical Perspective

Yue Xing, Xiaofeng Lin, Qifan Song +3

Pre-training is known to generate universal representations for downstream tasks in large-scale deep learning such as large language models. Existing literature, e.g., \cite{kim202…