Transformers as Unsupervised Learning Algorithms: A study on Gaussian Mixtures

Zhiheng Chen, Ruofan Wu, Guanhua Fang

The transformer architecture has demonstrated remarkable capabilities in modern artificial intelligence, among which the capability of implicitly learning an internal model during…