Transformer Circuits Can Realize Clustering Algorithms

Kenneth L. Clarkson, Lior Horesh, Takuya Ito +2

Although transformers are most commonly optimized as statistical sequence models, it is unclear to what extent they can implement and learn exact algorithmic computations. Here, we…