An evolutionary perspective on modes of learning in Transformers

Alexander Y. Ku, Thomas L. Griffiths, Stephanie C. Y. Chan

The success of Transformers lies in their ability to improve inference through two complementary strategies: the permanent refinement of model parameters via in-weight learning (IW…