A Meta-Learning Perspective on Transformers for Causal Language Modeling

Xinbo Wu, Lav R. Varshney

The Transformer architecture has become prominent in developing large causal language models. However, mechanisms to explain its capabilities are not well understood. Focused on th…