Probability Distributions Computed by Autoregressive Transformers

Andy Yang, Anej Svete, Jiaoda Li +4

Most expressivity results for transformers treat them as language recognizers -- devices that accept or reject strings -- rather than as they are used in practice: as language mode…