σ-GPTs: A New Approach to Autoregressive Models

Arnaud Pannatier, Evann Courdier, François Fleuret

Autoregressive models, such as the GPT family, use a fixed order, usually left-to-right, to generate sequences. However, this is not a necessity. In this paper, we challenge this a…