Attention Sinks in Diffusion Transformers: A Causal Analysis

Fangzheng Wu, Brian Summa

Attention sinks -- tokens that receive disproportionate attention mass -- are assumed to be functionally important in autoregressive language models, but their role in diffusion tr…