Understanding Clustering in Slot Attention via Particle Dynamics

Vasudev Joy, Rajat Rasal, Avinash Kori +2

Studying attention through the lens of interacting particle dynamics has shown how token clustering can emerge from the underlying dynamics. We extend this perspective to slot atte…