Context-Gated Associative Retrieval: From Theory to Transformers

Moulik Choraria, Argyrios Gerogiannis, Vidhata Jayaraman +2

Hopfield networks and their generalizations have established deep connections among biological associative memories, statistical physics, and transformers. Yet most models treat re…