How do Language Models Bind Entities in Context?

Jiahai Feng, Jacob Steinhardt

To correctly use in-context information, language models (LMs) must bind entities to their attributes. For example, given a context describing a "green square" and a "blue circle",…