Words That Make Language Models Perceive

Sophie L. Wang, Phillip Isola, Brian Cheung

Large language models (LLMs) trained purely on text ostensibly lack any direct perceptual experience, yet their internal representations are implicitly shaped by multimodal regular…