4 papers
Context Structure Reshapes the Representational Geometry of Language Models
Eghbal A. Hosseini, Yuxuan Li, Yasaman Bahri +2
Large Language Models (LLMs) have been shown to organize the representations of input sequences into straighter neural trajectories in their deep layers, which has been hypothesize…
Just-in-time and distributed task representations in language models
Yuxuan Li, Declan Campbell, Stephanie C. Y. Chan +1
Many of language models' impressive capabilities originate from their in-context learning: based on instructions or examples, they can infer and perform new tasks without weight up…
Visual symbolic mechanisms: Emergent symbol processing in vision language models
Rim Assouel, Declan Campbell, Yoshua Bengio +1
To accurately process a visual scene, observers must bind features together to represent individual objects. This capacity is necessary, for instance, to distinguish an image conta…
Emergent Symbolic Mechanisms Support Abstract Reasoning in Large Language Models
Yukang Yang, Declan Campbell, Kaixuan Huang +3
Many recent studies have found evidence for emergent reasoning capabilities in large language models (LLMs), but debate persists concerning the robustness of these capabilities, an…