On the Emergence and Test-Time Use of Structural Information in Large Language Models

Michelle Chao Chen, Moritz Miller, Bernhard Schölkopf +1

Learning structural information from observational data is central to producing new knowledge outside the training corpus. This holds for mechanistic understanding in scientific di…