Human Language Shapes the Emerging Physical Reasoning in Large Language Models

Abstract

Whether large language models (LLMs) acquire adequate representations of physical reality by predicting the next token remains unresolved. Molecular conformations provide a quantitative test: we asked 28 LLMs to rank 810 geometries of 27 molecules from atomic coordinates alone. Frontier LLMs approach the accuracy of specialized force fields, but their ability is bound to well-defined chemical concepts. They rank conformers with more hydrogen bonds as more stable, inverting the true order when the hydrogen-bonded conformer is less stable. On pairs with equal hydrogen-bond counts, they are most accurate for molecules whose explanations invoke hydrogen bonding, whereas molecules governed by ring strain remain difficult. Thus, the physical understanding emerging in LLMs inherits the structure and blind spots of human scientific discourse, a machine analogue of linguistic relativity.

arXiv comment32 pages; 3 main-text figures and 8 supplementary figures. Supplementary Materials are included. Code and data are available at https://github.com/TheorChemGroup/LLMConfBench