Can "consciousness" be observed from large language model (LLM) internal states? Dissecting LLM representations obtained from Theory of Mind test with Integrated Information Theory and Span Representation analysis

Abstract

Integrated Information Theory (IIT) provides a quantitative framework for explaining consciousness phenomenon, positing that conscious systems comprise elements integrated through causal properties. We apply IIT 3.0 and 4.0 -- the latest iterations of this framework -- to sequences of Large Language Model (LLM) representations, analyzing data derived from existing Theory of Mind (ToM) test results. Our study systematically investigates whether the differences of ToM test performances, when presented in the LLM representations, can be revealed by IIT estimates, i.e., ${\Phi }^{\max }$ (IIT 3.0), $\Phi$ (IIT 4.0), Conceptual Information (IIT 3.0), and $\Phi$-structure (IIT 4.0). Furthermore, we compare these metrics with the Span Representations independent of any estimate for consciousness. This additional effort aims to differentiate between potential "consciousness" phenomena and inherent separations within LLM representational space. We conduct comprehensive experiments examining variations across LLM transformer layers and linguistic spans from stimuli. Our results suggest that sequences of contemporary Transformer-based LLM representations lack statistically significant indicators of observed "consciousness" phenomena but exhibit intriguing patterns under $\text{spatio}$-permutational analyses. The Appendix and code are available as Supplementary Materials at: https://doi.org/10.1016/j.nlp.2025.100163.

Published as a journal paper at: https://doi.org/10.1016/j.nlp.2025.100163

References in corpus (13)

• Neural Machine Translation by Jointly Learning to Align and Translate
• Sparks of Artificial General Intelligence: Early experiments with GPT-4
• Emergent Abilities of Large Language Models
• Large Language Models: A Survey
• Evaluating Large Language Models in Theory of Mind Tasks
• Consciousness in Artificial Intelligence: Insights from the Science of Consciousness
• GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers
• Could a Large Language Model be Conscious?
• PyPhi: A toolbox for integrated information theory
• Agent AI: Surveying the Horizons of Multimodal Interaction
• LLMs achieve adult human performance on higher-order theory of mind tasks
• Dissociating Artificial Intelligence from Artificial Consciousness
• Language models are better than humans at next-token prediction