8 citations · 13 across the 15 of their papers we have counts for
16 papers
Integrating Simulation and Chain-of-thought Reasoning in Multimodal-Language Models For Physical Reasoning
YingQiao Wang, Eric Bigelow, Tomer Ullman +2
Multimodal language models often need to reason about the physical dynamics of the world in order to answer questions appropriately. Various methods have emerged for solving physic…
Directing large language models to follow the letter or spirit of the law
Peng Qian, Andrew Li, Sam Chen +3
The distinction between the spirit and letter of the law is a central issue across research and everyday life, and a growing concern for building safe, intelligent machines. What i…
CogGym: Towards Large-Scale Comparative Evaluation of Human and Machine Cognition
Lance Ying, Jinzhou Wu, Yingshan Susan Wang +53
Understanding and modeling human intelligence are parallel goals shared by artificial intelligence (AI) and cognitive science. As AI systems grow increasingly capable, in what ways…
Agents of Chaos
Natalie Shapira, Chris Wendler, Avery Yen +35
We report an exploratory red-teaming study of autonomous language-model-powered agents deployed in a live laboratory environment with persistent memory, email accounts, Discord acc…
Modeling The Object Representations Underlying Human Physical Reasoning
Andrey Gizdov, Andrea Procopio, Yichen Li +4
Humans appear to represent objects when reasoning about physics with coarse, volumetric "bodies" that smooth concavities, trading fine visual detail for efficient physical predicti…
Towards aligned body representations in vision models
Andrey Gizdov, Andrea Procopio, Yichen Li +2
Human physical reasoning relies on internal "body" representations - coarse, volumetric approximations that capture an object's extent and support intuitive predictions about motio…
Belief Dynamics Reveal the Dual Nature of In-Context Learning and Activation Steering
Eric Bigelow, Daniel Wurgaft, YingQiao Wang +4
Large language models (LLMs) can be controlled at inference time through prompts (in-context learning) and internal activations (activation steering). Different accounts have been…
Chain of Time: In-Context Physical Simulation with Image Generation Models
YingQiao Wang, Eric Bigelow, Boyi Li +1
We propose a novel cognitively-inspired prompting method to stabilize and interpret the physical simulation produced by autoregressive image generation models, called Chain-of-Time…
Emergence of Hierarchical Emotion Organization in Large Language Models
Maya Okawa, Bo Zhao, Eric J. Bigelow +4
As large language models (LLMs) increasingly power conversational agents, understanding how they model users' emotional states is critical for ethical deployment. Inspired by emoti…
Cognitive models can reveal interpretable value trade-offs in language models
Sonia K. Murthy, Rosie Zhao, Jennifer Hu +4
Value trade-offs are an integral part of human decision-making and language use, however, current tools for interpreting such dynamic and multi-faceted notions of values in languag…
Re-evaluating Theory of Mind evaluation in large language models
Jennifer Hu, Felix Sosa, Tomer Ullman
The question of whether large language models (LLMs) possess Theory of Mind (ToM) -- often defined as the ability to reason about others' mental states -- has sparked significant s…
Shades of Zero: Distinguishing Impossibility from Inconceivability
Jennifer Hu, Felix Sosa, Tomer Ullman
Some things are impossible, but some things may be even more impossible than impossible. Levitating a feather using one's mind is impossible in our world, but fits into our intuiti…
Forking Paths in Neural Text Generation
Eric Bigelow, Ari Holtzman, Hidenori Tanaka +1
Estimating uncertainty in Large Language Models (LLMs) is important for properly evaluating LLMs, and ensuring safety for users. However, prior approaches to uncertainty estimation…
The Illusion-Illusion: Vision Language Models See Illusions Where There Are None
Tomer Ullman
Illusions are entertaining, but they are also a useful diagnostic tool in cognitive science, philosophy, and neuroscience. A typical illusion shows a gap between how something `rea…
Relations, Negations, and Numbers: Looking for Logic in Generative Text-to-Image Models
Colin Conwell, Rupert Tawiah-Quashie, Tomer Ullman
Despite remarkable progress in multi-modal AI research, there is a salient domain in which modern AI continues to lag considerably behind even human children: the reliable deployme…
One fish, two fish, but not the whole sea: Alignment reduces language models' conceptual diversity
Sonia K. Murthy, Tomer Ullman, Jennifer Hu
Researchers in social science and psychology have recently proposed using large language models (LLMs) as replacements for humans in behavioral research. In addition to arguments a…