35 citations · 75 across the 19 of their papers we have counts for
21 papers
CogGym: Towards Large-Scale Comparative Evaluation of Human and Machine Cognition
Lance Ying, Jinzhou Wu, Yingshan Susan Wang +53
Understanding and modeling human intelligence are parallel goals shared by artificial intelligence (AI) and cognitive science. As AI systems grow increasingly capable, in what ways…
Sparks of In Silico Cognitive Science: Theories from Simulated Data Can Generalize to Humans
Akshay K. Jagadish, Younes Strittmatter, Nori Jacoby +4
Behavioral foundation models have been proposed as stand-ins for human participants across settings, but it is unclear whether theories discovered on them generalize to humans or m…
The Riddle Riddle: Testing Flexible Reasoning in Large Language Models and Humans
Bella Fascendini, Kathryn McGregor, Max D. Gupta +1
Humans flexibly adapt their reasoning strategies to the requirements of a given problem. Large language models (LLMs) have performed well on many cognitive tasks, however, it is un…
Closing the Loop to Discover Psychological Theories with an Automated Cognitive Scientist
Akshay K. Jagadish, Younes Strittmatter, Nori Jacoby +5
Across the sciences, autonomous systems are increasingly being used in closed-loop discovery, proposing new theories and designing and running experiments to test them. This approa…
Using Cognitive Models to Improve Language Model Simulation of Human Persuasion Games
Zirui Cheng, Zeyu Shen, Thomas L. Griffiths +1
People make decisions differently in strategic interactions. Some update beliefs like a Bayesian; others exhibit biases like motivated reasoning. Although creators of large languag…
Do Large Language Models Mentalize When They Teach?
Sevan K. Harootonian, Mark K. Ho, Thomas L. Griffiths +2
How do LLMs decide what to teach next: by reasoning about a learner's knowledge, or by using simpler rules of thumb? We test this in a controlled task previously used to study huma…
What is a Number, That a Large Language Model May Know It?
Raja Marjieh, Veniamin Veselovsky, Thomas L. Griffiths +1
Numbers are a basic part of how humans represent and describe the world around them. As a consequence, learning effective representations of numbers is critical for the success of…
On Benchmarking Human-Like Intelligence in Machines
Lance Ying, Katherine M. Collins, Lionel Wong +6
Recent advances in Artificial Intelligence (AI) have yielded powerful computational models that, by learning from vast amounts of human-generated data, are increasingly posited as…
RLHS: Mitigating Misalignment in RLHF with Hindsight Simulation
Kaiqu Liang, Haimin Hu, Ryan Liu +2
While Reinforcement Learning from Human Feedback (RLHF) has shown promise in aligning generative AI, we present empirical evidence that it can also cause severe, systematic misalig…
Mind Your Step (by Step): Chain-of-Thought can Reduce Performance on Tasks where Thinking Makes Humans Worse
Ryan Liu, Jiayi Geng, Addison J. Wu +3
Chain-of-thought (CoT) prompting has become a widely used strategy for improving large language and multimodal model performance. However, it is still an open question under which…
Large Language Models Assume People are More Rational than We Really are
Ryan Liu, Jiayi Geng, Joshua C. Peterson +2
In order for AI systems to communicate effectively with people, they must understand how we make decisions. However, people's decisions are not always rational, so the implicit int…
Learning with Language-Guided State Abstractions
Andi Peng, Ilia Sucholutsky, Belinda Z. Li +4
We describe a framework for using natural language to design state abstractions for imitation learning. Generalizable policy learning in high-dimensional observation spaces is faci…
Analyzing the Roles of Language and Vision in Learning from Limited Data
Allison Chen, Ilia Sucholutsky, Olga Russakovsky +1
Does language help make sense of the visual world? How important is it to actually see the world rather than having it described with words? These basic questions about the nature…
A Rational Analysis of the Speech-to-Song Illusion
Raja Marjieh, Pol van Rijn, Ilia Sucholutsky +3
The speech-to-song illusion is a robust psychological phenomenon whereby a spoken sentence sounds increasingly more musical as it is repeated. Despite decades of research, a comple…
Preference-Conditioned Language-Guided Abstraction
Andi Peng, Andreea Bobu, Belinda Z. Li +5
Learning from demonstrations is a common way for users to teach robots, but it is prone to spurious feature correlations. Recent work constructs state abstractions, i.e. visual rep…
Measuring Implicit Bias in Explicitly Unbiased Large Language Models
Xuechunzi Bai, Angelina Wang, Ilia Sucholutsky +1
Large language models (LLMs) can pass explicit social bias tests but still harbor implicit biases, similar to humans who endorse egalitarian beliefs yet exhibit subtle biases. Meas…
Concept Alignment
Sunayana Rane, Polyphony J. Bruna, Ilia Sucholutsky +2
Discussion of AI alignment (alignment between humans and AI systems) has focused on value alignment, broadly referring to creating AI systems that share human values. We argue that…
Learning Human-like Representations to Enable Learning Human Values
Andrea Wynn, Ilia Sucholutsky, Thomas L. Griffiths
How can we build AI systems that can learn any set of individual human values both quickly and safely, avoiding causing harm or violating societal standards for acceptable behavior…
Concept Alignment as a Prerequisite for Value Alignment
Sunayana Rane, Mark Ho, Ilia Sucholutsky +1
Value alignment is essential for building AI systems that can safely and reliably interact with people. However, what a person values -- and is even capable of valuing -- depends o…
Dimensions of Disagreement: Unpacking Divergence and Misalignment in Cognitive Science and Artificial Intelligence
Kerem Oktar, Ilia Sucholutsky, Tania Lombrozo +1
The increasing prevalence of artificial agents creates a correspondingly increasing need to manage disagreements between humans and artificial agents, as well as between artificial…
Getting aligned on representational alignment
Ilia Sucholutsky, Lukas Muttenthaler, Adrian Weller +30
Biological and artificial information processing systems form representations of the world that they can use to categorize, reason, plan, navigate, and make decisions. How can we m…