4 papers
When Do Models Admit They Are Wrong? Failure Disclosure Is Unstable Under Reinforcement Learning
Steven Y. Feng, Noah D. Goodman, Michael C. Frank +3
Outcome-based reinforcement learning can produce models with similar task performance but very different ways of communicating about their mistakes. We study failure disclosure: wh…
CogGym: Towards Large-Scale Comparative Evaluation of Human and Machine Cognition
Lance Ying, Jinzhou Wu, Yingshan Susan Wang +53
Understanding and modeling human intelligence are parallel goals shared by artificial intelligence (AI) and cognitive science. As AI systems grow increasingly capable, in what ways…
auto-psych: Automating the science of mind using agent-driven theory discovery and experimentation
Ben Prystawski, Kushin Mukherjee, Daniel Wurgaft +4
AI-based scientific automation is increasingly possible by using agents to generate hypotheses, design experiments, and analyze data. Data collection is a major bottleneck in this…
Is Child-Directed Speech Effective Training Data for Language Models?
Steven Y. Feng, Noah D. Goodman, Michael C. Frank
While high-performing language models are typically trained on hundreds of billions of words, human children become fluent language users with a much smaller amount of data. What a…