5 papers
(V)LMs generalize beyond surface co-occurrence: Evidence from cross-modal number agreement
Zach Studdiford, Kanishka Misra
Language models learn about grammatical number primarily from co-occurrence, and show frequency effects as a result---sometimes taken to indicate that they do not learn abstract ``…
Reasoning as Pattern Matching: Shared Mechanisms in Human and LLM Everyday Reasoning
Zach Studdiford, Gary Lupyan
When large language models (LLMs) fail to generalize or make content-sensitive errors in reasoning, it is often taken as evidence that LLMs are not truly reasoning, but rather perf…
Uncovering the Computational Ingredients of Human-Like Representations in LLMs
Zach Studdiford, Timothy T. Rogers, Kushin Mukherjee +1
The human ability to translate diverse perceptual and linguistic inputs into structured behavior has been thought to rest on learning robust representations of concepts. The rapid…
Evaluating Steering Techniques using Human Similarity Judgments
Zach Studdiford, Timothy T. Rogers, Siddharth Suresh +1
Current evaluations of Large Language Model (LLM) steering techniques focus on task-specific performance, overlooking how well steered representations align with human cognition. U…
Beyond Demographics: Aligning Role-playing LLM-based Agents Using Human Belief Networks
Yun-Shiuan Chuang, Krirk Nirunwiroj, Zach Studdiford +6
Creating human-like large language model (LLM) agents is crucial for faithful social simulation. Having LLMs role-play based on demographic information sometimes improves human lik…