35 citations · 68 across the 26 of their papers we have counts for
29 papers
Medical Model Synthesis Architectures: A Case Study
Katherine M. Collins, Marlene Berke, Ilia Sucholutsky +6
Medicine is rife with high-stakes uncertainty. Doctors routinely make clinical judgments and decisions that juggle many fundamental unknowns, like predictions about what might be c…
Improving the Efficiency of Language Agent Teams with Adaptive Task Graphs
Elizabeth Mieczkowski, Alexander Ku, Tiwalayo Eisape +5
Large language models (LLMs) are increasingly deployed in teams, yet existing coordination approaches often occupy two extremes. Highly structured methods rely on fixed roles, pipe…
Language Model Teams as Distributed Systems
Elizabeth Mieczkowski, Katherine M. Collins, Ilia Sucholutsky +2
Large language models (LLMs) are growing increasingly capable, prompting recent interest in LLM teams. Yet, despite increased deployment of LLM teams at scale, we lack a principled…
Measuring and mitigating overreliance to build human-compatible AI
Lujain Ibrahim, Katherine M. Collins, Sunnie S. Y. Kim +14
Large language models (LLMs) distinguish themselves from previous technologies by functioning as collaborative ``thought partners,'' capable of engaging more fluidly in natural lan…
Documenting Deployment with Fabric: A Repository of Real-World AI Governance
Mackenzie Jorgensen, Kendall Brogle, Katherine M. Collins +10
Artificial intelligence (AI) is increasingly integrated into society, from financial services and traffic management to creative writing. Academic literature on the deployment of A…
Using LLMs to Advance the Cognitive Science of Collectives
Ilia Sucholutsky, Katherine M. Collins, Nori Jacoby +2
LLMs are already transforming the study of individual cognition, but their application to studying collective cognition has been underexplored. We lay out how LLMs may be able to a…
Identifying, Evaluating, and Mitigating Risks of AI Thought Partnerships
Kerem Oktar, Katherine M. Collins, Jose Hernandez-Orallo +4
Artificial Intelligence (AI) systems have historically been used as tools that execute narrowly defined tasks. Yet recent advances in AI have unlocked possibilities for a new class…
When Should We Orchestrate Multiple Agents?
Umang Bhatt, Sanyam Kapoor, Mihir Upadhyay +6
Strategies for orchestrating the interactions between multiple agents, both human and artificial, can wildly overestimate performance and underestimate the cost of orchestration. W…
On Benchmarking Human-Like Intelligence in Machines
Lance Ying, Katherine M. Collins, Lionel Wong +6
Recent advances in Artificial Intelligence (AI) have yielded powerful computational models that, by learning from vast amounts of human-generated data, are increasingly posited as…
Revisiting Rogers' Paradox in the Context of Human-AI Interaction
Katherine M. Collins, Umang Bhatt, Ilia Sucholutsky
Humans learn about the world, and how to act in the world, in many ways: from individually conducting experiments to observing and reproducing others' behavior. Different learning…
Data for Mathematical Copilots: Better Ways of Presenting Proofs for Machine Learning
Simon Frieder, Jonas Bayer, Sam Looi +13
The datasets and benchmarks commonly used to train and evaluate the mathematical capabilities of AI-based mathematical copilots (primarily large language models) exhibit several sh…
Can Large Language Models Understand Symbolic Graphics Programs?
Zeju Qiu, Weiyang Liu, Haiwen Feng +7
Against the backdrop of enthusiasm for large language models (LLMs), there is a growing need to scientifically assess their capabilities and shortcomings. This is nontrivial in par…
Building Machines that Learn and Think with People
Katherine M. Collins, Ilia Sucholutsky, Umang Bhatt +10
What do we want from machine intelligence? We envision machines that are not just tools for thought, but partners in thought: reasonable, insightful, knowledgeable, reliable, and t…
People use fast, goal-directed simulation to reason about novel games
Cedegao E. Zhang, Katherine M. Collins, Lionel Wong +3
People can evaluate features of problems and their potential solutions well before we can effectively solve them. When considering a game we have never played, for instance, we mig…
Modulating Language Model Experiences through Frictions
Katherine M. Collins, Valerie Chen, Ilia Sucholutsky +6
Language models are transforming the ways that their users engage with the world. Despite impressive capabilities, over-consumption of language model outputs risks propagating unch…
Large Language Models Must Be Taught to Know What They Don't Know
Sanyam Kapoor, Nate Gruver, Manley Roberts +7
When using large language models (LLMs) in high-stakes applications, we need to know when we can trust their predictions. Some works argue that prompting high-performance LLMs is s…
Representational Alignment Supports Effective Machine Teaching
Ilia Sucholutsky, Katherine M. Collins, Maya Malaviya +11
A good teacher should not only be knowledgeable, but should also be able to communicate in a way that the student understands -- to share the student's representation of the world.…
AI for Mathematics: A Cognitive Science Perspective
Cedegao E. Zhang, Katherine M. Collins, Adrian Weller +1
Mathematics is one of the most powerful conceptual systems developed and used by the human species. Dreams of automated mathematicians have a storied history in artificial intellig…
Getting aligned on representational alignment
Ilia Sucholutsky, Lukas Muttenthaler, Adrian Weller +30
Biological and artificial information processing systems form representations of the world that they can use to categorize, reason, plan, navigate, and make decisions. How can we m…
Learning to Receive Help: Intervention-Aware Concept Embedding Models
Mateo Espinosa Zarlenga, Katherine M. Collins, Krishnamurthy Dvijotham +3
Concept Bottleneck Models (CBMs) tackle the opacity of neural architectures by constructing and explaining their predictions using a set of high-level concepts. A special property…
FeedbackLogs: Recording and Incorporating Stakeholder Feedback into Machine Learning Pipelines
Matthew Barker, Emma Kallina, Dhananjay Ashok +6
Even though machine learning (ML) pipelines affect an increasing array of stakeholders, there is little work on how input from stakeholders is recorded and incorporated. We propose…
The Neuro-Symbolic Inverse Planning Engine (NIPE): Modeling Probabilistic Social Inferences from Linguistic Inputs
Lance Ying, Katherine M. Collins, Megan Wei +5
Human beings are social creatures. We routinely reason about other agents, and a crucial component of this social reasoning is inferring people's goals as we learn about their acti…
Selective Concept Models: Permitting Stakeholder Customisation at Test-Time
Matthew Barker, Katherine M. Collins, Krishnamurthy Dvijotham +2
Concept-based models perform prediction using a set of concepts that are interpretable to stakeholders. However, such models often involve a fixed, large number of concepts, which…
Evaluating Language Models for Mathematics through Interactions
Katherine M. Collins, Albert Q. Jiang, Simon Frieder +11
There is much excitement about the opportunity to harness the power of large language models (LLMs) when building problem-solving assistants. However, the standard methodology of e…