134 citations · 242 across the 39 of their papers we have counts for
8 papers · 1 filter
Measuring and mitigating overreliance to build human-compatible AI
Lujain Ibrahim, Katherine M. Collins, Sunnie S. Y. Kim +14
Large language models (LLMs) distinguish themselves from previous technologies by functioning as collaborative ``thought partners,'' capable of engaging more fluidly in natural lan…
Identifying, Evaluating, and Mitigating Risks of AI Thought Partnerships
Kerem Oktar, Katherine M. Collins, Jose Hernandez-Orallo +4
Artificial Intelligence (AI) systems have historically been used as tools that execute narrowly defined tasks. Yet recent advances in AI have unlocked possibilities for a new class…
When Should We Orchestrate Multiple Agents?
Umang Bhatt, Sanyam Kapoor, Mihir Upadhyay +6
Strategies for orchestrating the interactions between multiple agents, both human and artificial, can wildly overestimate performance and underestimate the cost of orchestration. W…
Levels of Analysis for Large Language Models
Alexander Y. Ku, Declan Campbell, Xuechunzi Bai +10
Modern artificial intelligence systems, such as large language models, are increasingly powerful but also increasingly hard to understand. Recognizing this problem as analogous to…
What is a Number, That a Large Language Model May Know It?
Raja Marjieh, Veniamin Veselovsky, Thomas L. Griffiths +1
Numbers are a basic part of how humans represent and describe the world around them. As a consequence, learning effective representations of numbers is critical for the success of…
On Benchmarking Human-Like Intelligence in Machines
Lance Ying, Katherine M. Collins, Lionel Wong +6
Recent advances in Artificial Intelligence (AI) have yielded powerful computational models that, by learning from vast amounts of human-generated data, are increasingly posited as…
Revisiting Rogers' Paradox in the Context of Human-AI Interaction
Katherine M. Collins, Umang Bhatt, Ilia Sucholutsky
Humans learn about the world, and how to act in the world, in many ways: from individually conducting experiments to observing and reproducing others' behavior. Different learning…
Humanity's Last Exam
Long Phan, Alice Gatti, Ziwen Han +1168
Benchmarks are important tools for tracking the rapid advancements in large language model (LLM) capabilities. However, benchmarks are not keeping pace in difficulty: LLMs now achi…