75 citations · 326 across the 20 of their papers we have counts for
21 papers
Handling Missing Responses under Cluster Dependence with Applications to Language Model Evaluation
Zhenghao Zeng, David Arbour, Avi Feller +3
Human annotations play a crucial role in evaluating the performance of GenAI models. Two common challenges in practice, however, are missing annotations (the response variable of i…
How do Large Language Models Navigate Conflicts between Honesty and Helpfulness?
Ryan Liu, Theodore R. Sumers, Ishita Dasgupta +1
In day-to-day communication, people often approximate the truth - for example, rounding the time or omitting details - in order to be maximally helpful to the listener. How do larg…
Hierarchical reinforcement learning with natural language subgoals
Arun Ahuja, Kavya Kopparapu, Rob Fergus +1
Hierarchical reinforcement learning has been a compelling approach for achieving goal directed behavior over long sequences of actions. However, it has been challenging to implemen…
Passive learning of active causal strategies in agents and language models
Andrew Kyle Lampinen, Stephanie C Y Chan, Ishita Dasgupta +2
What can be learned about causality and experimentation from passive data? This question is salient given recent successes of passively-trained language models in interactive domai…
Machine Psychology
Thilo Hagendorff, Ishita Dasgupta, Marcel Binz +5
Large language models (LLMs) show increasingly advanced emergent capabilities and are being incorporated across various societal domains. Understanding their behavior and reasoning…
Collaborating with language models for embodied reasoning
Ishita Dasgupta, Christine Kaeser-Chen, Kenneth Marino +4
Reasoning in a complex and ambiguous environment is a key goal for Reinforcement Learning (RL) agents. While some sophisticated RL agents can successfully solve difficult tasks, th…
Distilling Internet-Scale Vision-Language Models into Embodied Agents
Theodore Sumers, Kenneth Marino, Arun Ahuja +2
Instruction-following agents must ground language into their observation and action spaces. Learning to ground language is challenging, typically requiring domain-specific engineer…
Learning to Navigate Wikipedia by Taking Random Walks
Manzil Zaheer, Kenneth Marino, Will Grathwohl +7
A fundamental ability of an intelligent web-based agent is seeking out and acquiring new information. Internet search engines reliably find the correct vicinity but the top results…
Transformers generalize differently from information stored in context vs in weights
Stephanie C. Y. Chan, Ishita Dasgupta, Junkyung Kim +3
Transformer models can use two fundamentally different kinds of information: information stored in weights during training, and information provided ``in-context'' at inference tim…
Language models show human-like content effects on reasoning tasks
Ishita Dasgupta, Andrew K. Lampinen, Stephanie C. Y. Chan +5
Reasoning is a key ability for an intelligent system. Large language models (LMs) achieve above-chance performance on abstract reasoning tasks, but exhibit many imperfections. Howe…
Using Natural Language and Program Abstractions to Instill Human Inductive Biases in Machines
Sreejan Kumar, Carlos G. Correa, Ishita Dasgupta +7
Strong inductive biases give humans the ability to quickly learn to perform a variety of tasks. Although meta-learning is a method to endow neural networks with useful inductive bi…
Can language models learn from explanations in context?
Andrew K. Lampinen, Ishita Dasgupta, Stephanie C. Y. Chan +6
Language Models (LMs) can perform new tasks by adapting to a few in-context examples. For humans, explanations that connect examples to task principles can improve learning. We the…
Disentangling Abstraction from Statistical Pattern Matching in Human and Machine Learning
Sreejan Kumar, Ishita Dasgupta, Nathaniel D. Daw +2
The ability to acquire abstract knowledge is a hallmark of human intelligence and is believed by many to be one of the core differences between humans and neural network models. Ag…
Tell me why! Explanations support learning relational and causal structure
Andrew K. Lampinen, Nicholas A. Roy, Ishita Dasgupta +8
Inferring the abstract relational and causal structure of the world is a major challenge for reinforcement-learning (RL) agents. For humans, language--particularly in the form of e…
Distinguishing rule- and exemplar-based generalization in learning systems
Ishita Dasgupta, Erin Grant, Thomas L. Griffiths
Machine learning systems often do not share the same inductive biases as humans and, as a result, extrapolate or generalize in ways that are inconsistent with our expectations. The…
Passive Attention in Artificial Neural Networks Predicts Human Visual Selectivity
Thomas A. Langlois, H. Charles Zhao, Erin Grant +3
Developments in machine learning interpretability techniques over the past decade have provided new tools to observe the image regions that are most informative for classification…
Are Convolutional Neural Networks or Transformers more like human vision?
Shikhar Tuli, Ishita Dasgupta, Erin Grant +1
Modern machine learning models for computer vision exceed humans in accuracy on specific visual recognition tasks, notably on datasets like ImageNet. However, high accuracy can be…
Meta-Learning of Structured Task Distributions in Humans and Machines
Sreejan Kumar, Ishita Dasgupta, Jonathan D. Cohen +2
In recent years, meta-learning, in which a model is trained on a family of tasks (i.e. a task distribution), has emerged as an approach to training neural networks to perform tasks…
Analyzing machine-learned representations: A natural language case study
Ishita Dasgupta, Demi Guo, Samuel J. Gershman +1
As modern deep networks become more complex, and get closer to human-like capabilities in certain domains, the question arises of how the representations and decision rules they le…
Causal Reasoning from Meta-reinforcement Learning
Ishita Dasgupta, Jane Wang, Silvia Chiappa +7
Discovering and exploiting the causal structure in the environment is a crucial challenge for intelligent agents. Here we explore whether causal reasoning can emerge via meta-reinf…
Evaluating Compositionality in Sentence Embeddings
Ishita Dasgupta, Demi Guo, Andreas Stuhlmüller +2
An important challenge for human-like AI is compositional semantics. Recent research has attempted to address this by using deep neural networks to learn vector space embeddings of…