822 citations · 2.9k across the 43 of their papers we have counts for
47 papers
EasyPPO: Stabilizing the Critic Is Key
Xuanyi Zhou, Qiuyang Mang, Huanzhi Mao +7
A key strength of Proximal Policy Optimization (PPO) is its learned critic, which uses historical trajectories collected during reinforcement learning to estimate expected returns…
How does Adversarial Influence Scale in Multi-Agent Systems?
Addison J. Wu, Jasin Cekinmez, Michel Liao +2
Multi-agent deliberation can improve performance, but what happens when some agents do not act in good faith? In practice, an agent may be deceptive and work to subvert the group,…
LeAct: Learning to Reason from Expert Actions
Ziran Yang, Chengshuai Shi, Raj Ghugare +3
Modern reasoning models depend on reasoning data, today sourced from human annotations or distilled from stronger LLMs. However, a rich and largely untapped source of supervision l…
Cognitive Architectures for Language Agents
Theodore R. Sumers, Shunyu Yao, Karthik Narasimhan +1
Recent efforts have augmented large language models (LLMs) with external resources (e.g., the Internet) or internal control flows (e.g., prompt chaining) for tasks requiring ground…
Scaling Laws for Imitation Learning in Single-Agent Games
Jens Tuyls, Dhruv Madeka, Kari Torkkola +3
Imitation Learning (IL) is one of the most widely used methods in machine learning. Yet, many works find it is often unable to fully recover the underlying expert behavior, even in…
Tree of Thoughts: Deliberate Problem Solving with Large Language Models
Shunyu Yao, Dian Yu, Jeffrey Zhao +4
Language models are increasingly being deployed for general problem solving across a wide range of tasks, but are still confined to token-level, left-to-right decision-making proce…
Reflexion: Language Agents with Verbal Reinforcement Learning
Noah Shinn, Federico Cassano, Edward Berman +3
Large language models (LLMs) have been increasingly used to interact with external environments (e.g., games, compilers, APIs) as goal-driven agents. However, it remains challengin…
ReAct: Synergizing Reasoning and Acting in Language Models
Shunyu Yao, Jeffrey Zhao, Dian Yu +4
While large language models (LLMs) have demonstrated impressive capabilities across tasks in language understanding and interactive decision making, their abilities for reasoning (…
WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agents
Shunyu Yao, Howard Chen, John Yang +1
Existing benchmarks for grounding language in interactive environments either lack real-world linguistic elements, or prove difficult to scale up due to substantial human involveme…
Leveraging Language for Accelerated Learning of Tool Manipulation
Allen Z. Ren, Bharat Govil, Tsung-Yen Yang +2
Robust and generalized tool manipulation requires an understanding of the properties and affordances of different tools. We investigate whether linguistic information about a tool…
Can Rationalization Improve Robustness?
Howard Chen, Jacqueline He, Karthik Narasimhan +1
A growing line of work has investigated the development of neural NLP models that can produce rationales--subsets of input that can explain their model predictions. In this paper,…
Using Natural Language and Program Abstractions to Instill Human Inductive Biases in Machines
Sreejan Kumar, Carlos G. Correa, Ishita Dasgupta +7
Strong inductive biases give humans the ability to quickly learn to perform a variety of tasks. Although meta-learning is a method to endow neural networks with useful inductive bi…
CARETS: A Consistency And Robustness Evaluative Test Suite for VQA
Carlos E. Jimenez, Olga Russakovsky, Karthik Narasimhan
We introduce CARETS, a systematic test suite to measure consistency and robustness of modern VQA models through a series of six fine-grained capability tests. In contrast to existi…
DataMUX: Data Multiplexing for Neural Networks
Vishvak Murahari, Carlos E. Jimenez, Runzhe Yang +1
In this paper, we introduce data multiplexing (DataMUX), a technique that enables deep neural networks to process multiple inputs simultaneously using a single compact representati…
Multi-Query Video Retrieval
Zeyu Wang, Yu Wu, Karthik Narasimhan +1
Retrieving target videos based on text descriptions is a task of great practical value and has received increasing attention over the past few years. Despite recent progress, imper…
Multi-Stage Episodic Control for Strategic Exploration in Text Games
Jens Tuyls, Shunyu Yao, Sham Kakade +1
Text adventure games present unique challenges to reinforcement learning methods due to their combinatorially large action spaces and sparse rewards. The interplay of these two fac…
SILG: The Multi-environment Symbolic Interactive Language Grounding Benchmark
Victor Zhong, Austin W. Hanjie, Sida I. Wang +2
Existing work in language grounding typically study single environments. How do we build unified models that apply across multiple environments? We propose the multi-environment Sy…
Revelio: ML-Generated Debugging Queries for Distributed Systems
Pradeep Dogga, Karthik Narasimhan, Anirudh Sivaraman +3
A major difficulty in debugging distributed systems lies in manually determining which of the many available debugging tools to use and how to query its logs. Our own study of a pr…
Self-Attention Networks Can Process Bounded Hierarchical Languages
Shunyu Yao, Binghui Peng, Christos Papadimitriou +1
Despite their impressive performance in NLP, self-attention networks were recently proved to be limited for processing formal languages with hierarchical structure, such as $\maths…
Reading and Acting while Blindfolded: The Need for Semantics in Text Game Agents
Shunyu Yao, Karthik Narasimhan, Matthew Hausknecht
Text-based games simulate worlds and interact with players using natural language. Recent work has used them as a testbed for autonomous language-understanding agents, with the mot…
Grounding Language to Entities and Dynamics for Generalization in Reinforcement Learning
Austin W. Hanjie, Victor Zhong, Karthik Narasimhan
We investigate the use of natural language to drive the generalization of control policies and introduce the new multi-task environment Messenger with free-form text manuals descri…
Connecting Context-specific Adaptation in Humans to Meta-learning
Rachit Dubey, Erin Grant, Michael Luo +2
Cognitive control, the ability of a system to adapt to the demands of a task, is an integral part of cognition. A widely accepted fact about cognitive control is that it is context…
Projection-Based Constrained Policy Optimization
Tsung-Yen Yang, Justinian Rosca, Karthik Narasimhan +1
We consider the problem of learning control policies that optimize a reward function while satisfying constraints due to considerations of safety, fairness, or other costs. We prop…
Keep CALM and Explore: Language Models for Action Generation in Text-based Games
Shunyu Yao, Rohan Rao, Matthew Hausknecht +1
Text-based games present a unique challenge for autonomous agents to operate in natural language and handle enormous action spaces. In this paper, we propose the Contextual Action…