Autonomous Agents Modelling Other Agents: A Comprehensive Survey and Open Problems
arXiv:1709.08071 · doi:10.1016/j.artint.2018.01.002
Abstract
Much research in artificial intelligence is concerned with the development of autonomous agents that can interact effectively with other agents. An important aspect of such agents is the ability to reason about the behaviours of other agents, by constructing models which make predictions about various properties of interest (such as actions, goals, beliefs) of the modelled agents. A variety of modelling approaches now exist which vary widely in their methodology and underlying assumptions, catering to the needs of the different sub-communities within which they were developed and reflecting the different practical uses for which they are intended. The purpose of the present article is to provide a comprehensive survey of the salient modelling methods which can be found in the literature. The article concludes with a discussion of open problems which may form the basis for fruitful future research.
Final manuscript (46 pages), published in Artificial Intelligence Journal. The arXiv version also contains a table of contents after the abstract, but is otherwise identical to the AIJ version. Keywords: autonomous agents, multiagent systems, modelling other agents, opponent modelling
References in corpus (11)
- CP-nets: A Tool for Representing and Reasoning withConditional Ceteris Paribus Preference Statements
- Tractable Inference for Complex Stochastic Processes
- Bayes' Bluff: Opponent Modelling in Poker
- Probabilistic State-Dependent Grammars for Plan Recognition
- Policy Recognition in the Abstract Hidden Markov Model
- Opponent Modeling in Deep Reinforcement Learning
- Monitoring Teams by Overhearing: A Multi-Agent Plan-Recognition Approach
- Interactive POMDP Lite: Towards Practical Planning to Predict and Exploit Intentions for Interacting with Self-Interested Agents
- On Convergence and Optimality of Best-Response Learning with Policy Types in Multiagent Systems
- E-HBA: Using Action Policies for Expert Advice and Agent Typification
- Are You Doing What I Think You Are Doing? Criticising Uncertain Agent Models
Cited by in corpus (72)
- A Survey and Critique of Multiagent Deep Reinforcement Learning
- The Hanabi Challenge: A New Frontier for AI Research
- A Survey of Zero-shot Generalisation in Deep Reinforcement Learning
- Multi-Objective Multi-Agent Decision Making: A Utility-based Analysis and Survey
- Dealing with Non-Stationarity in Multi-Agent Deep Reinforcement Learning
- Machine Theory of Mind
- Stabilizing Generative Adversarial Networks: A Survey
- Towards Next-Generation Urban Decision Support Systems through AI-Powered Construction of Scientific Ontology using Large Language Models -- A Case in Optimizing Intermodal Freight Transportation
- Watch-And-Help: A Challenge for Social Perception and Human-AI Collaboration
- Plan-Then-Execute: An Empirical Study of User Trust and Team Performance When Using LLM Agents As A Daily Assistant
- FABRIC: A Framework for the Design and Evaluation of Collaborative Robots with Extended Human Adaptation
- Collaborative Visual Navigation
- Finding Friend and Foe in Multi-Agent Games
- Learning to Incentivize Other Learning Agents
- Open Problems in Cooperative AI
- Scaling Multi-Agent Reinforcement Learning with Selective Parameter Sharing
- Baby Intuitions Benchmark (BIB): Discerning the goals, preferences, and actions of others
- Learning Multi-Agent Coordination through Connectivity-driven Communication
- Exploring the Role of Common Model of Cognition in Designing Adaptive Coaching Interactions for Health Behavior Change
- Intelligent Software Web Agents: A Gap Analysis
- Model-Based Opponent Modeling
- Modeling Theory of Mind in Multi-Agent Games Using Adaptive Feedback Control
- Inverse Reinforcement Learning via Nonparametric Spatio-Temporal Subgoal Modeling
- Robust Opponent Modeling via Adversarial Ensemble Reinforcement Learning in Asymmetric Imperfect-Information Games
- Neural Recursive Belief States in Multi-Agent Reinforcement Learning
- Opponent Aware Reinforcement Learning
- Manipulating hidden-Markov-model inferences by corrupting batch data
- Learning to Communicate Implicitly By Actions
- Needs-driven Heterogeneous Multi-Robot Cooperation in Rescue Missions
- Incorporating Pragmatic Reasoning Communication into Emergent Language
- Variational Autoencoders for Opponent Modeling in Multi-Agent Systems
- PsiPhi-Learning: Reinforcement Learning with Demonstrations using Successor Features and Inverse Temporal Difference Learning
- How Bad is Selfish Driving? Bounding the Inefficiency of Equilibria in Urban Driving Games
- Towards Collaborative Plan Acquisition through Theory of Mind Modeling in Situated Dialogue
- Towards Open Ad Hoc Teamwork Using Graph-based Policy Learning
- Non-cooperative Multi-agent Systems with Exploring Agents
- Stochastic Reinforcement Learning
- Agent Modelling under Partial Observability for Deep Reinforcement Learning
- Towards Efficient Detection and Optimal Response against Sophisticated Opponents
- Combining Theory of Mind and Abduction for Cooperation under Imperfect Information
- Robust Stochastic Bayesian Games for Behavior Space Coverage
- Multi-Agent Interactions Modeling with Correlated Policies
- On the Importance of Opponent Modeling in Auction Markets
- Acceptable Planning: Influencing Individual Behavior to Reduce Transportation Energy Expenditure of a City
- Learning Complex Multi-Agent Policies in Presence of an Adversary
- Task-Agnostic Online Reinforcement Learning with an Infinite Mixture of Gaussian Processes
- (Ir)rationality in AI: State of the Art, Research Challenges and Open Questions
- Interactive AI with a Theory of Mind
- Dynamic Relational Inference in Multi-Agent Trajectories
- Interactive Agent Modeling by Learning to Probe
- Scalable Safety-Critical Policy Evaluation with Accelerated Rare Event Sampling
- Multi-Turn Beam Search for Neural Dialogue Modeling
- AGENT: A Benchmark for Core Psychological Reasoning
- Theory of Mind with Guilt Aversion Facilitates Cooperative Reinforcement Learning
- Adaptive Agent Architecture for Real-time Human-Agent Teaming
- Learning to Classify and Imitate Trading Agents in Continuous Double Auction Markets
- Behaviour-conditioned policies for cooperative reinforcement learning tasks
- Contributions to Large Scale Bayesian Inference and Adversarial Machine Learning
- Multi-Agent Imitation Learning with Copulas
- L2E: Learning to Exploit Your Opponent
- PLAN-B: Predicting Likely Alternative Next Best Sequences for Action Prediction
- Searching with Opponent-Awareness
- Visual Perspective Taking for Opponent Behavior Modeling
- Link Prediction for Temporally Consistent Networks
- Antarjami: Exploring psychometric evaluation through a computer-based game
- Interaction-Aware Multi-Agent Reinforcement Learning for Mobile Agents with Individual Goals
- Simultaneous Learning from Human Pose and Object Cues for Real-Time Activity Recognition
- Autonomous Industrial Management via Reinforcement Learning: Self-Learning Agents for Decision-Making -- A Review
- Agent Probing Interaction Policies
- Implementing Argumentation-enabled Empathic Agents
- Empathic Autonomous Agents
- Directionality Reinforcement Learning to Operate Multi-Agent System without Communication