A Survey of Embodied AI: From Simulators to Research Tasks
arXiv:2103.04918
Abstract
There has been an emerging paradigm shift from the era of "internet AI" to "embodied AI", where AI algorithms and agents no longer learn from datasets of images, videos or text curated primarily from the internet. Instead, they learn through interactions with their environments from an egocentric perception similar to humans. Consequently, there has been substantial growth in the demand for embodied AI simulators to support various embodied AI research tasks. This growing interest in embodied AI is beneficial to the greater pursuit of Artificial General Intelligence (AGI), but there has not been a contemporary and comprehensive survey of this field. This paper aims to provide an encyclopedic survey for the field of embodied AI, from its simulators to its research. By evaluating nine current embodied AI simulators with our proposed seven features, this paper aims to understand the simulators in their provision for use in embodied AI research and their limitations. Lastly, this paper surveys the three main research tasks in embodied AI -- visual exploration, visual navigation and embodied question answering (QA), covering the state-of-the-art approaches, evaluation metrics and datasets. Finally, with the new insights revealed through surveying the field, the paper will provide suggestions for simulator-for-task selections and recommendations for the future directions of the field.
arXiv commentThis work has been accepted by IEEE Transactions on Emerging Topics in Computational Intelligence
References in corpus (37)
- Playing Atari with Deep Reinforcement Learning
- Language Models are Few-Shot Learners
- High-Dimensional Continuous Control Using Generalized Advantage Estimation
- VIME: Variational Information Maximizing Exploration
- On Evaluation of Embodied Navigation Agents
- The Replica Dataset: A Digital Replica of Indoor Spaces
- Large-Scale Study of Curiosity-Driven Learning
- Matterport3D: Learning from RGB-D Data in Indoor Environments
- AI2-THOR: An Interactive 3D Environment for Visual AI
- Exploration by Random Network Distillation
- Learning to Explore using Active Neural SLAM
- Object Goal Navigation using Goal-Oriented Semantic Exploration
- Sim2Real Predictivity: Does Evaluation in Simulation Predict Real-World Performance?
- Episodic Curiosity through Reachability
- robosuite: A Modular Simulation Framework and Benchmark for Robot Learning
- Dark, Beyond Deep: A Paradigm Shift to Cognitive AI with Humanlike Common Sense
- ObjectNav Revisited: On Evaluation of Embodied Agents Navigating to Objects
- ThreeDWorld: A Platform for Interactive Multi-Modal Physical Simulation
- CHALET: Cornell House Agent Learning Environment
- Unifying Map and Landmark Based Representations for Visual Navigation
- AllenAct: A Framework for Embodied AI Research
- iGibson 1.0: a Simulation Environment for Interactive Tasks in Large Realistic Scenes
- Neural Predictive Belief Representations
- Benchmarking Classic and Learned Navigation in Complex 3D Environments
- PlaTe: Visually-Grounded Planning with Transformers in Procedural Tasks
- Learning Exploration Policies for Navigation
- Habitat-Matterport 3D Dataset (HM3D): 1000 Large-scale 3D Environments for Embodied AI
- Visual Semantic Navigation using Scene Priors
- DD-PPO: Learning Near-Perfect PointGoal Navigators from 2.5 Billion Frames
- VRKitchen: an Interactive 3D Virtual Environment for Task-oriented Learning
- From Seeing to Moving: A Survey on Learning for Visual Indoor Navigation (VIN)
- Simultaneous Mapping and Target Driven Navigation
- Learning to Visually Navigate in Photorealistic Environments Without any Supervision
- Where to Look Next: Unsupervised Active Visual Exploration on 360° Input
- Auxiliary Tasks Speed Up Learning PointGoal Navigation
- Learning Object-conditioned Exploration using Distributed Soft Actor Critic
- SPACE: A Simulator for Physical Interactions and Causal Learning in 3D Environments
Cited by in corpus (4)
- A Benchmark for Modeling Violation-of-Expectation in Physical Reasoning Across Event Categories
- SPACE: A Simulator for Physical Interactions and Causal Learning in 3D Environments
- AVoE: A Synthetic 3D Dataset on Understanding Violation of Expectation for Artificial Cognition
- Messing Up 3D Virtual Environments: Transferable Adversarial 3D Objects