MINOS: Multimodal Indoor Simulator for Navigation in Complex Environments
arXiv:1712.03931
Abstract
We present MINOS, a simulator designed to support the development of multisensory models for goal-directed navigation in complex indoor environments. The simulator leverages large datasets of complex 3D environments and supports flexible configuration of multimodal sensor suites. We use MINOS to benchmark deep-learning-based navigation methods, to analyze the influence of environmental complexity on navigation performance, and to carry out a controlled study of multimodality in sensorimotor learning. The experiments show that current deep reinforcement learning approaches fail in large realistic environments. The experiments also indicate that multimodality is beneficial in learning to navigate cluttered scenes. MINOS is released open-source to the research community at http://minosworld.org . A video that shows MINOS can be found at https://youtu.be/c0mL9K64q84
MINOS is a simulator designed to support research on end-to-end navigation
References in corpus (1)
Cited by in corpus (61)
- On Evaluation of Embodied Navigation Agents
- Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learning
- AI2-THOR: An Interactive 3D Environment for Visual AI
- Object Goal Navigation using Goal-Oriented Semantic Exploration
- Dark, Beyond Deep: A Paradigm Shift to Cognitive AI with Humanlike Common Sense
- ObjectNav Revisited: On Evaluation of Embodied Agents Navigating to Objects
- Rearrangement: A Challenge for Embodied AI
- Spatial Action Maps for Mobile Manipulation
- CHALET: Cornell House Agent Learning Environment
- A Survey of Deep Network Solutions for Learning Control in Robotics: From Reinforcement to Imitation
- Chasing Ghosts: Instruction Following as Bayesian State Tracking
- Semi-parametric Topological Memory for Navigation
- The StreetLearn Environment and Dataset
- Embodied Visual Navigation with Automatic Curriculum Learning in Real Environments
- Benchmarking Classic and Learned Navigation in Complex 3D Environments
- Reinforced Cross-Modal Matching and Self-Supervised Imitation Learning for Vision-Language Navigation
- VRKitchen: an Interactive 3D Virtual Environment for Task-oriented Learning
- MeetUp! A Corpus of Joint Activity Dialogues in a Visual Environment
- Habitat 2.0: Training Home Assistants to Rearrange their Habitat
- Guided Feature Transformation (GFT): A Neural Language Grounding Module for Embodied Agents
- Embodied Visual Recognition
- A Behavioral Approach to Visual Navigation with Graph Localization Networks
- Language Tasks and Language Games: On Methodology in Current Natural Language Processing Research
- Towards Learning a Generic Agent for Vision-and-Language Navigation via Pre-training
- VirtualHome: Simulating Household Activities via Programs
- Learning models for visual 3D localization with implicit mapping
- VR-Goggles for Robots: Real-to-sim Domain Adaptation for Visual Control
- To Learn or Not to Learn: Analyzing the Role of Learning for Navigation in Virtual Environments
- The Regretful Agent: Heuristic-Aided Navigation through Progress Estimation
- Embodied Question Answering in Photorealistic Environments with Point Cloud Perception
- REVERIE: Remote Embodied Visual Referring Expression in Real Indoor Environments
- Deep Learning for Embodied Vision Navigation: A Survey
- OpenRooms: An End-to-End Open Framework for Photorealistic Indoor Scene Datasets
- Multi-View Learning for Vision-and-Language Navigation
- Why Build an Assistant in Minecraft?
- Motion Perception in Reinforcement Learning with Dynamic Objects
- CORA: Benchmarks, Baselines, and Metrics as a Platform for Continual Reinforcement Learning Agents
- LayoutMP3D: Layout Annotation of Matterport3D
- Graph Attention Memory for Visual Navigation
- Arena: A General Evaluation Platform and Building Toolkit for Multi-Agent Intelligence
- Multi-Target Embodied Question Answering
- Learning to Move with Affordance Maps
- Sim-Real Joint Reinforcement Transfer for 3D Indoor Navigation
- Bayesian Relational Memory for Semantic Visual Navigation
- 360SD-Net: 360° Stereo Depth Estimation with Learnable Cost Volume
- Unsupervised Reinforcement Learning of Transferable Meta-Skills for Embodied Navigation
- SOON: Scenario Oriented Object Navigation with Graph-based Exploration
- SSCNav: Confidence-Aware Semantic Scene Completion for Visual Semantic Navigation
- GeoSim: Realistic Video Simulation via Geometry-Aware Composition for Self-Driving
- Predicting Performance of SLAM Algorithms
- ArraMon: A Joint Navigation-Assembly Instruction Interpretation Task in Dynamic Environments
- Large Batch Simulation for Deep Reinforcement Learning
- Landmark Policy Optimization for Object Navigation Task
- SIGVerse: A cloud-based VR platform for research on social and embodied human-robot interaction
- MaAST: Map Attention with Semantic Transformersfor Efficient Visual Navigation
- A Few Shot Adaptation of Visual Navigation Skills to New Observations using Meta-Learning
- Multimodal Aggregation Approach for Memory Vision-Voice Indoor Navigation with Meta-Learning
- The Surprising Effectiveness of Visual Odometry Techniques for Embodied PointGoal Navigation
- Language coverage and generalization in RNN-based continuous sentence embeddings for interacting agents
- Deliberate Exploration Supports Navigation in Unfamiliar Worlds
- Building Intelligent Autonomous Navigation Agents