iGibson 1.0: a Simulation Environment for Interactive Tasks in Large Realistic Scenes

Abstract

We present iGibson 1.0, a novel simulation environment to develop robotic solutions for interactive tasks in large-scale realistic scenes. Our environment contains 15 fully interactive home-sized scenes with 108 rooms populated with rigid and articulated objects. The scenes are replicas of real-world homes, with distribution and the layout of objects aligned to those of the real world. iGibson 1.0 integrates several key features to facilitate the study of interactive tasks: i) generation of high-quality virtual sensor signals (RGB, depth, segmentation, LiDAR, flow and so on), ii) domain randomization to change the materials of the objects (both visual and physical) and/or their shapes, iii) integrated sampling-based motion planners to generate collision-free trajectories for robot bases and arms, and iv) intuitive human-iGibson interface that enables efficient collection of human demonstrations. Through experiments, we show that the full interactivity of the scenes enables agents to learn useful visual representations that accelerate the training of downstream manipulation tasks. We also show that iGibson 1.0 features enable the generalization of navigation agents, and that the human-iGibson interface and integrated motion planners facilitate efficient imitation learning of human demonstrated (mobile) manipulation behaviors. iGibson 1.0 is open-source, equipped with comprehensive examples and documentation. For more information, visit our project website: http://svl.stanford.edu/igibson/

References in corpus (18)

• Continuous control with deep reinforcement learning
• Soft Actor-Critic Algorithms and Applications
• On Evaluation of Embodied Navigation Agents
• Learning to Navigate in Complex Environments
• Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learning
• Matterport3D: Learning from RGB-D Data in Indoor Environments
• AI2-THOR: An Interactive 3D Environment for Visual AI
• Intelligent bidirectional rapidly-exploring random trees for optimal motion planning in complex cluttered environments
• robosuite: A Modular Simulation Framework and Benchmark for Robot Learning
• FFRob: Leveraging Symbolic Planning for Efficient Task and Motion Planning
• ThreeDWorld: A Platform for Interactive Multi-Modal Physical Simulation
• Micro-Batch Training with Batch-Channel Normalization and Weight Standardization
• The Best of Both Modes: Separately Leveraging RGB and Depth for Unseen Object Instance Segmentation
• ReLMoGen: Leveraging Motion Generation in Reinforcement Learning for Mobile Manipulation
• DoorGym: A Scalable Door Opening Environment And Baseline Agent
• Situational Fusion of Visual Representation for Visual Navigation
• 3D-FRONT: 3D Furnished Rooms with layOuts and semaNTics
• How to Close Sim-Real Gap? Transfer with Segmentation!

Cited by in corpus (20)

• A Survey of Zero-shot Generalisation in Deep Reinforcement Learning
• Unity Perception: Generate Synthetic Data for Computer Vision
• iGibson 2.0: Object-Centric Simulation for Robot Learning of Everyday Household Tasks
• Habitat-Matterport 3D Dataset (HM3D): 1000 Large-scale 3D Environments for Embodied AI
• From Machine Learning to Robotics: Challenges and Opportunities for Embodied Intelligence
• A Survey of Embodied AI: From Simulators to Research Tasks
• Habitat 2.0: Training Home Assistants to Rearrange their Habitat
• Collaborative Visual Navigation
• D3D-HOI: Dynamic 3D Human-Object Interactions from Videos
• SECANT: Self-Expert Cloning for Zero-Shot Generalization of Visual Policies
• Deep Learning for Embodied Vision Navigation: A Survey
• OpenRooms: An End-to-End Open Framework for Photorealistic Indoor Scene Datasets
• TASKOGRAPHY: Evaluating robot task planning over large 3D scene graphs
• CORA: Benchmarks, Baselines, and Metrics as a Platform for Continual Reinforcement Learning Agents
• Co-GAIL: Learning Diverse Strategies for Human-Robot Collaboration
• LUMINOUS: Indoor Scene Generation for Embodied AI Challenges
• Shaping embodied agent behavior with activity-context priors from egocentric video
• Learning to Navigate Sidewalks in Outdoor Environments
• DeepPanoContext: Panoramic 3D Scene Understanding with Holistic Scene Context Graph and Relation-based Optimization
• Transfer Reinforcement Learning across Homotopy Classes