Gibson Env: Real-World Perception for Embodied Agents
arXiv:1808.10654
Abstract
Developing visual perception models for active agents and sensorimotor control are cumbersome to be done in the physical world, as existing algorithms are too slow to efficiently learn in real-time and robots are fragile and costly. This has given rise to learning-in-simulation which consequently casts a question on whether the results transfer to real-world. In this paper, we are concerned with the problem of developing real-world perception for active agents, propose Gibson Virtual Environment for this purpose, and showcase sample perceptual tasks learned therein. Gibson is based on virtualizing real spaces, rather than using artificially designed ones, and currently includes over 1400 floor spaces from 572 full buildings. The main characteristics of Gibson are: I. being from the real-world and reflecting its semantic complexity, II. having an internal synthesis mechanism, "Goggles", enabling deploying the trained models in real-world without needing further domain adaptation, III. embodiment of agents and making them subject to constraints of physics and space.
Access the code, dataset, and project website at http://gibsonenv.vision/ . CVPR 2018
Cited by in corpus (12)
- Vision-and-Language Navigation: A Survey of Tasks, Methods, and Future Directions
- Spatial Action Maps for Mobile Manipulation
- Embodied Visual Navigation with Automatic Curriculum Learning in Real Environments
- Learning Robotic Navigation from Experience: Principles, Methods, and Recent Results
- Gym-Ignition: Reproducible Robotic Simulations for Reinforcement Learning
- Learn to Predict How Humans Manipulate Large-sized Objects from Interactive Motions
- Focus on Impact: Indoor Exploration with Intrinsic Motivation
- Enhancing exploration algorithms for navigation with visual SLAM
- RF-Annotate: Automatic RF-Supervised Image Annotation of Common Objects in Context
- Self-Supervised Place Recognition by Refining Temporal and Featural Pseudo Labels from Panoramic Data
- Toddler-Guidance Learning: Impacts of Critical Period on Multimodal AI Agents
- PACE: Data-Driven Virtual Agent Interaction in Dense and Cluttered Environments