OpenLORIS-Object: A Robotic Vision Dataset and Benchmark for Lifelong Deep Learning

Abstract

The recent breakthroughs in computer vision have benefited from the availability of large representative datasets (e.g. ImageNet and COCO) for training. Yet, robotic vision poses unique challenges for applying visual algorithms developed from these standard computer vision datasets due to their implicit assumption over non-varying distributions for a fixed set of tasks. Fully retraining models each time a new task becomes available is infeasible due to computational, storage and sometimes privacy issues, while naïve incremental strategies have been shown to suffer from catastrophic forgetting. It is crucial for the robots to operate continuously under open-set and detrimental conditions with adaptive visual perceptual systems, where lifelong learning is a fundamental capability. However, very few datasets and benchmarks are available to evaluate and compare emerging techniques. To fill this gap, we provide a new lifelong robotic vision dataset ("OpenLORIS-Object") collected via RGB-D cameras. The dataset embeds the challenges faced by a robot in the real-life application and provides new benchmarks for validating lifelong object recognition algorithms. Moreover, we have provided a testbed of $9$ state-of-the-art lifelong learning algorithms. Each of them involves $48$ tasks with $4$ evaluation metrics over the OpenLORIS-Object dataset. The results demonstrate that the object recognition task in the ever-changing difficulty environments is far from being solved and the bottlenecks are at the forward/backward transfer designs. Our dataset and benchmark are publicly available at at \href{https://lifelong-robotic-vision.github.io/dataset/object}{\underline{https://lifelong-robotic-vision.github.io/dataset/object}}.

arXiv comment7 pages, 7 figures, 4 tables

References in corpus (15)

• Overcoming catastrophic forgetting in neural networks
• Continual Learning Through Synaptic Intelligence
• An Empirical Investigation of Catastrophic Forgetting in Gradient-Based Neural Networks
• Progress & Compress: A scalable framework for continual learning
• Alleviating catastrophic forgetting using context-dependent gating and synaptic stabilization
• Generative Adversarial Networks in Computer Vision: A Survey and Taxonomy
• Generative replay with feedback connections as a general strategy for continual learning
• Lifelong Learning with Dynamically Expandable Networks
• Deep Generative Dual Memory Network for Continual Learning
• Don't forget, there is more than forgetting: new metrics for Continual Learning
• Incremental Classifier Learning with Generative Adversarial Networks
• A Strategy for an Uncompromising Incremental Learner
• Are We Ready for Service Robots? The OpenLORIS-Scene Datasets for Lifelong SLAM
• Neural Dynamics Discovery via Gaussian Process Recurrent Neural Networks
• Synthetic-Neuroscore: Using A Neuro-AI Interface for Evaluating Generative Adversarial Networks

Cited by in corpus (3)

• Are We Ready for Service Robots? The OpenLORIS-Scene Datasets for Lifelong SLAM
• CatNet: Class Incremental 3D ConvNets for Lifelong Egocentric Gesture Recognition
• A Neuro-AI Interface for Evaluating Generative Adversarial Networks