SA-Net: Deep Neural Network for Robot Trajectory Recognition from RGB-D Streams

Abstract

Learning from demonstration (LfD) and imitation learning offer new paradigms for transferring task behavior to robots. A class of methods that enable such online learning require the robot to observe the task being performed and decompose the sensed streaming data into sequences of state-action pairs, which are then input to the methods. Thus, recognizing the state-action pairs correctly and quickly in sensed data is a crucial prerequisite for these methods. We present SA-Net a deep neural network architecture that recognizes state-action pairs from RGB-D data streams. SA-Net performed well in two diverse robotic applications of LfD -- one involving mobile ground robots and another involving a robotic manipulator -- which demonstrates that the architecture generalizes well to differing contexts. Comprehensive evaluations including deployment on a physical robot show that \sanet{} significantly improves on the accuracy of the previous method that utilizes traditional image processing and segmentation.

arXiv comment(in press)

References in corpus (6)

• Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
• YOLOv3: An Incremental Improvement
• Maximum Entropy Deep Inverse Reinforcement Learning
• Temporal Activity Detection in Untrimmed Videos with Recurrent Neural Networks
• Freeway Merging in Congested Traffic based on Multipolicy Decision Making with Passive Actor Critic
• Action Recognition: From Static Datasets to Moving Robots

Cited by in corpus (4)

• PathoNet: Deep learning assisted evaluation of Ki-67 and tumor infiltrating lymphocytes (TILs) as prognostic factors in breast cancer; A large dataset and baseline
• AZTR: Aerial Video Action Recognition with Auto Zoom and Temporal Reasoning
• CrowdTeacher: Robust Co-teaching with Noisy Answers & Sample-specific Perturbations for Tabular Data
• FireCommander: An Interactive, Probabilistic Multi-agent Environment for Heterogeneous Robot Teams