Visual Affordance and Function Understanding: A Survey
arXiv:1807.06775
Abstract
Nowadays, robots are dominating the manufacturing, entertainment and healthcare industries. Robot vision aims to equip robots with the ability to discover information, understand it and interact with the environment. These capabilities require an agent to effectively understand object affordances and functionalities in complex visual domains. In this literature survey, we first focus on Visual affordances and summarize the state of the art as well as open problems and research gaps. Specifically, we discuss sub-problems such as affordance detection, categorization, segmentation and high-level reasoning. Furthermore, we cover functional scene understanding and the prevalent functional descriptors used in the literature. The survey also provides necessary background to the problem, sheds light on its significance and highlights the existing challenges for affordance and functionality learning.
arXiv comment26 pages, 22 images
References in corpus (17)
- Very Deep Convolutional Networks for Large-Scale Image Recognition
- Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
- OverFeat: Integrated Recognition, Localization and Detection using Convolutional Networks
- DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs
- Semantic Understanding of Scenes through the ADE20K Dataset
- DeepDriving: Learning Affordance for Direct Perception in Autonomous Driving
- Love Thy Neighbors: Image Annotation by Exploiting Image Metadata
- Building a Large-scale Multimodal Knowledge Base System for Answering Visual Queries
- Attribute Recognition by Joint Recurrent Learning of Context and Correlation
- AffordanceNet: An End-to-End Deep Learning Approach for Object Affordance Detection
- Learning Social Affordance for Human-Robot Interaction
- Transfer of Tool Affordance and Manipulation Cues with 3D Vision Data
- Learning to Act Properly: Predicting and Explaining Affordances from Images
- Social Scene Understanding: End-to-End Multi-Person Action Localization and Collective Activity Recognition
- Learning Social Affordance Grammar from Videos: Transferring Human Interactions to Human-Robot Interactions
- Adaptive Binarization for Weakly Supervised Affordance Segmentation
- Geometric Affordances from a Single Example via the Interaction Tensor
Cited by in corpus (9)
- Dark, Beyond Deep: A Paradigm Shift to Cognitive AI with Humanlike Common Sense
- RPM-Net: Recurrent Prediction of Motion and Parts from Point Cloud
- Affordance Transfer Learning for Human-Object Interaction Detection
- Block Annotation: Better Image Annotation for Semantic Segmentation with Sub-Image Decomposition
- 3D AffordanceNet: A Benchmark for Visual Object Affordance Understanding
- Deep Learning to See: Towards New Foundations of Computer Vision
- Affordance as general value function: A computational model
- One-Shot Object Affordance Detection in the Wild
- Grounded Human-Object Interaction Hotspots from Video (Extended Abstract)