Improving Classification by Improving Labelling: Introducing Probabilistic Multi-Label Object Interaction Recognition
arXiv:1703.08338
Abstract
This work deviates from easy-to-define class boundaries for object interactions. For the task of object interaction recognition, often captured using an egocentric view, we show that semantic ambiguities in verbs and recognising sub-interactions along with concurrent interactions result in legitimate class overlaps (Figure 1). We thus aim to model the mapping between observations and interaction classes, as well as class overlaps, towards a probabilistic multi-label classifier that emulates human annotators. Given a video segment containing an object interaction, we model the probability for a verb, out of a list of possible verbs, to be used to annotate that interaction. The proba- bility is learnt from crowdsourced annotations, and is tested on two public datasets, comprising 1405 video sequences for which we provide annotations on 90 verbs. We outper- form conventional single-label classification by 11% and 6% on the two datasets respectively, and show that learning from annotation probabilities outperforms majority voting and enables discovery of co-occurring labels.
References in corpus (12)
- UCF101: A Dataset of 101 Human Actions Classes From Videos in The Wild
- Deep Captioning with Multimodal Recurrent Neural Networks (m-RNN)
- Convolutional Two-Stream Network Fusion for Video Action Recognition
- Deep Convolutional Ranking for Multilabel Image Annotation
- Hollywood in Homes: Crowdsourcing Data Collection for Activity Understanding
- Describing Videos by Exploiting Temporal Structure
- Learning Spatiotemporal Features with 3D Convolutional Networks
- Language Models for Image Captioning: The Quirks and What Works
- Video Paragraph Captioning Using Hierarchical Recurrent Neural Networks
- Multi-Label Image Classification with Regional Latent Semantic Dependencies
- Deep Compositional Captioning: Describing Novel Object Categories without Paired Training Data
- Exploit Bounding Box Annotations for Multi-label Object Recognition