Towards an Unequivocal Representation of Actions
arXiv:1805.04026
Abstract
This work introduces verb-only representations for actions and interactions; the problem of describing similar motions (e.g. 'open door', 'open cupboard'), and distinguish differing ones (e.g. 'open door' vs 'open bottle') using verb-only labels. Current approaches for action recognition neglect legitimate semantic ambiguities and class overlaps between verbs (Fig. 1), relying on the objects to disambiguate interactions. We deviate from single-verb labels and introduce a mapping between observations and multiple verb labels - in order to create an Unequivocal Representation of Actions. The new representation benefits from increased vocabulary and a soft assignment to an enriched space of verb labels. We learn these representations as multi-output regression, using a two-stream fusion CNN. The proposed approach outperforms conventional single-verb labels (also known as majority voting) on three egocentric datasets for both recognition and retrieval.
References in corpus (8)
- Efficient Estimation of Word Representations in Vector Space
- UCF101: A Dataset of 101 Human Actions Classes From Videos in The Wild
- The Kinetics Human Action Video Dataset
- CNN: Single-label to Multi-label
- Convolutional Two-Stream Network Fusion for Video Action Recognition
- Hollywood in Homes: Crowdsourcing Data Collection for Activity Understanding
- Describing Common Human Visual Actions in Images
- Improving Classification by Improving Labelling: Introducing Probabilistic Multi-Label Object Interaction Recognition