Local Temporal Bilinear Pooling for Fine-grained Action Parsing

Abstract

Fine-grained temporal action parsing is important in many applications, such as daily activity understanding, human motion analysis, surgical robotics and others requiring subtle and precise operations in a long-term period. In this paper we propose a novel bilinear pooling operation, which is used in intermediate layers of a temporal convolutional encoder-decoder net. In contrast to other work, our proposed bilinear pooling is learnable and hence can capture more complex local statistics than the conventional counterpart. In addition, we introduce exact lower-dimension representations of our bilinear forms, so that the dimensionality is reduced with neither information loss nor extra computation. We perform intensive experiments to quantitatively analyze our model and show the superior performances to other state-of-the-art work on various datasets.

arXiv comment11 pages, 2 figures. Cam.R

References in corpus (11)

• Action Recognition with Trajectory-Pooled Deep-Convolutional Descriptors
• Searching for Activation Functions
• Multimodal Compact Bilinear Pooling for Visual Question Answering and Visual Grounding
• Non-local Neural Networks
• Compact Generalized Non-local Network
• Multi-modal Factorized Bilinear Pooling with Co-Attention Learning for Visual Question Answering
• Improved Bilinear Pooling with CNNs
• Towards Faster Training of Global Covariance Pooling Networks by Iterative Matrix Square Root Normalization
• Temporal Human Action Segmentation via Dynamic Clustering
• Learning Motion in Feature Space: Locally-Consistent Deformable Convolution Networks for Fine-Grained Action Detection
• Second-order Democratic Aggregation