LIGAR: Lightweight General-purpose Action Recognition

Abstract

Growing amount of different practical tasks in a video understanding problem has addressed the great challenge aiming to design an universal solution, which should be available for broad masses and suitable for the demanding edge-oriented inference. In this paper we are focused on designing a network architecture and a training pipeline to tackle the mentioned challenges. Our architecture takes the best from the previous ones and brings the ability to be successful not only in appearance-based action recognition tasks but in motion-based problems too. Furthermore, the induced label noise problem is formulated and Adaptive Clip Selection (ACS) framework is proposed to deal with it. Together it makes the LIGAR framework the general-purpose action recognition solution. We also have reported the extensive analysis on the general and gesture datasets to show the excellent trade-off between the performance and the accuracy in comparison to the state-of-the-art solutions. Training code is available at: https://github.com/openvinotoolkit/training_extensions. For the efficient edge-oriented inference all trained models can be exported into the OpenVINO format.

References in corpus (23)

• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• mixup: Beyond Empirical Risk Minimization
• UCF101: A Dataset of 101 Human Actions Classes From Videos in The Wild
• Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour
• ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices
• Convolutional Two-Stream Network Fusion for Video Action Recognition
• Temporal Segment Networks: Towards Good Practices for Deep Action Recognition
• High-Performance Large-Scale Image Recognition Without Normalization
• ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Design
• TSM: Temporal Shift Module for Efficient Video Understanding
• SlowFast Networks for Video Recognition
• MS-ASL: A Large-Scale Data Set and Benchmark for Understanding American Sign Language
• STM: SpatioTemporal and Motion Encoding for Action Recognition
• PAN: Towards Fast Action Recognition via Learning Persistence of Appearance
• MoViNets: Mobile Video Networks for Efficient Video Recognition
• Divide and Conquer the Embedding Space for Metric Learning
• Noise-resistant Deep Metric Learning with Ranking-based Instance Selection
• ImageNet-21K Pretraining for the Masses
• ASL Recognition with Metric-Learning based Lightweight Network
• Learning to Combat Noisy Labels via Classification Margins
• A New Training Framework for Deep Neural Network
• Efficient Spatialtemporal Context Modeling for Action Recognition
• Global Context Networks