SlowFast Networks for Video Recognition

Abstract

We present SlowFast networks for video recognition. Our model involves (i) a Slow pathway, operating at low frame rate, to capture spatial semantics, and (ii) a Fast pathway, operating at high frame rate, to capture motion at fine temporal resolution. The Fast pathway can be made very lightweight by reducing its channel capacity, yet can learn useful temporal information for video recognition. Our models achieve strong performance for both action classification and detection in video, and large improvements are pin-pointed as contributions by our SlowFast concept. We report state-of-the-art accuracy on major video recognition benchmarks, Kinetics, Charades and AVA. Code has been made available at: https://github.com/facebookresearch/SlowFast

arXiv commentTechnical report

References in corpus (13)

• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
• Improving neural networks by preventing co-adaptation of feature detectors
• Two-Stream Convolutional Networks for Action Recognition in Videos
• The Kinetics Human Action Video Dataset
• Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour
• SGDR: Stochastic Gradient Descent with Warm Restarts
• Spatiotemporal Residual Networks for Video Action Recognition
• A Short Note on the Kinetics-700-2020 Human Action Dataset
• A Short Note about Kinetics-600
• The ActivityNet Large-Scale Activity Recognition Challenge 2018 Summary
• A Better Baseline for AVA
• Exploiting Spatial-Temporal Modelling and Multi-Modal Fusion for Human Action Recognition

Cited by in corpus (57)

• GCNet: Non-local Networks Meet Squeeze-Excitation Networks and Beyond
• Drop an Octave: Reducing Spatial Redundancy in Convolutional Neural Networks with Octave Convolution
• A Survey on 3D Skeleton-Based Action Recognition Using Learning Method
• STM: SpatioTemporal and Motion Encoding for Action Recognition
• Asymmetric Residual Neural Network for Accurate Human Activity Recognition
• Video Representation Learning by Dense Predictive Coding
• V4D:4D Convolutional Neural Networks for Video-level Representation Learning
• Learning Spatiotemporal Features via Video and Text Pair Discrimination
• Spatio-Temporal Graph for Video Captioning with Knowledge Distillation
• OREBA: A Dataset for Objectively Recognizing Eating Behaviour and Associated Intake
• Make Skeleton-based Action Recognition Model Smaller, Faster and Better
• TEINet: Towards an Efficient Architecture for Video Recognition
• AssembleNet: Searching for Multi-Stream Neural Connectivity in Video Architectures
• AdaFuse: Adaptive Temporal Fusion Network for Efficient Action Recognition
• Grouped Spatial-Temporal Aggregation for Efficient Action Recognition
• AViD Dataset: Anonymized Videos from Diverse Countries
• Rethinking Motion Representation: Residual Frames with 3D ConvNets for Better Action Recognition
• Multi-Label Classification with Label Graph Superimposing
• Dynamic Mini-batch SGD for Elastic Distributed Training: Learning in the Limbo of Resources
• EnsembleNet: End-to-End Optimization of Multi-headed Models
• A Study on Action Detection in the Wild
• Deep Analysis of CNN-based Spatio-temporal Representations for Action Recognition
• Unsupervised Learning from Video with Deep Neural Embeddings
• Temporal Interlacing Network
• Disentangled Non-Local Neural Networks
• TubeTK: Adopting Tubes to Track Multi-Object in a One-Step Training Model
• Intra- and Inter-Action Understanding via Temporal Action Parsing
• Relation Modeling in Spatio-Temporal Action Localization
• Towards Real-Time Action Recognition on Mobile Devices Using Deep Models
• Three Branches: Detecting Actions With Richer Features
• RoVISQ: Reduction of Video Service Quality via Adversarial Attacks on Deep Learning-based Video Compression
• Zero-Shot Learning with Knowledge Enhanced Visual Semantic Embeddings
• Explaining Motion Relevance for Activity Recognition in Video Deep Learning Models
• Learning Efficient Video Representation with Video Shuffle Networks
• Context-Aware RCNN: A Baseline for Action Detection in Videos
• Discriminability Distillation in Group Representation Learning
• Residual Frames with Efficient Pseudo-3D CNN for Human Action Recognition
• Interpretable and Trustworthy Deepfake Detection via Dynamic Prototypes
• Beyond the Camera: Neural Networks in World Coordinates
• Unsupervised Discovery of Actions in Instructional Videos
• Deformable Tube Network for Action Detection in Videos
• AFO-TAD: Anchor-free One-Stage Detector for Temporal Action Detection
• LIGAR: Lightweight General-purpose Action Recognition
• Knowledge Integration Networks for Action Recognition
• Class Semantics-based Attention for Action Detection
• Top-1 Solution of Multi-Moments in Time Challenge 2019
• Spatial Priming for Detecting Human-Object Interactions
• Multi-Scale Feature Fusion using Parallel-Attention Block for COVID-19 Chest X-ray Diagnosis
• A Unified Framework for Shot Type Classification Based on Subject Centric Lens
• Writing in The Air: Unconstrained Text Recognition from Finger Movement Using Spatio-Temporal Convolution
• Learning-Based Human Segmentation and Velocity Estimation Using Automatic Labeled LiDAR Sequence for Training
• Temporal Action Localization with Variance-Aware Networks
• Global Context Networks
• Image to Video Domain Adaptation Using Web Supervision
• iqiyi Submission to ActivityNet Challenge 2019 Kinetics-700 challenge: Hierarchical Group-wise Attention
• PANDA: A Gigapixel-level Human-centric Video Dataset
• 3D attention mechanism for fine-grained classification of table tennis strokes using a Twin Spatio-Temporal Convolutional Neural Networks