Models Matter, So Does Training: An Empirical Study of CNNs for Optical Flow Estimation
arXiv:1809.05571
Abstract
We investigate two crucial and closely related aspects of CNNs for optical flow estimation: models and training. First, we design a compact but effective CNN model, called PWC-Net, according to simple and well-established principles: pyramidal processing, warping, and cost volume processing. PWC-Net is 17 times smaller in size, 2 times faster in inference, and 11\% more accurate on Sintel final than the recent FlowNet2 model. It is the winning entry in the optical flow competition of the robust vision challenge. Next, we experimentally analyze the sources of our performance gains. In particular, we use the same training procedure of PWC-Net to retrain FlowNetC, a sub-network of FlowNet2. The retrained FlowNetC is 56\% more accurate on Sintel final than the previously trained one and even 5\% more accurate than the FlowNet2 model. We further improve the training procedure and increase the accuracy of PWC-Net on Sintel by 10\% and on KITTI 2012 and 2015 by 20\%. Our newly trained model parameters and training protocols will be available on https://github.com/NVlabs/PWC-Net
References in corpus (8)
- Very Deep Convolutional Networks for Large-Scale Image Recognition
- Deep Residual Learning for Image Recognition
- Caffe: Convolutional Architecture for Fast Feature Embedding
- Multi-Scale Context Aggregation by Dilated Convolutions
- Going Deeper with Convolutions
- PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volume
- LiteFlowNet: A Lightweight Convolutional Neural Network for Optical Flow Estimation
- The One Hundred Layers Tiramisu: Fully Convolutional DenseNets for Semantic Segmentation
Cited by in corpus (9)
- RAFT: Recurrent All-Pairs Field Transforms for Optical Flow
- SelFlow: Self-Supervised Learning of Optical Flow
- Hierarchical Contrastive Motion Learning for Video Action Recognition
- A Fusion Approach for Multi-Frame Optical Flow Estimation
- Self-Supervised Pillar Motion Learning for Autonomous Driving
- Deep Reference Generation with Multi-Domain Hierarchical Constraints for Inter Prediction
- A Deep Temporal Fusion Framework for Scene Flow Using a Learnable Motion Model and Occlusions
- Learning Optical Flow from a Few Matches
- An Empirical Evaluation Study on the Training of SDC Features for Dense Pixel Matching