PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volume
arXiv:1709.02371
Abstract
We present a compact but effective CNN model for optical flow, called PWC-Net. PWC-Net has been designed according to simple and well-established principles: pyramidal processing, warping, and the use of a cost volume. Cast in a learnable feature pyramid, PWC-Net uses the cur- rent optical flow estimate to warp the CNN features of the second image. It then uses the warped features and features of the first image to construct a cost volume, which is processed by a CNN to estimate the optical flow. PWC-Net is 17 times smaller in size and easier to train than the recent FlowNet2 model. Moreover, it outperforms all published optical flow methods on the MPI Sintel final pass and KITTI 2015 benchmarks, running at about 35 fps on Sintel resolution (1024x436) images. Our models are available on https://github.com/NVlabs/PWC-Net.
CVPR 2018 camera ready version (with github link to Caffe and PyTorch code)
Cited by in corpus (13)
- Pyramid Stereo Matching Network
- Super SloMo: High Quality Estimation of Multiple Intermediate Frames for Video Interpolation
- A robust single-pixel particle image velocimetry based on fully convolutional networks with cross-correlation embedded
- DDCNet: Deep Dilated Convolutional Neural Network for Dense Prediction
- Models Matter, So Does Training: An Empirical Study of CNNs for Optical Flow Estimation
- Occlusion Aware Unsupervised Learning of Optical Flow
- Joint Unsupervised Learning of Optical Flow and Depth by Watching Stereo Videos
- ENG: End-to-end Neural Geometry for Robust Depth and Pose Estimation using CNNs
- Every Pixel Counts: Unsupervised Geometry Learning with Holistic 3D Motion Understanding
- Asphalt Concrete Characterization Using Digital Image Correlation: A Systematic Review of Best Practices, Applications, and Future Vision
- Reblur2Deblur: Deblurring Videos via Self-Supervised Learning
- Follow the Attention: Combining Partial Pose and Object Motion for Fine-Grained Action Detection
- Learning Parallax Attention for Stereo Image Super-Resolution