Deep Exemplar-based Video Colorization
arXiv:1906.09909
Abstract
This paper presents the first end-to-end network for exemplar-based video colorization. The main challenge is to achieve temporal consistency while remaining faithful to the reference style. To address this issue, we introduce a recurrent framework that unifies the semantic correspondence and color propagation steps. Both steps allow a provided reference image to guide the colorization of every frame, thus reducing accumulated propagation errors. Video frames are colorized in sequence based on the colorization history, and its coherency is further enforced by the temporal consistency loss. All of these components, learned end-to-end, help produce realistic videos with good temporal stability. Experiments show our result is superior to the state-of-the-art methods both quantitatively and qualitatively.
References in corpus (11)
- Very Deep Convolutional Networks for Large-Scale Image Recognition
- GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
- Spectral Normalization for Generative Adversarial Networks
- The relativistic discriminator: a key element missing from standard GAN
- Non-local Neural Networks
- Deep Koalarization: Image Colorization using CNNs and Inception-ResNet-v2
- Visual Attribute Transfer through Deep Image Analogy
- Real-Time User-Guided Image Colorization with Learned Deep Priors
- Switchable Temporal Propagation Network
- Structural Consistency and Controllability for Diverse Colorization
- Learning Blind Video Temporal Consistency