Painting Many Pasts: Synthesizing Time Lapse Videos of Paintings
arXiv:2001.01026
Abstract
We introduce a new video synthesis task: synthesizing time lapse videos depicting how a given painting might have been created. Artists paint using unique combinations of brushes, strokes, and colors. There are often many possible ways to create a given painting. Our goal is to learn to capture this rich range of possibilities. Creating distributions of long-term videos is a challenge for learning-based video synthesis methods. We present a probabilistic model that, given a single image of a completed painting, recurrently synthesizes steps of the painting process. We implement this model as a convolutional neural network, and introduce a novel training scheme to enable learning from a limited dataset of painting time lapses. We demonstrate that this model can be used to sample many time steps, enabling long-term stochastic video synthesis. We evaluate our method on digital and watercolor paintings collected from video websites, and show that human raters find our synthetic videos to be similar to time lapse videos produced by real artists. Our code is available at https://xamyzhao.github.io/timecraft.
arXiv comment10 pages, CVPR 2020
References in corpus (10)
- Very Deep Convolutional Networks for Large-Scale Image Recognition
- Auto-Encoding Variational Bayes
- TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems
- Semi-Supervised Learning with Deep Generative Models
- Video (language) modeling: a baseline for generative models of natural videos
- Learning to Generate Long-term Future via Hierarchical Prediction
- Visual Dynamics: Probabilistic Future Frame Synthesis via Cross Convolutional Networks
- Synthesizing Programs for Images using Reinforced Adversarial Learning
- Artist Agent: A Reinforcement Learning Approach to Automatic Stroke Generation in Oriental Ink Painting
- LPaintB: Learning to Paint from Self-Supervision