Unsupervised Doodling and Painting with Improved SPIRAL

Abstract

We investigate using reinforcement learning agents as generative models of images (extending arXiv:1804.01118). A generative agent controls a simulated painting environment, and is trained with rewards provided by a discriminator network simultaneously trained to assess the realism of the agent's samples, either unconditional or reconstructions. Compared to prior work, we make a number of improvements to the architectures of the agents and discriminators that lead to intriguing and at times surprising results. We find that when sufficiently constrained, generative agents can learn to produce images with a degree of visual abstraction, despite having only ever seen real photographs (no human brush strokes). And given enough time with the painting environment, they can produce images with considerable realism. These results show that, under the right circumstances, some aspects of human drawing can emerge from simulated embodiment, without the need for external supervision, imitation or social cues. Finally, we note the framework's potential for use in creative applications.

arXiv commentSee https://learning-to-paint.github.io for an interactive version of this paper, with videos

References in corpus (12)

• Adam: A Method for Stochastic Optimization
• Large Scale GAN Training for High Fidelity Natural Image Synthesis
• Pixel Recurrent Neural Networks
• Progressive Growing of GANs for Improved Quality, Stability, and Variation
• A Neural Representation of Sketch Drawings
• Learning to Paint With Model-based Deep Reinforcement Learning
• Pragmatic inference and visual abstraction enable contextual flexibility during visual communication
• Neural Painters: A learned differentiable constraint for generating brushstroke paintings
• Learning to Sketch with Deep Q Networks and Demonstrated Strokes
• Learning To Simulate
• Unsupervised Image to Sequence Translation with Canvas-Drawer Networks
• Generating Diverse Programs with Instruction Conditioned Reinforced Adversarial Learning

Cited by in corpus (11)

• Why Do Line Drawings Work? A Realism Hypothesis
• Creative Sketch Generation
• Text and Style Conditioned GAN for Generation of Offline Handwriting Lines
• Learning abstract structure for drawing by efficient motor program induction
• From Scratch to Sketch: Deep Decoupled Hierarchical Reinforcement Learning for Robotic Sketching Agent
• Differentiable Drawing and Sketching
• Multi-Plane Program Induction with 3D Box Priors
• Deep Robot Sketching: An application of Deep Q-Learning Networks for human-like sketching
• From Language Games to Drawing Games
• Combining Semantic Guidance and Deep Reinforcement Learning For Generating Human Level Paintings
• Learning to See via Epiretinal Implant Stimulation in silico with Model-Based Deep Reinforcement Learning