A Neural Representation of Sketch Drawings

Abstract

We present sketch-rnn, a recurrent neural network (RNN) able to construct stroke-based drawings of common objects. The model is trained on thousands of crude human-drawn images representing hundreds of classes. We outline a framework for conditional and unconditional sketch generation, and describe new robust training methods for generating coherent sketch drawings in a vector format.

Cited by in corpus (142)

• A Hierarchical Latent Vector Model for Learning Long-Term Structure in Music
• Deep Learning for Procedural Content Generation
• PointCNN: Convolution On $\mathcal{X}$-Transformed Points
• Continual Learning for Recurrent Neural Networks: an Empirical Evaluation
• Friend, Collaborator, Student, Manager: How Design of an AI-Driven Game Level Editor Affects Creators
• Pervasive Label Errors in Test Sets Destabilize Machine Learning Benchmarks
• Choice Over Control: How Users Write with Large Language Models using Diegetic and Non-Diegetic Prompting
• A Deep Generative Model for Graph Layout
• Super-resolution image display using diffractive decoders
• Deep Vectorization of Technical Drawings
• Content and Style Aware Generation of Text-line Images for Handwriting Recognition
• Facial-Sketch Synthesis: A New Challenge
• NeuralTailor: Reconstructing Sewing Pattern Structures from 3D Point Clouds of Garments
• Learning to Paint With Model-based Deep Reinforcement Learning
• Computer-Aided Design as Language
• Pragmatic inference and visual abstraction enable contextual flexibility during visual communication
• SketchGraphs: A Large-Scale Dataset for Modeling Relational Geometry in Computer-Aided Design
• Learning a Latent Space of Multitrack Measures
• Multi-Graph Transformer for Free-Hand Sketch Recognition
• DeepSVG: A Hierarchical Generative Network for Vector Graphics Animation
• ClipGen: A Deep Generative Model for Clipart Vectorization and Synthesis
• Sketch-based Creativity Support Tools using Deep Learning
• Visual Intelligence through Human Interaction
• Deep Learning for Free-Hand Sketch: A Survey
• StarNet: Pedestrian Trajectory Prediction using Deep Neural Network in Star Topology
• Unsupervised Doodling and Painting with Improved SPIRAL
• Skill Rating for Generative Models
• Cross-Modal Hierarchical Modelling for Fine-Grained Sketch Based Image Retrieval
• Learning to Sketch with Deep Q Networks and Demonstrated Strokes
• Creative Sketch Generation
• One Sketch for All: One-Shot Personalized Sketch Segmentation
• Sketch-R2CNN: An Attentive Network for Vector Sketch Recognition
• Neural Attribute Machines for Program Generation
• Stroke-based sketched symbol reconstruction and segmentation
• A New Multi-vehicle Trajectory Generator to Simulate Vehicle-to-Vehicle Encounters
• Interactive Sketch & Fill: Multiclass Sketch-to-Image Translation
• DeepVecFont: Synthesizing High-quality Vector Fonts via Dual-modality Learning
• Im2Oil: Stroke-Based Oil Painting Rendering with Linearly Controllable Fineness Via Adaptive Sampling
• Learning abstract structure for drawing by efficient motor program induction
• Deep hybrid model with satellite imagery: how to combine demand modeling and computer vision for behavior analysis?
• Model-based Reinforcement Learning for Predictions and Control for Limit Order Books
• Semi-parametric Image Inpainting
• CoSE: Compositional Stroke Embeddings
• Sketch Less for More: On-the-Fly Fine-Grained Sketch Based Image Retrieval
• Auto Completion of User Interface Layout Design Using Transformer-Based Tree Decoders
• SFSegNet: Parse Freehand Sketches using Deep Fully Convolutional Networks
• More Photos are All You Need: Semi-Supervised Learning for Fine-Grained Sketch Based Image Retrieval
• From Scratch to Sketch: Deep Decoupled Hierarchical Reinforcement Learning for Robotic Sketching Agent
• Unsupervised Image to Sequence Translation with Canvas-Drawer Networks
• Machine learning based co-creative design framework
• Who Left the Dogs Out? 3D Animal Reconstruction with Expectation Maximization in the Loop
• StyleMeUp: Towards Style-Agnostic Sketch-Based Image Retrieval
• Differentiable Drawing and Sketching
• Adaptive Parameterization for Neural Dialogue Generation
• Neuralogram: A Deep Neural Network Based Representation for Audio Signals
• Encoding Musical Style with Transformer Autoencoders
• Biases in Generative Art -- A Causal Look from the Lens of Art History
• The DIDI dataset: Digital Ink Diagram data
• PaintBot: A Reinforcement Learning Approach for Natural Media Painting
• Breaking the Activation Function Bottleneck through Adaptive Parameterization
• Sketch-BERT: Learning Sketch Bidirectional Encoder Representation from Transformers by Self-supervised Learning of Sketch Gestalt
• Sketchforme: Composing Sketched Scenes from Text Descriptions for Interactive Applications
• Vectorization and Rasterization: Self-Supervised Learning for Sketch and Handwriting
• Interactive Neural Painting
• Dialog on a canvas with a machine
• Photo-Sketching: Inferring Contour Drawings from Images
• Teaching GANs to Sketch in Vector Format
• GANwriting: Content-Conditioned Generation of Styled Handwritten Word Images
• PSDoodle: Fast App Screen Search via Partial Screen Doodle
• Training Normalizing Flows with the Information Bottleneck for Competitive Generative Classification
• NeuNetS: An Automated Synthesis Engine for Neural Network Design
• SketchyCOCO: Image Generation from Freehand Scene Sketches
• Unsupervised Sketch-to-Photo Synthesis
• BézierSketch: A generative model for scalable vector sketches
• LUCSS: Language-based User-customized Colourization of Scene Sketches
• Emergent Graphical Conventions in a Visual Communication Game
• Creating User Interface Mock-ups from High-Level Text Descriptions with Deep-Learning Models
• Learning Neurosymbolic Generative Models via Program Synthesis
• Session-based Sequential Skip Prediction via Recurrent Neural Networks
• SketchGNN: Semantic Sketch Segmentation with Graph Neural Networks
• Generating new concepts with hybrid neuro-symbolic models
• Learning Task-General Representations with Generative Neuro-Symbolic Modeling
• Deep Residual Mixture Models
• Mechanisms of Artistic Creativity in Deep Learning Neural Networks
• Coevo: a collaborative design platform with artificial agents
• A General Model for Aggregating Annotations Across Simple, Complex, and Multi-Object Annotation Tasks
• Trick or TReAT: Thematic Reinforcement for Artistic Typography
• Towards Movement Generation with Audio Features
• On Predictive Information in RNNs
• LPaintB: Learning to Paint from Self-Supervision
• Continuous 3D Multi-Channel Sign Language Production via Progressive Transformers and Mixture Density Networks
• SketchEmbedNet: Learning Novel Concepts by Imitating Drawings
• SketchDesc: Learning Local Sketch Descriptors for Multi-view Correspondence
• Geometric Losses for Distributional Learning
• VMAV-C: A Deep Attention-based Reinforcement Learning Algorithm for Model-based Control
• Deep Self-Supervised Representation Learning for Free-Hand Sketch
• Adversarial recovery of agent rewards from latent spaces of the limit order book
• Contextualizing Enhances Gradient Based Meta Learning
• On Learning Semantic Representations for Million-Scale Free-Hand Sketches
• Cloud2Curve: Generation and Vectorization of Parametric Sketches
• Text-to-Image Generation Grounded by Fine-Grained User Attention
• From Language Games to Drawing Games
• Deep Learning for Identifying Potential Conceptual Shifts for Co-creative Drawing
• Generative Adversarial Networks are Special Cases of Artificial Curiosity (1990) and also Closely Related to Predictability Minimization (1991)
• Humans learn too: Better Human-AI Interaction using Optimized Human Inputs
• Imaginative Walks: Generative Random Walk Deviation Loss for Improved Unseen Learning Representation
• Combining Semantic Guidance and Deep Reinforcement Learning For Generating Human Level Paintings
• Large-Scale Meta-Learning with Continual Trajectory Shifting
• Can I teach a robot to replicate a line art
• Learning Calibratable Policies using Programmatic Style-Consistency
• Vitruvion: A Generative Model of Parametric CAD Sketches
• Goal-Driven Sequential Data Abstraction
• Sketch Generation with Drawing Process Guided by Vector Flow and Grayscale
• Probabilistic Verification of Fairness Properties via Concentration
• CIZSL++: Creativity Inspired Generative Zero-Shot Learning
• Universal Perceptual Grouping
• Generative Creativity: Adversarial Learning for Bionic Design
• Spatiotemporal Data Fusion for Precipitation Nowcasting
• Mutltimodal AI Companion for Interactive Fairytale Co-creation
• Multi-class Novelty Detection Using Mix-up Technique
• Learning to See via Epiretinal Implant Stimulation in silico with Model-Based Deep Reinforcement Learning
• CanvasVAE: Learning to Generate Vector Graphic Documents
• Sketch-Inspector: a Deep Mixture Model for High-Quality Sketch Generation of Cats
• Fine-Grained Instance-Level Sketch-Based Video Retrieval
• Synthesizing human-like sketches from natural images using a conditional convolutional decoder
• A Framework to Learn with Interpretation
• Harmonic Recomposition using Conditional Autoregressive Modeling
• An Infinite Parade of Giraffes: Expressive Augmentation and Complexity Layers for Cartoon Drawing
• Dual Reinforcement-Based Specification Generation for Image De-Rendering
• Do Time Constraints Re-Prioritize Attention to Shapes During Visual Photo Inspection?
• Online trajectory recovery from offline handwritten Japanese kanji characters
• Unsupervised Program Synthesis for Images By Sampling Without Replacement
• FaCells. Teaching Machines the Language of Lines: Per Point Attribute Scores for Face-Sketch Classification
• LiveSketch: Query Perturbations for Guided Sketch-based Visual Search
• MT-VAE: Learning Motion Transformations to Generate Multimodal Human Dynamics
• Designing Games for Enabling Co-creation with Social Agents
• Shallow Art: Art Extension Through Simple Machine Learning
• Generating the support with extreme value losses
• Deep Reinforced Attention Regression for Partial Sketch Based Image Retrieval
• SketchTransfer: A Challenging New Task for Exploring Detail-Invariance and the Abstractions Learned by Deep Networks
• Processsing Simple Geometric Attributes with Autoencoders
• Generation Drawing/Grinding Trajectoy Based on Hierarchical CVAE