MarioNette: Self-Supervised Sprite Learning
arXiv:2104.14553
Abstract
Artists and video game designers often construct 2D animations using libraries of sprites -- textured patches of objects and characters. We propose a deep learning approach that decomposes sprite-based video animations into a disentangled representation of recurring graphic elements in a self-supervised manner. By jointly learning a dictionary of possibly transparent patches and training a network that places them onto a canvas, we deconstruct sprite-based content into a sparse, consistent, and explicit representation that can be easily used in downstream tasks, like editing or analysis. Our framework offers a promising approach for discovering recurring visual patterns in image collections without supervision.
arXiv commentAccepted to NeurIPS 2021
References in corpus (10)
- Decoupled Weight Decay Regularization
- Categorical Reparameterization with Gumbel-Softmax
- MONet: Unsupervised Scene Decomposition and Representation
- GAN Dissection: Visualizing and Understanding Generative Adversarial Networks
- Investigating Human Priors for Playing Video Games
- SPACE: Unsupervised Object-Oriented Scene Representation via Spatial Attention and Decomposition
- Partial Convolution based Padding
- Efficient inference in occlusion-aware generative models of images
- SCALOR: Generative World Models with Scalable Object Representations
- StampNet: unsupervised multi-class object discovery