Controllable Data Generation by Deep Learning: A Review
arXiv:2207.09542 · doi:10.1145/3648609
Abstract
Designing and generating new data under targeted properties has been attracting various critical applications such as molecule design, image editing and speech synthesis. Traditional hand-crafted approaches heavily rely on expertise experience and intensive human efforts, yet still suffer from the insufficiency of scientific knowledge and low throughput to support effective and efficient data generation. Recently, the advancement of deep learning has created the opportunity for expressive methods to learn the underlying representation and properties of data. Such capability provides new ways of determining the mutual relationship between the structural patterns and functional properties of the data and leveraging such relationships to generate structural data, given the desired properties. This article is a systematic review that explains this promising research area, commonly known as controllable deep data generation. First, the article raises the potential challenges and provides preliminaries. Then the article formally defines controllable deep data generation, proposes a taxonomy on various techniques and summarizes the evaluation metrics in this specific domain. After that, the article introduces exciting applications of controllable deep data generation, experimentally analyzes and compares existing works. Finally, this article highlights the promising future directions of controllable deep data generation and identifies five potential challenges.
This survey has been accepted by ACM Computing Surveys
References in corpus (31)
- Deep Learning in Neural Networks: An Overview
- Conditional Generative Adversarial Nets
- A Brief Survey of Deep Reinforcement Learning
- Diffusion Models Beat GANs on Image Synthesis
- Score-Based Generative Modeling through Stochastic Differential Equations
- Classifier-Free Diffusion Guidance
- Deep learning for molecular design - a review of the state of the art
- Guided Image Generation with Conditional Invertible Neural Networks
- MoFlow: An Invertible Flow Model for Generating Molecular Graphs
- A Survey on Neural Speech Synthesis
- Human Motion Diffusion Model
- Discovering Hidden Factors of Variation in Deep Networks
- A survey on text generation using generative adversarial networks
- MelNet: A Generative Model for Audio in the Frequency Domain
- GraphNVP: An Invertible Flow Model for Generating Molecular Graphs
- Sample Efficiency Matters: A Benchmark for Practical Molecular Optimization
- Multi-reference Tacotron by Intercross Training for Style Disentangling,Transfer and Control in Speech Synthesis
- An Application of Generative Adversarial Networks for Super Resolution Medical Imaging
- A Survey of Inductive Biases for Factorial Representation-Learning
- GAUCHE: A Library for Gaussian Processes in Chemistry
- Interpretable Deep Graph Generation with Node-Edge Co-Disentanglement
- MolGenSurvey: A Systematic Survey in Machine Learning Models for Molecule Design
- GraphEBM: Molecular Graph Generation with Energy-Based Models
- Review of end-to-end speech synthesis technology based on deep learning
- A Survey on Deep Graph Generation: Methods and Applications
- Model architectures to extrapolate emotional expressions in DNN-based text-to-speech
- C5T5: Controllable Generation of Organic Molecules with Transformers
- Deep Generation of Heterogeneous Networks
- MEGATRON-CNTRL: Controllable Story Generation with External Knowledge Using Large-Scale Language Models
- Enhancing Diffusion-Based Image Synthesis with Robust Classifier Guidance
- Speech Synthesis and Control Using Differentiable DSP