Deep Extrapolation for Attribute-Enhanced Generation
arXiv:2107.02968
Abstract
Attribute extrapolation in sample generation is challenging for deep neural networks operating beyond the training distribution. We formulate a new task for extrapolation in sequence generation, focusing on natural language and proteins, and propose GENhance, a generative framework that enhances attributes through a learned latent space. Trained on movie reviews and a computed protein stability dataset, GENhance can generate strongly-positive text reviews and highly stable protein sequences without being exposed to similar data during training. We release our benchmark tasks and models to contribute to the study of generative modeling extrapolation and data-driven design in biology and chemistry.
arXiv commentNeurIPS 2021
References in corpus (20)
- SMOTE: Synthetic Minority Over-sampling Technique
- Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
- HuggingFace's Transformers: State-of-the-art Natural Language Processing
- CTRL: A Conditional Transformer Language Model for Controllable Generation
- Style Transfer from Non-Parallel Text by Cross-Alignment
- MolGAN: An implicit generative model for small molecular graphs
- Fine-Tuning Language Models from Human Preferences
- Learning Imbalanced Datasets with Label-Distribution-Aware Margin Loss
- Generalization and Equilibrium in Generative Adversarial Nets (GANs)
- Wasserstein Auto-Encoders
- Delving into Deep Imbalanced Regression
- How Neural Networks Extrapolate: From Feedforward to Graph Neural Networks
- Optimizing Molecules using Efficient Queries from Property Evaluations
- Unsupervised Text Style Transfer using Language Models as Discriminators
- Controlling Output Length in Neural Encoder-Decoders
- ProGen: Language Modeling for Protein Generation
- Design by adaptive sampling
- Model Inversion Networks for Model-Based Optimization
- Autofocused oracles for model-based design
- A Stable Variational Autoencoder for Text Modelling