789 citations · 3.9k across the 31 of their papers we have counts for
6 papers · 1 filter
GeDi: Generative Discriminator Guided Sequence Generation
Ben Krause, Akhilesh Deepak Gotmare, Bryan McCann +4
While large-scale language models (LMs) are able to imitate the distribution of natural language well enough to generate realistic text, it is difficult to control which regions of…
Unsupervised Paraphrasing with Pretrained Language Models
Tong Niu, Semih Yavuz, Yingbo Zhou +3
Paraphrase generation has benefited extensively from recent progress in the designing of training objectives and model architectures. However, previous explorations have largely fo…
Char2Subword: Extending the Subword Embedding Space Using Robust Character Compositionality
Gustavo Aguilar, Bryan McCann, Tong Niu +3
Byte-pair encoding (BPE) is a ubiquitous algorithm in the subword tokenization process of language models as it provides multiple benefits. However, this process is solely based on…
Mirostat: A Neural Text Decoding Algorithm that Directly Controls Perplexity
Sourya Basu, Govardana Sachitanandam Ramachandran, Nitish Shirish Keskar +1
Neural text decoding is important for generating high-quality texts using language models. To generate high-quality text, popular decoding algorithms like top-k, top-p (nucleus), a…
Improving out-of-distribution generalization via multi-task self-supervised pretraining
Isabela Albuquerque, Nikhil Naik, Junnan Li +2
Self-supervised feature representations have been shown to be useful for supervised classification, few-shot learning, and adversarial robustness. We show that features obtained us…
ProGen: Language Modeling for Protein Generation
Ali Madani, Bryan McCann, Nikhil Naik +5
Generative modeling for protein engineering is key to solving fundamental problems in synthetic biology, medicine, and material science. We pose protein engineering as an unsupervi…