466 citations · 1.1k across the 8 of their papers we have counts for
23 papers
Modeling Multi-hop Question Answering as Single Sequence Prediction
Semih Yavuz, Kazuma Hashimoto, Yingbo Zhou +2
Fusion-in-decoder (Fid) (Izacard and Grave, 2020) is a generative question answering (QA) model that leverages passage retrieval with a pre-trained transformer and pushed the state…
GeDi: Generative Discriminator Guided Sequence Generation
Ben Krause, Akhilesh Deepak Gotmare, Bryan McCann +4
While large-scale language models (LMs) are able to imitate the distribution of natural language well enough to generate realistic text, it is difficult to control which regions of…
Unsupervised Paraphrasing with Pretrained Language Models
Tong Niu, Semih Yavuz, Yingbo Zhou +3
Paraphrase generation has benefited extensively from recent progress in the designing of training objectives and model architectures. However, previous explorations have largely fo…
Char2Subword: Extending the Subword Embedding Space Using Robust Character Compositionality
Gustavo Aguilar, Bryan McCann, Tong Niu +3
Byte-pair encoding (BPE) is a ubiquitous algorithm in the subword tokenization process of language models as it provides multiple benefits. However, this process is solely based on…
Mirostat: A Neural Text Decoding Algorithm that Directly Controls Perplexity
Sourya Basu, Govardana Sachitanandam Ramachandran, Nitish Shirish Keskar +1
Neural text decoding is important for generating high-quality texts using language models. To generate high-quality text, popular decoding algorithms like top-k, top-p (nucleus), a…
Improving out-of-distribution generalization via multi-task self-supervised pretraining
Isabela Albuquerque, Nikhil Naik, Junnan Li +2
Self-supervised feature representations have been shown to be useful for supervised classification, few-shot learning, and adversarial robustness. We show that features obtained us…