1.3k citations · 2.9k across the 17 of their papers we have counts for
17 papers
Dr Genre: Reinforcement Learning from Decoupled LLM Feedback for Generic Text Rewriting
Yufei Li, John Nham, Ganesh Jawahar +7
Generic text rewriting is a prevalent large language model (LLM) application that covers diverse real-world tasks, such as style transfer, fact correction, and email editing. These…
Characterizing Tradeoffs in Language Model Decoding with Informational Interpretations
Chung-Ching Chang, William W. Cohen, Yun-Hsuan Sung
We propose a theoretical framework for formulating language model decoder algorithms with dynamic programming and information theory. With dynamic programming, we lift the design o…
KL-Divergence Guided Temperature Sampling
Chung-Ching Chang, David Reitter, Renat Aksitov +1
Temperature sampling is a conventional approach to diversify large language model predictions. As temperature increases, the prediction becomes diverse but also vulnerable to hallu…
CoLT5: Faster Long-Range Transformers with Conditional Computation
Joshua Ainslie, Tao Lei, Michiel de Jong +9
Many natural language processing tasks benefit from long inputs, but processing long documents with Transformers is expensive -- not only due to quadratic attention complexity but…
Characterizing Attribution and Fluency Tradeoffs for Retrieval-Augmented Large Language Models
Renat Aksitov, Chung-Ching Chang, David Reitter +2
Despite recent progress, it has been difficult to prevent semantic hallucinations in generative Large Language Models. One common solution to this is augmenting LLMs with a retriev…
Knowledge Prompts: Injecting World Knowledge into Language Models through Soft Prompts
Cicero Nogueira dos Santos, Zhe Dong, Daniel Cer +4
Soft prompts have been recently proposed as a tool for adapting large frozen language models (LMs) to new tasks. In this work, we repurpose soft prompts to the task of injecting wo…
LongT5: Efficient Text-To-Text Transformer for Long Sequences
Mandy Guo, Joshua Ainslie, David Uthus +4
Recent work has shown that either (1) increasing the input length or (2) increasing model size can improve the performance of Transformer-based neural models. In this paper, we pre…
Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision
Chao Jia, Yinfei Yang, Ye Xia +7
Pre-trained representations are becoming crucial for many NLP and perception tasks. While representation learning in NLP has transitioned to training on raw text without human anno…
Multilingual Universal Sentence Encoder for Semantic Retrieval
Yinfei Yang, Daniel Cer, Amin Ahmad +9
We introduce two pre-trained retrieval focused multilingual sentence encoding models, respectively based on the Transformer and CNN model architectures. The models embed text from…
Hierarchical Document Encoder for Parallel Corpus Mining
Mandy Guo, Yinfei Yang, Keith Stevens +5
We explore using multilingual document embeddings for nearest neighbor mining of parallel data. Three document-level representations are investigated: (i) document embeddings gener…
Improving Multilingual Sentence Embedding using Bi-directional Dual Encoder with Additive Margin Softmax
Yinfei Yang, Gustavo Hernandez Abrego, Steve Yuan +6
In this paper, we present an approach to learn multilingual sentence embeddings using a bi-directional dual-encoder with additive margin softmax. The embeddings are able to achieve…
Learning Cross-Lingual Sentence Representations via a Multi-task Dual-Encoder Model
Muthuraman Chidambaram, Yinfei Yang, Daniel Cer +4
A significant roadblock in multilingual neural language modeling is the lack of labeled non-English data. One potential method for overcoming this issue is learning cross-lingual t…
Effective Parallel Corpus Mining using Bilingual Sentence Embeddings
Mandy Guo, Qinlan Shen, Yinfei Yang +8
This paper presents an effective approach for parallel corpus mining using bilingual sentence embeddings. Our embedding models are trained to produce similar representations exclus…
Learning Semantic Textual Similarity from Conversations
Yinfei Yang, Steve Yuan, Daniel Cer +7
We present a novel approach to learn representations for sentence-level semantic similarity using conversational data. Our method trains an unsupervised model to predict conversati…
Universal Sentence Encoder
Daniel Cer, Yinfei Yang, Sheng-yi Kong +10
We present models for encoding sentences into embedding vectors that specifically target transfer learning to other NLP tasks. The models are efficient and result in accurate perfo…
Efficient Natural Language Response Suggestion for Smart Reply
Matthew Henderson, Rami Al-Rfou, Brian Strope +6
This paper presents a computationally efficient machine-learned method for natural language response suggestion. Feed-forward neural networks using n-gram embedding features encode…
Conversational Contextual Cues: The Case of Personalization and History for Response Ranking
Rami Al-Rfou, Marc Pickett, Javier Snaider +3
We investigate the task of modeling open-domain, multi-turn, unstructured, multi-participant, conversational dialogue. We specifically study the effect of incorporating different e…