413 citations · 469 across the 9 of their papers we have counts for
9 papers
Approximating Human-Like Few-shot Learning with GPT-based Compression
Cynthia Huang, Yuqing Xie, Zhiying Jiang +2
In this work, we conceptualize the learning process as information compression. We seek to equip generative pre-trained models with human-like learning capabilities that enable dat…
Latte-Mix: Measuring Sentence Semantic Similarity with Latent Categorical Mixtures
M. Li, H. Bai, L. Tan +2
Measuring sentence semantic similarity using pre-trained language models such as BERT generally yields unsatisfactory zero-shot performance, and one main reason is ineffective toke…
Don't Change Me! User-Controllable Selective Paraphrase Generation
Mohan Zhang, Luchen Tan, Zhengkai Tu +4
In the paraphrase generation task, source sentences often contain phrases that should not be altered. Which phrases, however, can be context dependent and can vary by application.…
Segatron: Segment-Aware Transformer for Language Modeling and Understanding
He Bai, Peng Shi, Jimmy Lin +5
Transformers are powerful for sequence modeling. Nearly all state-of-the-art language models and pre-trained language models are based on the Transformer architecture. However, it…
Semantics of the Unwritten: The Effect of End of Paragraph and Sequence Tokens on Text Generation with GPT2
He Bai, Peng Shi, Jimmy Lin +5
The semantics of a text is manifested not only by what is read, but also by what is not read. In this article, we will study how the implicit "not read" information such as end-of-…
Two Birds, One Stone: A Simple, Unified Model for Text Generation from Structured and Unstructured Data
Hamidreza Shahidi, Ming Li, Jimmy Lin
A number of researchers have recently questioned the necessity of increasingly complex neural network (NN) architectures. In particular, several recent papers have shown that simpl…