1.5k citations · 1.7k across the 4 of their papers we have counts for
11 papers
Text and Code Embeddings by Contrastive Pre-Training
Arvind Neelakantan, Tao Xu, Raul Puri +22
Text embeddings are useful features in many applications such as semantic search and computing text similarity. Previous work typically trains models customized for different use c…
Evaluating Large Language Models Trained on Code
Mark Chen, Jerry Tworek, Heewoo Jun +55
We introduce Codex, a GPT language model fine-tuned on publicly available code from GitHub, and study its Python code-writing capabilities. A distinct production version of Codex p…
Local Knowledge Powered Conversational Agents
Sashank Santhanam, Wei Ping, Raul Puri +3
State-of-the-art conversational agents have advanced significantly in conjunction with the use of large transformer-based language models. However, even with these advancements, co…
BioMegatron: Larger Biomedical Domain Language Model
Hoo-Chang Shin, Yang Zhang, Evelina Bakhturina +4
There has been an influx of biomedical domain-specific language models, showing language models pre-trained on biomedical text perform better on biomedical domain benchmarks than t…
MEGATRON-CNTRL: Controllable Story Generation with External Knowledge Using Large-Scale Language Models
Peng Xu, Mostofa Patwary, Mohammad Shoeybi +4
Existing pre-trained large language models have shown unparalleled generative capabilities. However, they are not controllable. In this paper, we propose MEGATRON-CNTRL, a novel fr…
Large Scale Multi-Actor Generative Dialog Modeling
Alex Boyd, Raul Puri, Mohammad Shoeybi +2
Non-goal oriented dialog agents (i.e. chatbots) aim to produce varying and engaging conversations with a user; however, they typically exhibit either inconsistent personality acros…