506 citations · 1.8k across the 7 of their papers we have counts for
8 papers
PaLI: A Jointly-Scaled Multilingual Language-Image Model
Xi Chen, Xiao Wang, Soravit Changpinyo +26
Effective scaling and a flexible task interface enable large language models to excel at many tasks. We present PaLI (Pathways Language and Image model), a model that extends this…
A Flexible Approach to Automated RNN Architecture Generation
Martin Schrimpf, Stephen Merity, James Bradbury +1
The process of designing neural architectures requires expert knowledge and extensive trial and error. While automated architecture search may simplify these requirements, the recu…
Block-diagonal Hessian-free Optimization for Training Neural Networks
Huishuai Zhang, Caiming Xiong, James Bradbury +1
Second-order methods for neural network optimization have several advantages over methods based on first-order gradient descent, including better scaling to large mini-batch sizes…
Non-Autoregressive Neural Machine Translation
Jiatao Gu, James Bradbury, Caiming Xiong +2
Existing approaches to neural machine translation condition each output word on previously generated outputs. We introduce a model that avoids this autoregressive property and prod…
Towards Neural Machine Translation with Latent Tree Attention
James Bradbury, Richard Socher
Building models that take advantage of the hierarchical structure of language without a priori annotation is a longstanding goal in natural language processing. We introduce such a…
Learned in Translation: Contextualized Word Vectors
Bryan McCann, James Bradbury, Caiming Xiong +1
Computer vision has benefited from initializing multiple deep layers with weights pretrained on large supervised training sets like ImageNet. Natural language processing (NLP) typi…
Quasi-Recurrent Neural Networks
James Bradbury, Stephen Merity, Caiming Xiong +1
Recurrent neural networks are a powerful tool for modeling sequential data, but the dependence of each timestep's computation on the previous timestep's output limits parallelism a…
Pointer Sentinel Mixture Models
Stephen Merity, Caiming Xiong, James Bradbury +1
Recent neural network sequence models with softmax classifiers have achieved their best language modeling performance only with very large hidden states and large vocabularies. Eve…