708 citations · 974 across the 9 of their papers we have counts for
11 papers
Transcending Scaling Laws with 0.1% Extra Compute
Yi Tay, Jason Wei, Hyung Won Chung +13
Scaling language models improves performance but comes with significant computational costs. This paper proposes UL2R, a method that substantially improves existing language models…
UL2: Unifying Language Learning Paradigms
Yi Tay, Mostafa Dehghani, Vinh Q. Tran +11
Existing pre-trained models are generally geared towards a particular class of problems. To date, there seems to be still no consensus on what the right architecture and pre-traini…
HyperPrompt: Prompt-based Task-Conditioning of Transformers
Yun He, Huaixiu Steven Zheng, Yi Tay +9
Prompt-Tuning is a new paradigm for finetuning pre-trained language models in a parameter-efficient way. Here, we explore the use of HyperNetworks to generate hyper-prompts: we pro…
LaMDA: Language Models for Dialog Applications
Romal Thoppilan, Daniel De Freitas, Jamie Hall +57
We present LaMDA: Language Models for Dialog Applications. LaMDA is a family of Transformer-based neural language models specialized for dialog, which have up to 137B parameters an…
ExT5: Towards Extreme Multi-Task Scaling for Transfer Learning
Vamsi Aribandi, Yi Tay, Tal Schuster +11
Despite the recent success of multi-task learning and transfer learning for natural language processing (NLP), few works have systematically studied the effect of scaling up the nu…
Joint Contextual Modeling for ASR Correction and Language Understanding
Yue Weng, Sai Sumanth Miryala, Chandra Khatri +8
The quality of automatic speech recognition (ASR) is critical to Dialogue Systems as ASR errors propagate to and directly impact downstream tasks such as language understanding (LU…
Exploration Based Language Learning for Text-Based Games
Andrea Madotto, Mahdi Namazifar, Joost Huizinga +7
This work presents an exploration and imitation-learning-based agent capable of state-of-the-art performance in playing text-based computer games. Text-based computer games describ…
Flexibly-Structured Model for Task-Oriented Dialogues
Lei Shu, Piero Molino, Mahdi Namazifar +4
This paper proposes a novel end-to-end architecture for task-oriented dialogue systems. It is based on a simple and practical yet very effective sequence-to-sequence approach, wher…
OCC: A Smart Reply System for Efficient In-App Communications
Yue Weng, Huaixiu Zheng, Franziska Bell +1
Smart reply systems have been developed for various messaging platforms. In this paper, we introduce Uber's smart reply system: one-click-chat (OCC), which is a key enhanced featur…
COTA: Improving the Speed and Accuracy of Customer Support through Ranking and Deep Networks
Piero Molino, Huaixiu Zheng, Yi-Chia Wang
For a company looking to provide delightful user experiences, it is of paramount importance to take care of any customer issues. This paper proposes COTA, a system to improve speed…
Prostate Cancer Diagnosis using Deep Learning with 3D Multiparametric MRI
Saifeng Liu, Huaixiu Zheng, Yesu Feng +1
A novel deep learning architecture (XmasNet) based on convolutional neural networks was developed for the classification of prostate cancer lesions, using the 3D multiparametric MR…