84 citations · 278 across the 14 of their papers we have counts for
14 papers
SEMQA: Semi-Extractive Multi-Source Question Answering
Tal Schuster, Adam D. Lelkes, Haitian Sun +4
Recently proposed long-form question answering (QA) systems, supported by large language models (LLMs), have shown promising capabilities. Yet, attributing and verifying their gene…
How Does Generative Retrieval Scale to Millions of Passages?
Ronak Pradeep, Kai Hui, Jai Gupta +5
Popularized by the Differentiable Search Index, the emerging paradigm of generative retrieval re-frames the classic information retrieval problem into a sequence-to-sequence modeli…
Dense Feature Memory Augmented Transformers for COVID-19 Vaccination Search Classification
Jai Gupta, Yi Tay, Chaitanya Kamath +5
With the devastating outbreak of COVID-19, vaccines are one of the crucial lines of defense against mass infection in this global pandemic. Given the protection they provide, vacci…
DSI++: Updating Transformer Memory with New Documents
Sanket Vaibhav Mehta, Jai Gupta, Yi Tay +6
Differentiable Search Indices (DSIs) encode a corpus of documents in model parameters and use the same model to answer user queries directly. Despite the strong performance of DSI…
Confident Adaptive Language Modeling
Tal Schuster, Adam Fisch, Jai Gupta +5
Recent advances in Transformer-based large language models (LLMs) have led to significant performance improvements across many tasks. These gains come with a drastic increase in th…
ED2LM: Encoder-Decoder to Language Model for Faster Document Re-ranking Inference
Kai Hui, Honglei Zhuang, Tao Chen +8
State-of-the-art neural models typically encode document-query pairs using cross-attention for re-ranking. To this end, models generally utilize an encoder-only (like BERT) paradig…
HyperPrompt: Prompt-based Task-Conditioning of Transformers
Yun He, Huaixiu Steven Zheng, Yi Tay +9
Prompt-Tuning is a new paradigm for finetuning pre-trained language models in a parameter-efficient way. Here, we explore the use of HyperNetworks to generate hyper-prompts: we pro…
A New Generation of Perspective API: Efficient Multilingual Character-level Transformers
Alyssa Lees, Vinh Q. Tran, Yi Tay +4
On the world wide web, toxic content detectors are a crucial line of defense against potentially hateful and offensive messages. As such, building highly effective classifiers that…
Transformer Memory as a Differentiable Search Index
Yi Tay, Vinh Q. Tran, Mostafa Dehghani +10
In this paper, we demonstrate that information retrieval can be accomplished with a single Transformer, in which all information about the corpus is encoded in the parameters of th…
Vaccine Search Patterns Provide Insights into Vaccination Intent
Sean Malahy, Mimi Sun, Keith Spangler +20
Despite ample supply of COVID-19 vaccines, the proportion of fully vaccinated individuals remains suboptimal across much of the US. Rapid vaccination of additional people will prev…
ExT5: Towards Extreme Multi-Task Scaling for Transfer Learning
Vamsi Aribandi, Yi Tay, Tal Schuster +11
Despite the recent success of multi-task learning and transfer learning for natural language processing (NLP), few works have systematically studied the effect of scaling up the nu…
Google COVID-19 Vaccination Search Insights: Anonymization Process Description
Shailesh Bavadekar, Adam Boulanger, John Davis +22
This report describes the aggregation and anonymization process applied to the COVID-19 Vaccination Search Insights (published at http://goo.gle/covid19vaccinationinsights), a publ…
Charformer: Fast Character Transformers via Gradient-based Subword Tokenization
Yi Tay, Vinh Q. Tran, Sebastian Ruder +7
State-of-the-art models in natural language processing rely on separate rigid subword tokenization algorithms, which limit their generalization ability and adaptation to new settin…
OmniNet: Omnidirectional Representations from Transformers
Yi Tay, Mostafa Dehghani, Vamsi Aribandi +6
This paper proposes Omnidirectional Representations from Transformers (OmniNet). In OmniNet, instead of maintaining a strictly horizontal receptive field, each token is allowed to…