Showing 2023Show all
2 papers · 1 filter
cs.CL2023
Summarization-Based Document IDs for Generative Retrieval with Language Models
Haoxin Li, Daniel Cheng, Phillip Keung +2
Generative retrieval (Wang et al., 2022; Tay et al., 2022) is a popular approach for end-to-end document retrieval that directly generates document identifiers given an input query…
cs.CL2023
NarrowBERT: Accelerating Masked Language Model Pretraining and Inference
Haoxin Li, Phillip Keung, Daniel Cheng +2
Large-scale language model pretraining is a very successful form of self-supervised learning in natural language processing, but it is increasingly expensive to perform as the mode…