117 citations · 367 across the 21 of their papers we have counts for
10 papers · 1 filter
Quick Dense Retrievers Consume KALE: Post Training Kullback Leibler Alignment of Embeddings for Asymmetrical dual encoders
Daniel Campos, Alessandro Magnani, ChengXiang Zhai
In this paper, we consider the problem of improving the inference latency of language model-based dense retrieval systems by introducing structural compression and model size asymm…
To Asymmetry and Beyond: Structured Pruning of Sequence to Sequence Models for Improved Inference Efficiency
Daniel Campos, ChengXiang Zhai
Sequence-to-sequence language models can be used to produce abstractive summaries which are coherent, relevant, and concise. Still, model sizes can make deployment in latency-sensi…
oBERTa: Improving Sparse Transfer Learning via improved initialization, distillation, and pruning regimes
Daniel Campos, Alexandre Marques, Mark Kurtz +1
In this paper, we introduce the range of oBERTa language models, an easy-to-use set of language models which allows Natural Language Processing (NLP) practitioners to obtain betwee…
Compressing Cross-Lingual Multi-Task Models at Qualtrics
Daniel Campos, Daniel Perry, Samir Joshi +4
Experience management is an emerging business area where organizations focus on understanding the feedback of customers and employees in order to improve their end-to-end experienc…
Sparse*BERT: Sparse Models Generalize To New tasks and Domains
Daniel Campos, Alexandre Marques, Tuan Nguyen +2
Large Language Models have become the core architecture upon which most modern natural language processing (NLP) systems build. These models can consistently deliver impressive acc…
The Optimal BERT Surgeon: Scalable and Accurate Second-Order Pruning for Large Language Models
Eldar Kurtic, Daniel Campos, Tuan Nguyen +5
Transformer-based language models have become a key building block for natural language processing. While these models are extremely accurate, they can be too large and computation…
Curriculum learning for language modeling
Daniel Campos
Language Models like ELMo and BERT have provided robust representations of natural language, which serve as the language understanding component for a diverse range of downstream t…
XGLUE: A New Benchmark Dataset for Cross-lingual Pre-training, Understanding and Generation
Yaobo Liang, Nan Duan, Yeyun Gong +21
In this paper, we introduce XGLUE, a new benchmark dataset that can be used to train large-scale cross-lingual pre-trained models using multilingual and bilingual corpora and evalu…
Open Domain Web Keyphrase Extraction Beyond Language Modeling
Lee Xiong, Chuan Hu, Chenyan Xiong +2
This paper studies keyphrase extraction in real-world scenarios where documents are from diverse domains and have variant content quality. We curate and release OpenKP, a large sca…
MS MARCO: A Human Generated MAchine Reading COmprehension Dataset
Payal Bajaj, Daniel Campos, Nick Craswell +12
We introduce a large scale MAchine Reading COmprehension dataset, which we name MS MARCO. The dataset comprises of 1,010,916 anonymized questions---sampled from Bing's search query…