35 citations · 139 across the 19 of their papers we have counts for
19 papers
PRISM Edit: One Vector for All Temporal Answers
Chen Huang, Qi Zheng, Ruiqin Zheng +2
Model editing keeps large language models (LLMs) up to date without retraining, but temporal facts expose a limitation of the prevailing locate-and-edit paradigm: an update is not…
Separate the Wheat from the Chaff: Winnowing Down Divergent Views in Retrieval Augmented Generation
Song Wang, Zihan Chen, Peng Wang +5
Retrieval-augmented generation (RAG) enhances large language models (LLMs) by integrating external knowledge sources to address their limitations in accessing up-to-date or special…
Configurable Foundation Models: Building LLMs from a Modular Perspective
Chaojun Xiao, Zhengyan Zhang, Chenyang Song +20
Advancements in LLMs have recently unveiled challenges tied to computational efficiency and continual scalability due to their requirements of huge parameters, making the applicati…
Certifiably Robust RAG against Retrieval Corruption
Chong Xiang, Tong Wu, Zexuan Zhong +3
Retrieval-augmented generation (RAG) is susceptible to retrieval corruption attacks, where malicious passages injected into retrieval results can lead to inaccurate model responses…
Lory: Fully Differentiable Mixture-of-Experts for Autoregressive Language Model Pre-training
Zexuan Zhong, Mengzhou Xia, Danqi Chen +1
Mixture-of-experts (MoE) models facilitate efficient scaling; however, training the router network introduces the challenge of optimizing a non-differentiable, discrete objective.…
Reliable, Adaptable, and Attributable Language Models with Retrieval
Akari Asai, Zexuan Zhong, Danqi Chen +4
Parametric language models (LMs), which are trained on vast amounts of web data, exhibit remarkable flexibility and capability. However, they still face practical challenges such a…
REST: Retrieval-Based Speculative Decoding
Zhenyu He, Zexuan Zhong, Tianle Cai +2
We introduce Retrieval-Based Speculative Decoding (REST), a novel algorithm designed to speed up language model generation. The key insight driving the development of REST is the o…
Poisoning Retrieval Corpora by Injecting Adversarial Passages
Zexuan Zhong, Ziqing Huang, Alexander Wettig +1
Dense retrievers have achieved state-of-the-art performance in various information retrieval tasks, but to what extent can they be safely deployed in real-world applications? In th…
Privacy Implications of Retrieval-Based Language Models
Yangsibo Huang, Samyak Gupta, Zexuan Zhong +2
Retrieval-based language models (LMs) have demonstrated improved interpretability, factuality, and adaptability compared to their parametric counterparts, by incorporating retrieve…
MQuAKE: Assessing Knowledge Editing in Language Models via Multi-Hop Questions
Zexuan Zhong, Zhengxuan Wu, Christopher D. Manning +2
The information stored in large language models (LLMs) falls out of date quickly, and retraining from scratch is often not an option. This has recently given rise to a range of tec…
NN-Adapter: Efficient Domain Adaptation for Black-Box Language Models
Yangsibo Huang, Daogao Liu, Zexuan Zhong +2
Fine-tuning a language model on a new domain is standard practice for domain adaptation. However, it can be infeasible when it comes to modern large-scale language models such as G…
Structured Pruning Learns Compact and Accurate Models
Mengzhou Xia, Zexuan Zhong, Danqi Chen
The growing size of neural language models has led to increased attention in model compression. The two predominant approaches are pruning, which gradually removes weights from a p…
Training Language Models with Memory Augmentation
Zexuan Zhong, Tao Lei, Danqi Chen
Recent work has improved language models (LMs) remarkably by equipping them with a non-parametric memory component. However, most existing approaches only introduce mem-ories at te…
Recovering Private Text in Federated Learning of Language Models
Samyak Gupta, Yangsibo Huang, Zexuan Zhong +3
Federated learning allows distributed users to collaboratively train a model while keeping each user's data private. Recently, a growing body of work has demonstrated that an eaves…
Should You Mask 15% in Masked Language Modeling?
Alexander Wettig, Tianyu Gao, Zexuan Zhong +1
Masked language models (MLMs) conventionally mask 15% of tokens due to the belief that more masking would leave insufficient context to learn good representations; this masking rat…
Simple Entity-Centric Questions Challenge Dense Retrievers
Christopher Sciavolino, Zexuan Zhong, Jinhyuk Lee +1
Open-domain question answering has exploded in popularity recently due to the success of dense retrieval models, which have surpassed sparse models using only a few supervised trai…
Factual Probing Is [MASK]: Learning vs. Learning to Recall
Zexuan Zhong, Dan Friedman, Danqi Chen
Petroni et al. (2019) demonstrated that it is possible to retrieve world facts from a pre-trained language model by expressing them as cloze-style prompts and interpret the model's…
A Frustratingly Easy Approach for Entity and Relation Extraction
Zexuan Zhong, Danqi Chen
End-to-end relation extraction aims to identify named entities and extract relations between them. Most recent work models these two subtasks jointly, either by casting them in one…
MULDEF: Multi-model-based Defense Against Adversarial Examples for Neural Networks
Siwakorn Srisakaokul, Yuhao Zhang, Zexuan Zhong +3
Despite being popularly used in many applications, neural network models have been found to be vulnerable to adversarial examples, i.e., carefully crafted examples aiming to mislea…