37 citations · 40 across the 19 of their papers we have counts for
19 papers
The Stability of Singular Distribution: A Spectral Perspective on the Two-Phase Dynamics of Language Model Pre-training
Hongtao Zhang, Wenjie Zhou, Chenxi Jia +2
Large language model pre-training typically exhibits a two-phase trajectory: a fast initial loss drop followed by a prolonged slow improvement. We identify an underlying spectral p…
Extra-Merge: Tracing the Rank-1 Subspace of Model Merging in Language Model Pre-Training
Wenjie Zhou, Bohan Wang, Hongtao Zhang +3
Model merging has emerged as a lightweight paradigm for enhancing Large Language Models (LLMs), yet its underlying mechanisms remain poorly understood. In this work, we analyze lat…
Relevance Supervision Can Teach Neural Retrievers to Prefer LLM-Generated Text
Wei Huang, Keping Bi, Yinqiong Cai +3
Recent studies show that neural retrievers often exhibit source bias, ranking LLM-generated passages above semantically similar human-written ones, which threatens the reliability…
NAPPure: Adversarial Purification for Robust Image Classification under Non-Additive Perturbations
Junjie Nan, Jianing Li, Wei Chen +2
Adversarial purification has achieved great success in combating adversarial image perturbations, which are usually assumed to be additive. However, non-additive adversarial pertur…
How Do LLM-Generated Texts Impact Term-Based Retrieval Models?
Wei Huang, Keping Bi, Yinqiong Cai +3
As more content generated by large language models (LLMs) floods into the Internet, information retrieval (IR) systems now face the challenge of distinguishing and handling a blend…
RaPA: Enhancing Transferable Targeted Attacks via Random Parameter Pruning
Tongrui Su, Qingbin Li, Shengyu Zhu +2
Compared to untargeted attacks, targeted transfer-based attack is still suffering from much lower Attack Success Rates (ASRs), although significant improvements have been achieved…
CLIPure: Purification in Latent Space via CLIP for Adversarially Robust Zero-Shot Classification
Mingkun Zhang, Keping Bi, Wei Chen +2
In this paper, we aim to build an adversarially robust zero-shot image classifier. We ground our work on CLIP, a vision-language pre-trained encoder model that can perform zero-sho…
CausalDiff: Causality-Inspired Disentanglement via Diffusion Model for Adversarial Defense
Mingkun Zhang, Keping Bi, Wei Chen +3
Despite ongoing efforts to defend neural classifiers from adversarial attacks, they remain vulnerable, especially to unseen attacks. In contrast, humans are difficult to be cheated…
Generative Retrieval Meets Multi-Graded Relevance
Yubao Tang, Ruqing Zhang, Jiafeng Guo +3
Generative retrieval represents a novel approach to information retrieval. It uses an encoder-decoder architecture to directly produce relevant document identifiers (docids) for qu…
Classifier Guidance Enhances Diffusion-based Adversarial Purification by Preserving Predictive Information
Mingkun Zhang, Jianing Li, Wei Chen +2
Adversarial purification is one of the promising approaches to defend neural networks against adversarial attacks. Recently, methods utilizing diffusion probabilistic models have a…
Evaluating Implicit Bias in Large Language Models by Attacking From a Psychometric Perspective
Yuchen Wen, Keping Bi, Wei Chen +2
As large language models (LLMs) become an important way of information access, there have been increasing concerns that LLMs may intensify the spread of unethical content, includin…
Listwise Generative Retrieval Models via a Sequential Learning Process
Yubao Tang, Ruqing Zhang, Jiafeng Guo +3
Recently, a novel generative retrieval (GR) paradigm has been proposed, where a single sequence-to-sequence model is learned to directly generate a list of relevant document identi…
A Unified Causal View of Instruction Tuning
Lu Chen, Wei Huang, Ruqing Zhang +3
Instruction tuning on a mixture of tasks has improved zero-shot capabilities in natural language processing (NLP). Nevertheless, existing methods often learn features that exhibit…
Perturbation-Invariant Adversarial Training for Neural Ranking Models: Improving the Effectiveness-Robustness Trade-Off
Yu-An Liu, Ruqing Zhang, Mingkun Zhang +4
Neural ranking models (NRMs) have shown great success in information retrieval (IR). But their predictions can easily be manipulated using adversarial examples, which are crafted b…
CAME: Competitively Learning a Mixture-of-Experts Model for First-stage Retrieval
Yinqiong Cai, Yixing Fan, Keping Bi +4
The first-stage retrieval aims to retrieve a subset of candidate documents from a huge collection both effectively and efficiently. Since various matching patterns can exist betwee…
Inducing Causal Structure for Abstractive Text Summarization
Lu Chen, Ruqing Zhang, Wei Huang +3
The mainstream of data-driven abstractive summarization models tends to explore the correlations rather than the causal relationships. Among such correlations, there can be spuriou…
L^2R: Lifelong Learning for First-stage Retrieval with Backward-Compatible Representations
Yinqiong Cai, Keping Bi, Yixing Fan +3
First-stage retrieval is a critical task that aims to retrieve relevant document candidates from a large-scale collection. While existing retrieval models have achieved impressive…
Black-box Adversarial Attacks against Dense Retrieval Models: A Multi-view Contrastive Learning Method
Yu-An Liu, Ruqing Zhang, Jiafeng Guo +4
Neural ranking models (NRMs) and dense retrieval (DR) models have given rise to substantial improvements in overall retrieval performance. In addition to their effectiveness, and m…
Continual Learning for Generative Retrieval over Dynamic Corpora
Jiangui Chen, Ruqing Zhang, Jiafeng Guo +4
Generative retrieval (GR) directly predicts the identifiers of relevant documents (i.e., docids) based on a parametric model. It has achieved solid performance on many ad-hoc retri…