Publications (61)
Unleashing the Power of LLMs in Dense Retrieval with Query Likelihood Modeling
Hengran Zhang, Keping Bi, Jiafeng Guo +5
Dense retrieval is a crucial task in Information Retrieval (IR), serving as the basis for downstream tasks such as re-ranking and augmenting generation. Recently, large language mo…
Bridging Queries and Tables through Entities in Table Retrieval
Da Li, Keping Bi, Jiafeng Guo +1
Table retrieval is essential for accessing information stored in structured tabular formats; however, it remains less explored than text retrieval. The content of the table primari…
Explainable Product Search with a Dynamic Relation Embedding Model
Qingyao Ai, Yongfeng Zhang, Keping Bi +1
Product search is one of the most popular methods for customers to discover products online. Most existing studies on product search focus on developing effective retrieval models…
Distilling a Small Utility-Based Passage Selector to Enhance Retrieval-Augmented Generation
Hengran Zhang, Keping Bi, Jiafeng Guo +4
Retrieval-augmented generation (RAG) enhances large language models (LLMs) by incorporating retrieved information. Standard retrieval process prioritized relevance, focusing on top…
Revisiting Iterative Relevance Feedback for Document and Passage Retrieval
Keping Bi, Qingyao Ai, W. Bruce Croft
As more and more search traffic comes from mobile phones, intelligent assistants, and smart-home devices, new challenges (e.g., limited presentation space) and opportunities come u…
Can LLM Rerankers Predict Their Own Ranking Performance?
Shiyu Ni, Keping Bi, Jiafeng Guo +3
Retrieval effectiveness varies substantially across queries, making it important to estimate ranking quality before relevance judgments are available. Query performance prediction…
How Knowledge Popularity Influences and Enhances LLM Knowledge Boundary Perception
Shiyu Ni, Keping Bi, Jiafeng Guo +1
Large language models (LLMs) often fail to recognize their knowledge boundaries, producing confident yet incorrect answers. In this paper, we investigate how knowledge popularity a…
AuthorityBench: Benchmarking LLM Authority Perception for Reliable Retrieval-Augmented Generation
Zhihui Yao, Hengran Zhang, Keping Bi
Retrieval-Augmented Generation (RAG) enhances Large Language Models (LLMs) with external knowledge but remains vulnerable to low-authority sources that can propagate misinformation…
Unbiased Learning to Rank with Unbiased Propensity Estimation
Qingyao Ai, Keping Bi, Cheng Luo +2
Learning to rank with biased click data is a well-known challenge. A variety of methods has been explored to debias click data for learning to rank such as click models, result int…
Injecting External Knowledge into the Reasoning Process Enhances Retrieval-Augmented Generation
Minghao Tang, Shiyu Ni, Jiafeng Guo +1
Retrieval-augmented generation (RAG) has been widely adopted to augment large language models (LLMs) with external knowledge for knowledge-intensive tasks. However, its effectivene…
Unbiased Learning to Rank with Query-Level Click Propensity Estimation: Beyond Pointwise Observation and Relevance
Lulu Yu, Keping Bi, Jiafeng Guo +3
Most existing unbiased learning-to-rank (ULTR) approaches are based on the user examination hypothesis, which assumes that users will click a result only if it is both relevant and…
Asking Clarifying Questions Based on Negative Feedback in Conversational Search
Keping Bi, Qingyao Ai, W. Bruce Croft
Users often need to look through multiple search result pages or reformulate queries when they have complex information-seeking needs. Conversational search systems make it possibl…
Iterative Relevance Feedback for Answer Passage Retrieval with Passage-level Semantic Match
Keping Bi, Qingyao Ai, W. Bruce Croft
Relevance feedback techniques assume that users provide relevance judgments for the top k (usually 10) documents and then re-rank using a new query model based on those judgments.…
Learning a Deep Listwise Context Model for Ranking Refinement
Qingyao Ai, Keping Bi, Jiafeng Guo +1
Learning to rank has been intensively studied and widely applied in information retrieval. Typically, a global ranking function is learned from a set of labeled data, which can ach…
An Iterative Utility Judgment Framework Inspired by Philosophical Relevance via LLMs
Hengran Zhang, Keping Bi, Jiafeng Guo +1
Relevance and utility are two frequently used measures to evaluate the effectiveness of an information retrieval (IR) system. Relevance emphasizes the aboutness of a result to a qu…
L^2R: Lifelong Learning for First-stage Retrieval with Backward-Compatible Representations
Yinqiong Cai, Keping Bi, Yixing Fan +3
First-stage retrieval is a critical task that aims to retrieve relevant document candidates from a large-scale collection. While existing retrieval models have achieved impressive…
CAME: Competitively Learning a Mixture-of-Experts Model for First-stage Retrieval
Yinqiong Cai, Yixing Fan, Keping Bi +4
The first-stage retrieval aims to retrieve a subset of candidate documents from a huge collection both effectively and efficiently. Since various matching patterns can exist betwee…
Ensemble Ranking Model with Multiple Pretraining Strategies for Web Search
Xiaojie Sun, Lulu Yu, Yiting Wang +2
An effective ranking model usually requires a large amount of training data to learn the relevance between documents and queries. User clicks are often used as training data since…
Reconstructing Content with Collaborative Attention for Universal Multimodal Representation Learning
Jiahan Chen, Da Li, Hengran Zhang +6
Multimodal embedding models, rooted in multimodal large language models (MLLMs), have yielded significant performance improvements across diverse tasks such as retrieval and classi…
A Comparative Study of Specialized LLMs as Dense Retrievers
Hengran Zhang, Keping Bi, Jiafeng Guo
While large language models (LLMs) are increasingly deployed as dense retrievers, the impact of their domain-specific specialization on retrieval effectiveness remains underexplore…
Can LLM Annotations Replace User Clicks for Learning to Rank?
Lulu Yu, Keping Bi, Jiafeng Guo +4
Large-scale supervised data is essential for training modern ranking models, but obtaining high-quality human annotations is costly. Click data has been widely used as a low-cost a…
LINKAGE: Listwise Ranking among Varied-Quality References for Non-Factoid QA Evaluation via LLMs
Sihui Yang, Keping Bi, Wanqing Cui +2
Non-Factoid (NF) Question Answering (QA) is challenging to evaluate due to diverse potential answers and no objective criterion. The commonly used automatic evaluation metrics like…
AREDSUM: Adaptive Redundancy-Aware Iterative Sentence Ranking for Extractive Document Summarization
Keping Bi, Rahul Jha, W. Bruce Croft +1
Redundancy-aware extractive summarization systems score the redundancy of the sentences to be included in a summary either jointly with their salience information or separately as…
CIR at the NTCIR-17 ULTRE-2 Task
Lulu Yu, Keping Bi, Jiafeng Guo +1
The Chinese academy of sciences Information Retrieval team (CIR) has participated in the NTCIR-17 ULTRE-2 task. This paper describes our approaches and reports our results on the U…
Beyond Relevance: Utility-Centric Retrieval in the LLM Era
Hengran Zhang, Minghao Tang, Keping Bi +1
Information retrieval systems have traditionally optimized for topical relevance-the degree to which retrieved documents match a query. However, relevance only approximates a deepe…
LifeIR at the NTCIR-18 Lifelog-6 Task
Jiahan Chen, Da Li, Keping Bi
In recent years, sharing lifelogs recorded through wearable devices such as sports watches and GoPros, has gained significant popularity. Lifelogs involve various types of informat…
Are Large Language Models More Honest in Their Probabilistic or Verbalized Confidence?
Shiyu Ni, Keping Bi, Lulu Yu +1
Large language models (LLMs) have been found to produce hallucinations when the question exceeds their internal knowledge boundaries. A reliable model should have a clear perceptio…
A Transformer-based Embedding Model for Personalized Product Search
Keping Bi, Qingyao Ai, W. Bruce Croft
Product search is an important way for people to browse and purchase items on E-commerce platforms. While customers tend to make choices based on their personal tastes and preferen…
LLM-Specific Utility: A New Perspective for Retrieval-Augmented Generation
Hengran Zhang, Keping Bi, Jiafeng Guo +4
Retrieval-augmented generation (RAG) is typically optimized for topical relevance, yet its success ultimately depends on whether retrieved passages are useful for a large language…
Towards Fully Exploiting LLM Internal States to Enhance Knowledge Boundary Perception
Shiyu Ni, Keping Bi, Jiafeng Guo +3
Large language models (LLMs) exhibit impressive performance across diverse tasks but often struggle to accurately gauge their knowledge boundaries, leading to confident yet incorre…
A Study of Context Dependencies in Multi-page Product Search
Keping Bi, Choon Hui Teo, Yesh Dattatreya +2
In product search, users tend to browse results on multiple search result pages (SERPs) (e.g., for queries on clothing and shoes) before deciding which item to purchase. Users' cli…
Annotation-Efficient Universal Honesty Alignment
Shiyu Ni, Keping Bi, Jiafeng Guo +4
Honesty alignment-the ability of large language models (LLMs) to recognize their knowledge boundaries and express calibrated confidence-is essential for trustworthy deployment. Exi…
Leveraging User Behavior History for Personalized Email Search
Keping Bi, Pavel Metrikov, Chunyuan Li +1
An effective email search engine can facilitate users' search tasks and improve their communication efficiency. Users could have varied preferences on various ranking signals of an…
How Long Reasoning Chains Influence LLMs' Judgment of Answer Factuality
Minzhu Tu, Shiyu Ni, Keping Bi
Large language models (LLMs) has been widely adopted as a scalable surrogate for human evaluation, yet such judges remain imperfect and susceptible to surface-level biases. One pos…
Do LVLMs Know What They Know? A Systematic Study of Knowledge Boundary Perception in LVLMs
Zhikai Ding, Shiyu Ni, Keping Bi
Large vision-language models (LVLMs) demonstrate strong visual question answering (VQA) capabilities but are shown to hallucinate. A reliable model should perceive its knowledge bo…
MORE: Multi-mOdal REtrieval Augmented Generative Commonsense Reasoning
Wanqing Cui, Keping Bi, Jiafeng Guo +1
Since commonsense information has been recorded significantly less frequently than its existence, language models pre-trained by text generation have difficulty to learn sufficient…
Evaluating and Calibrating LLM Confidence on Questions with Multiple Correct Answers
Yuhan Wang, Shiyu Ni, Zhikai Ding +3
Confidence calibration is essential for making large language models (LLMs) reliable, yet existing training-free methods have been primarily studied under single-answer question an…
Feature-Enhanced Network with Hybrid Debiasing Strategies for Unbiased Learning to Rank
Lulu Yu, Yiting Wang, Xiaojie Sun +2
Unbiased learning to rank (ULTR) aims to mitigate various biases existing in user clicks, such as position bias, trust bias, presentation bias, and learn an effective ranker. In th…
Tailoring Table Retrieval from a Field-aware Hybrid Matching Perspective
Da Li, Keping Bi, Jiafeng Guo +1
Table retrieval, essential for accessing information through tabular data, is less explored compared to text retrieval. The row/column structure and distinct fields of tables (incl…
A Comparative Study of Training Objectives for Clarification Facet Generation
Shiyu Ni, Keping Bi, Jiafeng Guo +1
Due to the ambiguity and vagueness of a user query, it is essential to identify the query facets for the clarification of user intents. Existing work on query facet generation has…
Evaluating Implicit Bias in Large Language Models by Attacking From a Psychometric Perspective
Yuchen Wen, Keping Bi, Wei Chen +2
As large language models (LLMs) become an important way of information access, there have been increasing concerns that LLMs may intensify the spread of unethical content, includin…
Reproducibility Analysis and Enhancements for Multi-Aspect Dense Retriever with Aspect Learning
Keping Bi, Xiaojie Sun, Jiafeng Guo +1
Multi-aspect dense retrieval aims to incorporate aspect information (e.g., brand and category) into dual encoders to facilitate relevance matching. As an early and representative m…
Compressing then Matching: An Efficient Pre-training Paradigm for Multimodal Embedding
Da Li, Yuxiao Luo, Keping Bi +7
Multimodal Large Language Models advance multimodal representation learning by acquiring transferable semantic embeddings, thereby substantially enhancing performance across a rang…
When Do LLMs Need Retrieval Augmentation? Mitigating LLMs' Overconfidence Helps Retrieval Augmentation
Shiyu Ni, Keping Bi, Jiafeng Guo +1
Large Language Models (LLMs) have been found to have difficulty knowing they do not possess certain knowledge and tend to provide specious answers in such cases. Retrieval Augmenta…
Understanding Parametric Knowledge Injection in Retrieval-Augmented Generation
Minghao Tang, Shiyu Ni, Jingtong Wu +2
Context-grounded generation underpins many LLM applications, including long-document question answering (QA), conversational personalization, and retrieval-augmented generation (RA…
Leverage Implicit Feedback for Context-aware Product Search
Keping Bi, Choon Hui Teo, Yesh Dattatreya +2
Product search serves as an important entry point for online shopping. In contrast to web search, the retrieved results in product search not only need to be relevant but also shou…
CLIPure: Purification in Latent Space via CLIP for Adversarially Robust Zero-Shot Classification
Mingkun Zhang, Keping Bi, Wei Chen +2
In this paper, we aim to build an adversarially robust zero-shot image classifier. We ground our work on CLIP, a vision-language pre-trained encoder model that can perform zero-sho…
Data, Not Model: Explaining Bias toward LLM Texts in Neural Retrievers
Wei Huang, Keping Bi, Yinqiong Cai +3
Recent studies show that neural retrievers often display source bias, favoring passages generated by LLMs over human-written ones, even when both are semantically similar. This bia…
A Multi-Granularity-Aware Aspect Learning Model for Multi-Aspect Dense Retrieval
Xiaojie Sun, Keping Bi, Jiafeng Guo +5
Dense retrieval methods have been mostly focused on unstructured text and less attention has been drawn to structured data with various aspects, e.g., products with aspects such as…
Utility-Focused LLM Annotation for Retrieval and Retrieval-Augmented Generation
Hengran Zhang, Minghao Tang, Keping Bi +5
This paper explores the use of large language models (LLMs) for annotating document utility in training retrieval and retrieval-augmented generation (RAG) systems, aiming to reduce…
Bagging-Based Model Merging for Robust General Text Embeddings
Hengran Zhang, Keping Bi, Jiafeng Guo +4
General-purpose text embedding models underpin a wide range of NLP and information retrieval applications, and are typically trained on large-scale multi-task corpora to encourage…
CausalDiff: Causality-Inspired Disentanglement via Diffusion Model for Adversarial Defense
Mingkun Zhang, Keping Bi, Wei Chen +3
Despite ongoing efforts to defend neural classifiers from adversarial attacks, they remain vulnerable, especially to unseen attacks. In contrast, humans are difficult to be cheated…
Contextual Dual Learning Algorithm with Listwise Distillation for Unbiased Learning to Rank
Lulu Yu, Keping Bi, Shiyu Ni +1
Unbiased Learning to Rank (ULTR) aims to leverage biased implicit user feedback (e.g., click) to optimize an unbiased ranking model. The effectiveness of the existing ULTR methods…
Conversational Product Search Based on Negative Feedback
Keping Bi, Qingyao Ai, Yongfeng Zhang +1
Intelligent assistants change the way people interact with computers and make it possible for people to search for products through conversations when they have purchase needs. Dur…
Attention Grounded Enhancement for Visual Document Retrieval
Wanqing Cui, Wei Huang, Yazhi Guo +4
Visual document retrieval requires understanding heterogeneous and multi-modal content to satisfy implicit information needs. Recent advances use screenshot-based document encoding…
How Do LLM-Generated Texts Impact Term-Based Retrieval Models?
Wei Huang, Keping Bi, Yinqiong Cai +3
As more content generated by large language models (LLMs) floods into the Internet, information retrieval (IR) systems now face the challenge of distinguishing and handling a blend…
Artemis: A Novel Annotation Methodology for Indicative Single Document Summarization
Rahul Jha, Keping Bi, Yang Li +4
We describe Artemis (Annotation methodology for Rich, Tractable, Extractive, Multi-domain, Indicative Summarization), a novel hierarchical annotation process that produces indicati…
Training Dense Retrievers with Multiple Positive Passages
Benben Wang, Minghao Tang, Hengran Zhang +2
Modern knowledge-intensive systems, such as retrieval-augmented generation (RAG), rely on effective retrievers to establish the performance ceiling for downstream modules. However,…
Pre-training with Aspect-Content Text Mutual Prediction for Multi-Aspect Dense Retrieval
Xiaojie Sun, Keping Bi, Jiafeng Guo +5
Grounded on pre-trained language models (PLMs), dense retrieval has been studied extensively on plain text. In contrast, there has been little research on retrieving data with mult…
Learning a Fine-Grained Review-based Transformer Model for Personalized Product Search
Keping Bi, Qingyao Ai, W. Bruce Croft
Product search has been a crucial entry point to serve people shopping online. Most existing personalized product models follow the paradigm of representing and matching user inten…
Estimating Commonsense Plausibility through Semantic Shifts
Wanqing Cui, Wei Huang, Keping Bi +2
Commonsense plausibility estimation is critical for evaluating language models (LMs), yet existing generative approaches--reliant on likelihoods or verbalized judgments--struggle w…