29 citations · 97 across the 21 of their papers we have counts for
21 papers
Search-Time Contamination in Deep Research Agents: Measuring Performance Inflation in Public Benchmark Evaluation
Yongjie Wang, Xinyue Zhang, Kunhong Yao +4
Public benchmarks enable fair and reproducible evaluation of LLM reasoning, but they become fragile for deep research agents that actively search the web during inference. Such age…
LERA: LLM-Enhanced RAG for Ad Auction in Generative Chatbots
Haoran Sun, Xinrui Song, Xinyu Zhang +7
The integration of advertising auction mechanisms into large language model (LLM)-based chatbots presents a significant opportunity for commercialization, yet poses unique challeng…
Beyond Position Bias: Shifting Context Compression from Position-Driven to Semantic-Driven
Jiwei Tang, Zhijing Huang, Xinyu Zhang +5
Large Language Models (LLMs) have demonstrated exceptional performance across diverse tasks. However, their deployment in long-context scenarios faces high computational overhead a…
Tomato, Tomahto, Tomate: Do Multilingual Language Models Understand Based on Subword-Level Semantic Concepts?
Crystina Zhang, Jing Lu, Vinh Q. Tran +3
Human understanding of text depends on general semantic concepts of words rather than their superficial forms. To what extent does our human intuition transfer to language models?…
Words Worth a Thousand Pictures: Measuring and Understanding Perceptual Variability in Text-to-Image Generation
Raphael Tang, Xinyu Zhang, Lixinyu Xu +5
Diffusion models are the state of the art in text-to-image generation, but their perceptual variability remains understudied. In this paper, we examine how prompts affect image var…
Rank-without-GPT: Building GPT-Independent Listwise Rerankers on Open-Source Large Language Models
Xinyu Zhang, Sebastian Hofstätter, Patrick Lewis +2
Listwise rerankers based on large language models (LLM) are the zero-shot state-of-the-art. However, current works in this direction all depend on the GPT models, making it a singl…
"Knowing When You Don't Know": A Multilingual Relevance Assessment Dataset for Robust Retrieval-Augmented Generation
Nandan Thakur, Luiz Bonifacio, Xinyu Zhang +8
Retrieval-Augmented Generation (RAG) grounds Large Language Model (LLM) output by leveraging external knowledge sources to reduce factual hallucinations. However, prior work lacks…
What Do Llamas Really Think? Revealing Preference Biases in Language Model Representations
Raphael Tang, Xinyu Zhang, Jimmy Lin +1
Do large language models (LLMs) exhibit sociodemographic biases, even when they decline to respond? To bypass their refusal to "speak," we study this research question by probing c…
Found in the Middle: Permutation Self-Consistency Improves Listwise Ranking in Large Language Models
Raphael Tang, Xinyu Zhang, Xueguang Ma +2
Large language models (LLMs) exhibit positional bias in how they use context, which especially complicates listwise ranking. To address this, we propose permutation self-consistenc…
HAGRID: A Human-LLM Collaborative Dataset for Generative Information-Seeking with Attribution
Ehsan Kamalloo, Aref Jafari, Xinyu Zhang +2
The rise of large language models (LLMs) had a transformative impact on search, ushering in a new era of search engines that are capable of generating search results in natural lan…
GAIA Search: Hugging Face and Pyserini Interoperability for NLP Training Data Exploration
Aleksandra Piktus, Odunayo Ogundepo, Christopher Akiki +6
Noticing the urgent need to provide tools for fast and user-friendly qualitative analysis of large-scale textual corpora of the modern NLP, we propose to turn to the mature and wel…
Zero-Shot Listwise Document Reranking with a Large Language Model
Xueguang Ma, Xinyu Zhang, Ronak Pradeep +1
Supervised ranking methods based on bi-encoder or cross-encoder architectures have shown success in multi-stage text ranking tasks, but they require large amounts of relevance judg…
Evaluating Embedding APIs for Information Retrieval
Ehsan Kamalloo, Xinyu Zhang, Odunayo Ogundepo +4
The ever-increasing size of language models curtails their widespread availability to the community, thereby galvanizing many companies into offering access to large language model…
Simple Yet Effective Neural Ranking and Reranking Baselines for Cross-Lingual Information Retrieval
Jimmy Lin, David Alfonso-Hermelo, Vitor Jeronymo +8
The advent of multilingual language models has generated a resurgence of interest in cross-lingual information retrieval (CLIR), which is the task of searching documents in one lan…
Improving Out-of-Distribution Generalization of Neural Rerankers with Contextualized Late Interaction
Xinyu Zhang, Minghan Li, Jimmy Lin
Recent progress in information retrieval finds that embedding query and document representation into multi-vector yields a robust bi-encoder retriever on out-of-distribution datase…
Spacerini: Plug-and-play Search Engines with Pyserini and Hugging Face
Christopher Akiki, Odunayo Ogundepo, Aleksandra Piktus +4
We present Spacerini, a tool that integrates the Pyserini toolkit for reproducible information retrieval research with Hugging Face to enable the seamless construction and deployme…
Making a MIRACL: Multilingual Information Retrieval Across a Continuum of Languages
Xinyu Zhang, Nandan Thakur, Odunayo Ogundepo +6
MIRACL (Multilingual Information Retrieval Across a Continuum of Languages) is a multilingual dataset we have built for the WSDM 2023 Cup challenge that focuses on ad hoc retrieval…
Better Than Whitespace: Information Retrieval for Languages without Custom Tokenizers
Odunayo Ogundepo, Xinyu Zhang, Jimmy Lin
Tokenization is a crucial step in information retrieval, especially for lexical matching algorithms, where the quality of indexable tokens directly impacts the effectiveness of a r…
Certified Error Control of Candidate Set Pruning for Two-Stage Relevance Ranking
Minghan Li, Xinyu Zhang, Ji Xin +2
In information retrieval (IR), candidate set pruning has been commonly used to speed up two-stage relevance ranking. However, such an approach lacks accurate error control and ofte…
Towards Best Practices for Training Multilingual Dense Retrieval Models
Xinyu Zhang, Kelechi Ogueji, Xueguang Ma +1
Dense retrieval models using a transformer-based bi-encoder design have emerged as an active area of research. In this work, we focus on the task of monolingual retrieval in a vari…
Mr. TyDi: A Multi-lingual Benchmark for Dense Retrieval
Xinyu Zhang, Xueguang Ma, Peng Shi +1
We present Mr. TyDi, a multi-lingual benchmark dataset for mono-lingual retrieval in eleven typologically diverse languages, designed to evaluate ranking with learned dense represe…