59 citations · 110 across the 21 of their papers we have counts for
22 papers
Scaling Laws For Data-Constrained Dense Retrieval
Bowen Fu, Jingtao Zhan
Dense retrieval models are typically obtained by fine-tuning pretrained language models on supervised retrieval data, where high-quality annotations or interaction logs are often l…
Decoupled Mixture-of-Experts for Parametric Knowledge Injection
Baoqing Yue, Weihang Su, Qingyao Ai +5
Knowledge injection aims to equip large language models (LLMs) with external, domain-specific, or time-sensitive knowledge. Existing approaches typically face a trade-off between f…
TEC: A Collection of Human Trial-and-error Trajectories for Problem Solving
Xinkai Zhang, Jingtao Zhan, Yiqun Liu +1
Trial-and-error is a fundamental strategy for humans to solve complex problems and a necessary capability for Artificial Intelligence (AI) systems operating in real-world environme…
Relative-Based Scaling Law for Neural Language Models
Baoqing Yue, Jinyuan Zhou, Zixi Wei +3
Scaling laws aim to accurately predict model performance across different scales. Existing scaling-law studies almost exclusively rely on cross-entropy as the evaluation metric. Ho…
What Scales in Cross-Entropy Scaling Law?
Junxi Yan, Zixi Wei, Qingyao Ai +2
The cross-entropy scaling law has long served as a key tool for guiding the development of large language models. It shows that cross-entropy loss decreases in a predictable power-…
Dynamic and Parametric Retrieval-Augmented Generation
Weihang Su, Qingyao Ai, Jingtao Zhan +2
Retrieval-Augmented Generation (RAG) has become a foundational paradigm for equipping large language models (LLMs) with external knowledge, playing a critical role in information r…
Evaluating Intelligence via Trial and Error
Jingtao Zhan, Jiahao Zhao, Jiayu Li +7
Intelligence is a crucial trait for species to find solutions within a limited number of trial-and-error attempts. Building on this idea, we introduce Survival Game as a framework…
Foundations of GenIR
Qingyao Ai, Jingtao Zhan, Yiqun Liu
The chapter discusses the foundational impact of modern generative AI models on information access (IA) systems. In contrast to traditional AI, the large-scale training and superio…
StepTool: Enhancing Multi-Step Tool Usage in LLMs via Step-Grained Reinforcement Learning
Yuanqing Yu, Zhefan Wang, Weizhi Ma +4
Despite their powerful text generation capabilities, large language models (LLMs) still struggle to effectively utilize external tools to solve complex tasks, a challenge known as…
Prompt Refinement with Image Pivot for Text-to-Image Generation
Jingtao Zhan, Qingyao Ai, Yiqun Liu +5
For text-to-image generation, automatically refining user-provided natural language prompts into the keyword-enriched prompts favored by systems is essential for the user experienc…
Capability-aware Prompt Reformulation Learning for Text-to-Image Generation
Jingtao Zhan, Qingyao Ai, Yiqun Liu +2
Text-to-image generation systems have emerged as revolutionary tools in the realm of artistic creation, offering unprecedented ease in transforming textual prompts into visual art.…
Query Augmentation by Decoding Semantics from Brain Signals
Ziyi Ye, Jingtao Zhan, Qingyao Ai +4
Query augmentation is a crucial technique for refining semantically imprecise queries. Traditionally, query augmentation relies on extracting information from initially retrieved,…
Scaling Laws For Dense Retrieval
Yan Fang, Jingtao Zhan, Qingyao Ai +4
Scaling up neural models has yielded significant advancements in a wide array of tasks, particularly in language generation. Previous studies have found that the performance of neu…
Constructing Tree-based Index for Efficient and Effective Dense Retrieval
Haitao Li, Qingyao Ai, Jingtao Zhan +4
Recent studies have shown that Dense Retrieval (DR) techniques can significantly improve the performance of first-stage retrieval in IR systems. Despite its empirical effectiveness…
Disentangled Modeling of Domain and Relevance for Adaptable Dense Retrieval
Jingtao Zhan, Qingyao Ai, Yiqun Liu +4
Recent advance in Dense Retrieval (DR) techniques has significantly improved the effectiveness of first-stage retrieval. Trained with large-scale supervised data, DR models can enc…
Evaluating Interpolation and Extrapolation Performance of Neural Retrieval Models
Jingtao Zhan, Xiaohui Xie, Jiaxin Mao +4
A retrieval model should not only interpolate the training data but also extrapolate well to the queries that are different from the training data. While neural retrieval models ha…
Interpreting Dense Retrieval as Mixture of Topics
Jingtao Zhan, Jiaxin Mao, Yiqun Liu +3
Dense Retrieval (DR) reaches state-of-the-art results in first-stage retrieval, but little is known about the mechanisms that contribute to its success. Therefore, in this work, we…
Learning Discrete Representations via Constrained Clustering for Effective and Efficient Dense Retrieval
Jingtao Zhan, Jiaxin Mao, Yiqun Liu +3
Dense Retrieval (DR) has achieved state-of-the-art first-stage ranking effectiveness. However, the efficiency of most existing DR models is limited by the large memory cost of stor…
Jointly Optimizing Query Encoder and Product Quantization to Improve Retrieval Performance
Jingtao Zhan, Jiaxin Mao, Yiqun Liu +3
Recently, Information Retrieval community has witnessed fast-paced advances in Dense Retrieval (DR), which performs first-stage retrieval with embedding-based search. Despite the i…
Optimizing Dense Retrieval Model Training with Hard Negatives
Jingtao Zhan, Jiaxin Mao, Yiqun Liu +3
Ranking has always been one of the top concerns in information retrieval researches. For decades, the lexical matching signal has dominated the ad-hoc retrieval process, but solely…
Learning To Retrieve: How to Train a Dense Retrieval Model Effectively and Efficiently
Jingtao Zhan, Jiaxin Mao, Yiqun Liu +2
Ranking has always been one of the top concerns in information retrieval research. For decades, lexical matching signal has dominated the ad-hoc retrieval process, but it also has…
RepBERT: Contextualized Text Embeddings for First-Stage Retrieval
Jingtao Zhan, Jiaxin Mao, Yiqun Liu +2
Although exact term match between queries and documents is the dominant method to perform first-stage retrieval, we propose a different approach, called RepBERT, to represent docum…