18 citations · 135 across the 104 of their papers we have counts for
40 papers · 1 filter
A Silver Bullet or a Compromise for Full Attention? A Comprehensive Study of Gist Token-based Context Compression
Chenlong Deng, Zhisong Zhang, Kelong Mao +4
In this work, we provide a thorough investigation of gist-based context compression methods to improve long-context processing in large language models. We focus on two key questio…
Progressive Multimodal Reasoning via Active Retrieval
Guanting Dong, Chenghao Zhang, Mengjie Deng +3
Multi-step multimodal reasoning tasks pose significant challenges for multimodal large language models (MLLMs), and finding effective ways to enhance their performance in such scen…
Sliding Windows Are Not the End: Exploring Full Ranking with Long-Context Large Language Models
Wenhan Liu, Xinyu Ma, Yutao Zhu +4
Large Language Models (LLMs) have shown exciting performance in listwise passage ranking. Due to the limited input length, existing methods often adopt the sliding window strategy.…
RetroLLM: Empowering Large Language Models to Retrieve Fine-grained Evidence within Generation
Xiaoxi Li, Jiajie Jin, Yujia Zhou +4
Large language models (LLMs) exhibit remarkable generative capabilities but often suffer from hallucinations. Retrieval-augmented generation (RAG) offers an effective solution by i…
Boosting Long-Context Management via Query-Guided Activation Refilling
Hongjin Qian, Zheng Liu, Peitian Zhang +2
Processing long contexts poses a significant challenge for large language models (LLMs) due to their inherent context-window limitations and the computational burden of extensive k…
AssistRAG: Boosting the Potential of Large Language Models with an Intelligent Information Assistant
Yujia Zhou, Zheng Liu, Zhicheng Dou
The emergence of Large Language Models (LLMs) has significantly advanced natural language processing, but these models often generate factually incorrect information, known as "hal…
Little Giants: Synthesizing High-Quality Embedding Data at Scale
Haonan Chen, Liang Wang, Nan Yang +4
Synthetic data generation has become an increasingly popular way of training models without the need for large, manually labeled datasets. For tasks like text embedding, synthetic…
CORAL: Benchmarking Multi-turn Conversational Retrieval-Augmentation Generation
Yiruo Cheng, Kelong Mao, Ziliang Zhao +6
Retrieval-Augmented Generation (RAG) has become a powerful paradigm for enhancing large language models (LLMs) through external knowledge retrieval. Despite its widespread attentio…
A Survey of Conversational Search
Fengran Mo, Kelong Mao, Ziliang Zhao +7
As a cornerstone of modern information access, search engines have become indispensable in everyday life. With the rapid advancements in AI and natural language processing (NLP) te…
LLMs + Persona-Plug = Personalized LLMs
Jiongnan Liu, Yutao Zhu, Shuting Wang +6
Personalization plays a critical role in numerous language tasks and applications, since users with the same requirements may prefer diverse outputs based on their individual inter…
Trustworthiness in Retrieval-Augmented Generation Systems: A Survey
Yujia Zhou, Wenbo Zhang, Jingying Shao +10
Retrieval-Augmented Generation (RAG) has quickly grown into a pivotal paradigm in the development of Large Language Models (LLMs). Although existing research mainly emphasizes accu…
MemoRAG: Boosting Long Context Processing with Global Memory-Enhanced Retrieval Augmentation
Hongjin Qian, Zheng Liu, Peitian Zhang +4
Processing long contexts presents a significant challenge for large language models (LLMs). While recent advancements allow LLMs to handle much longer contexts than before (e.g., 3…
Enabling Discriminative Reasoning in LLMs for Legal Judgment Prediction
Chenlong Deng, Kelong Mao, Yuyao Zhang +1
Legal judgment prediction is essential for enhancing judicial efficiency. In this work, we identify that existing large language models (LLMs) underperform in this domain due to ch…
Query-oriented Data Augmentation for Session Search
Haonan Chen, Zhicheng Dou, Yutao Zhu +1
Modeling contextual information in a search session has drawn more and more attention when understanding complex user intents. Recent methods are all data-driven, i.e., they train…
YuLan: An Open-source Large Language Model
Yutao Zhu, Kun Zhou, Kelong Mao +35
Large language models (LLMs) have become the foundation of many applications, leveraging their extensive capabilities in processing and understanding natural language. While many o…
Learning Interpretable Legal Case Retrieval via Knowledge-Guided Case Reformulation
Chenlong Deng, Kelong Mao, Zhicheng Dou
Legal case retrieval for sourcing similar cases is critical in upholding judicial fairness. Different from general web search, legal case retrieval involves processing lengthy, com…
DomainRAG: A Chinese Benchmark for Evaluating Domain-specific Retrieval-Augmented Generation
Shuting Wang, Jiongnan Liu, Shiren Song +6
Retrieval-Augmented Generation (RAG) offers a promising solution to address various limitations of Large Language Models (LLMs), such as hallucination and difficulties in keeping u…
Disentangled Hyperbolic Representation Learning for Heterogeneous Graphs
Qijie Bai, Changli Nie, Haiwei Zhang +2
Heterogeneous graphs have attracted a lot of research interests recently due to the success for representing complex real-world systems. However, existing methods have two pain poi…
DemoRank: Selecting Effective Demonstrations for Large Language Models in Ranking Task
Wenhan Liu, Yutao Zhu, Zhicheng Dou
Recently, there has been increasing interest in applying large language models (LLMs) as zero-shot passage rankers. However, few studies have explored how to select appropriate in-…
RichRAG: Crafting Rich Responses for Multi-faceted Queries in Retrieval-Augmented Generation
Shuting Wang, Xin Yu, Mang Wang +3
Retrieval-augmented generation (RAG) effectively addresses issues of static knowledge and hallucination in large language models. Existing studies mostly focus on question scenario…
Are Long-LLMs A Necessity For Long-Context Tasks?
Hongjin Qian, Zheng Liu, Peitian Zhang +4
The learning and deployment of long-LLMs remains a challenging problem despite recent progresses. In this work, we argue that the long-LLMs are not a necessity to solve long-contex…
Compressing Lengthy Context With UltraGist
Peitian Zhang, Zheng Liu, Shitao Xiao +3
Compressing lengthy context is a critical but technically challenging problem. In this paper, we propose a new method called UltraGist, which is distinguished for its high-quality…
FlashRAG: A Modular Toolkit for Efficient Retrieval-Augmented Generation Research
Jiajie Jin, Yutao Zhu, Guanting Dong +7
With the advent of large language models (LLMs) and multimodal large language models (MLLMs), the potential of retrieval-augmented generation (RAG) has attracted considerable resea…
Extending Llama-3's Context Ten-Fold Overnight
Peitian Zhang, Ninglu Shao, Zheng Liu +4
We extend the context length of Llama-3-8B-Instruct from 8K to 80K via QLoRA fine-tuning. The entire training cycle is super efficient, which takes 8 hours on one 8xA800 (80G) GPU…