6 papers
ParamMute: Suppressing Knowledge-Critical FFNs for Faithful Retrieval-Augmented Generation
Pengcheng Huang, Zhenghao Liu, Yukun Yan +8
Large language models (LLMs) integrated with retrieval-augmented generation (RAG) have improved factuality by grounding outputs in external evidence. However, they remain susceptib…
ThinkNote: Enhancing Knowledge Integration and Utilization of Large Language Models via Constructivist Cognition Modeling
Zhipeng Xu, Zhenghao Liu, Yukun Yan +7
Large Language Models (LLMs) have demonstrated strong performance across a wide range of NLP tasks. However, they often exhibit suboptimal behaviors and inconsistencies when expose…
LISRec: Modeling User Preferences with Learned Item Shortcuts for Sequential Recommendation
Haidong Xin, Zhenghao Liu, Sen Mei +7
User-item interaction histories are pivotal for sequential recommendation systems but often include noise, such as unintended clicks or actions that fail to reflect genuine user pr…
Craw4LLM: Efficient Web Crawling for LLM Pretraining
Shi Yu, Zhiyuan Liu, Chenyan Xiong
Web crawl is a main source of large language models' (LLMs) pretraining data, but the majority of crawled web pages are discarded in pretraining due to low data quality. This paper…
RAG-DDR: Optimizing Retrieval-Augmented Generation Using Differentiable Data Rewards
Xinze Li, Sen Mei, Zhenghao Liu +9
Retrieval-Augmented Generation (RAG) has proven its effectiveness in mitigating hallucinations in Large Language Models (LLMs) by retrieving knowledge from external resources. To a…
Enhancing Dense Retrievers' Robustness with Group-level Reweighting
Peixuan Han, Zhenghao Liu, Zhiyuan Liu +1
The anchor-document data derived from web graphs offers a wealth of paired information for training dense retrieval models in an unsupervised manner. However, unsupervised data con…