7 papers
Beyond Factual Accuracy: Evaluating Global Reasoning Integrity in RAG Systems with LogicScore
Zhichao Yan, Yunxiao Zhao, Jiapu Wang +4
Current evaluation methods for Retrieval Augmented Generation (RAG) suffer from \textit{factual myopia}: they relentlessly emphasize factual accuracy yet neglect global logical int…
Consistency-Aware Editing for Entity-level Unlearning in Language Models
Xiaoqi Han, VÃctor Gutiérrez-Basulto, Ru Li +3
Large language models (LLMs) risk retaining sensitive, copyrighted, or harmful information from their training data. Entity-level unlearning addresses this issue by removing all kn…
Learnable Game-theoretic Policy Optimization for Data-centric Self-explanation Rationalization
Yunxiao Zhao, Zhiqiang Wang, Xingtong Yu +3
Rationalization, a data-centric framework, aims to build self-explanatory models to explain the prediction outcome by generating a subset of human-intelligible pieces of the input…
Atomic Fact Decomposition Helps Attributed Question Answering
Zhichao Yan, Jiapu Wang, Jiaoyan Chen +3
Attributed Question Answering (AQA) aims to provide both a trustworthy answer and a reliable attribution report for a given question. Retrieval is a widely adopted approach, includ…
Memorization Understanding: Do Large Language Models Have the Ability of Scenario Cognition?
Boxiang Ma, Ru Li, Yuanlong Wang +2
Driven by vast and diverse textual data, large language models (LLMs) have demonstrated impressive performance across numerous natural language processing (NLP) tasks. Yet, a criti…
Explaining Black-box Language Models with Knowledge Probing Systems: A Post-hoc Explanation Perspective
Yunxiao Zhao, Hao Xu, Zhiqiang Wang +3
Pre-trained Language Models (PLMs) are trained on large amounts of unlabeled data, yet they exhibit remarkable reasoning skills. However, the trustworthiness challenges posed by th…