6 papers
StressEval: Failure-Driven Dynamic Benchmarking for Knowledge-Intensive Reasoning in Large Language Models
Yongrui Chen, Yangyang Ma, Xiaoying Huang +4
Static benchmarks for LLMs are increasingly compromised by contamination and overfitting especially on knowledge intensive reasoning tasks While recent dynamic benchmarks can allev…
K-DeCore: Facilitating Knowledge Transfer in Continual Structured Knowledge Reasoning via Knowledge Decoupling
Yongrui Chen, Yi Huang, Yunchang Liu +5
Continual Structured Knowledge Reasoning (CSKR) focuses on training models to handle sequential tasks, where each task involves translating natural language questions into structur…
Pandora: A Code-Driven Large Language Model Agent for Unified Reasoning Across Diverse Structured Knowledge
Yongrui Chen, Junhao He, Linbo Fu +10
Unified Structured Knowledge Reasoning (USKR) aims to answer natural language questions (NLQs) by using structured sources such as tables, databases, and knowledge graphs in a unif…
Pandora: Leveraging Code-driven Knowledge Transfer for Unified Structured Knowledge Reasoning
Yongrui Chen, Junhao He, Linbo Fu +10
Unified Structured Knowledge Reasoning (USKR) aims to answer natural language questions by using structured sources such as tables, databases, and knowledge graphs in a unified way…
OneEval: Benchmarking LLM Knowledge-intensive Reasoning over Diverse Knowledge Bases
Yongrui Chen, Zhiqiang Liu, Jing Yu +21
Large Language Models (LLMs) have demonstrated substantial progress on reasoning tasks involving unstructured text, yet their capabilities significantly deteriorate when reasoning…
Magic Mushroom: A Customizable Benchmark for Fine-grained Analysis of Retrieval Noise Erosion in RAG Systems
Yuxin Zhang, Yan Wang, Yongrui Chen +4
Retrieval-Augmented Generation (RAG) systems enhance Large Language Models (LLMs) by incorporating external retrieved information, mitigating issues such as hallucination and outda…