4 papers
SKA-Bench: A Fine-Grained Benchmark for Evaluating Structured Knowledge Understanding of LLMs
Zhiqiang Liu, Enpei Niu, Yin Hua +4
Although large language models (LLMs) have made significant progress in understanding Structured Knowledge (SK) like KG and Table, existing evaluations for SK understanding are non…
OneEval: Benchmarking LLM Knowledge-intensive Reasoning over Diverse Knowledge Bases
Yongrui Chen, Zhiqiang Liu, Jing Yu +21
Large Language Models (LLMs) have demonstrated substantial progress on reasoning tasks involving unstructured text, yet their capabilities significantly deteriorate when reasoning…
SciCUEval: A Comprehensive Dataset for Evaluating Scientific Context Understanding in Large Language Models
Jing Yu, Yuqi Tang, Kehua Feng +8
Large Language Models (LLMs) have shown impressive capabilities in contextual understanding and reasoning. However, evaluating their performance across diverse scientific domains r…
TrustUQA: A Trustful Framework for Unified Structured Data Question Answering
Wen Zhang, Long Jin, Yushan Zhu +6
Natural language question answering (QA) over structured data sources such as tables and knowledge graphs have been widely investigated, especially with Large Language Models (LLMs…