7 papers
Can LLMs Write Reliable Rubrics? A Meta-Evaluation for Experiment Reproduction
Hanhua Hong, Yizhi Li, Jiaoyan Chen +4
The paper conducts a meta‑evaluation of rubrics generated by large language models for assessing the reproducibility of research papers, comparing intrinsic semantic similarity and…
KG2Code: Bridging Knowledge Graphs and Large Language Models via Executable Code for Question Answering
Yike Wu, Nan Hu, Guilin Qi +11
Recent research has explored the integration of knowledge graphs (KGs) with large language models (LLMs) to enhance their performance on downstream knowledge-intensive tasks, parti…
HiRAS: A Hierarchical Multi-Agent Framework for Paper-to-Code Generation and Execution
Hanhua Hong, Yizhi LI, Jiaoyan Chen +4
Recent advances in large language models have highlighted their potential to automate computational research, particularly reproducing experimental results. However, existing appro…
Embedding Ontologies via Incorporating Extensional and Intensional Knowledge
Keyu Wang, Guilin Qi, Jiaoyan Chen +2
Ontologies contain rich knowledge within domain, which can be divided into two categories, namely extensional knowledge and intensional knowledge. Extensional knowledge provides in…
CoTKR: Chain-of-Thought Enhanced Knowledge Rewriting for Complex Knowledge Graph Question Answering
Yike Wu, Yi Huang, Nan Hu +4
Recent studies have explored the use of Large Language Models (LLMs) with Retrieval Augmented Generation (RAG) for Knowledge Graph Question Answering (KGQA). They typically require…
MINTQA: A Multi-Hop Question Answering Benchmark for Evaluating LLMs on New and Tail Knowledge
Jie He, Nan Hu, Wanqiu Long +2
Large language models (LLMs) have demonstrated impressive capabilities in various reasoning tasks but face significant challenges with complex, knowledge-intensive multi-hop querie…