5 papers
QEDBENCH: Quantifying the Alignment Gap in Automated Evaluation of University-Level Mathematical Proofs
Santiago Gonzalez, Alireza Amiri Bavandpour, Peter Ye +48
As Large Language Models (LLMs) saturate elementary benchmarks, the research frontier has shifted from generation to the reliability of automated evaluation. We demonstrate that st…
Spec-Driven AI for Science: The ARIA Framework for Automated and Reproducible Data Analysis
Chuke Chen, Biao Luo, Nan Li +4
The rapid expansion of scientific data has widened the gap between analytical capability and research intent. Existing AI-based analysis tools, ranging from AutoML frameworks to ag…
ShizhenGPT: Towards Multimodal LLMs for Traditional Chinese Medicine
Junying Chen, Zhenyang Cai, Zhiheng Liu +10
Despite the success of large language models (LLMs) in various domains, their potential in Traditional Chinese Medicine (TCM) remains largely underexplored due to two critical barr…
KG-Augmented Executable CoT for Mathematical Coding
Xingyu Chen, Junxiu An, Jun Guo +2
In recent years, large language models (LLMs) have excelled in natural language processing tasks but face significant challenges in complex reasoning tasks such as mathematical rea…
Environmental large language model Evaluation (ELLE) dataset: A Benchmark for Evaluating Generative AI applications in Eco-environment Domain
Jing Guo, Nan Li, Ming Xu
Generative AI holds significant potential for ecological and environmental applications such as monitoring, data analysis, education, and policy support. However, its effectiveness…