34 citations · 65 across the 27 of their papers we have counts for
13 papers · 1 filter
InnoEval: On Research Idea Evaluation as a Knowledge-Grounded, Multi-Perspective Reasoning Problem
Shuofei Qiao, Yunxiang Wei, Xuehai Wang +10
The rapid evolution of Large Language Models has catalyzed a surge in scientific idea production, yet this leap has not been accompanied by a matching advance in idea evaluation. T…
Evaluating Reward Model Generalization via Pairwise Maximum Discrepancy Competitions
Shunyang Luo, Peibei Cao, Zhihui Zhu +3
Reward models (RMs) are central to aligning large language models, yet their practical effectiveness hinges on generalization to unseen prompts and shifting distributions. Most exi…
ClinDEF: A Dynamic Evaluation Framework for Large Language Models in Clinical Reasoning
Yuqi Tang, Jing Yu, Zichang Su +7
Clinical diagnosis begins with doctor-patient interaction, during which physicians iteratively gather information, determine examination and refine differential diagnosis through p…
CoT-Evo: Evolutionary Distillation of Chain-of-Thought for Scientific Reasoning
Kehua Feng, Keyan Ding, Zhihui Zhu +3
While chain-of-thought (CoT) distillation from advanced large language models (LLMs) has proven effective in general reasoning tasks, it struggles in scientific domains where even…
Learning an Efficient Multi-Turn Dialogue Evaluator from Multiple LLM Judges
Yuqi Tang, Kehua Feng, Yunfeng Wang +6
Evaluating the conversational abilities of large language models (LLMs) remains a challenging task. Current mainstream approaches primarily rely on the "LLM-as-a-judge" paradigm, w…
Boosting LLM's Molecular Structure Elucidation with Knowledge Enhanced Tree Search Reasoning
Xiang Zhuang, Bin Wu, Jiyu Cui +6
Molecular structure elucidation involves deducing a molecule's structure from various types of spectral data, which is crucial in chemical experimental analysis. While large langua…
OneEval: Benchmarking LLM Knowledge-intensive Reasoning over Diverse Knowledge Bases
Yongrui Chen, Zhiqiang Liu, Jing Yu +21
Large Language Models (LLMs) have demonstrated substantial progress on reasoning tasks involving unstructured text, yet their capabilities significantly deteriorate when reasoning…
SciCUEval: A Comprehensive Dataset for Evaluating Scientific Context Understanding in Large Language Models
Jing Yu, Yuqi Tang, Kehua Feng +8
Large Language Models (LLMs) have shown impressive capabilities in contextual understanding and reasoning. However, evaluating their performance across diverse scientific domains r…
SAFER: Advancing Safety Alignment via Efficient Ex-Ante Reasoning
Kehua Feng, Keyan Ding, Yuhao Wang +5
Recent advancements in large language models (LLMs) have accelerated progress toward artificial general intelligence, yet their potential to generate harmful content poses critical…
Advancing biomolecular understanding and design following human instructions
Xiang Zhuang, Keyan Ding, Tianwen Lyu +9
Understanding and designing biomolecules, such as proteins and small molecules, is central to advancing drug discovery, synthetic biology and enzyme engineering. Recent breakthroug…
SciSafeEval: A Comprehensive Benchmark for Safety Alignment of Large Language Models in Scientific Tasks
Tianhao Li, Jingyu Lu, Chuangxin Chu +12
Large language models (LLMs) have a transformative impact on a variety of scientific tasks across disciplines including biology, chemistry, medicine, and physics. However, ensuring…
SciKnowEval: Evaluating Multi-level Scientific Knowledge of Large Language Models
Kehua Feng, Xinyi Shen, Weijie Wang +4
Large language models (LLMs) are playing an increasingly important role in scientific research, yet there remains a lack of comprehensive benchmarks to evaluate the breadth and dep…
Scientific Large Language Models: A Survey on Biological & Chemical Domains
Qiang Zhang, Keyang Ding, Tianwen Lyv +22
Large Language Models (LLMs) have emerged as a transformative power in enhancing natural language comprehension, representing a significant stride toward artificial general intelli…