23 papers
Can Scientific Claims Be Removed from Large Language Models? A Systematic Evaluation of Claim-Level Unlearning
Snigdha Paul, Manasi Patwardhan, Arman Cohan
Language models (LMs) are trained on static scientific corpora, whereas scientific knowledge continuously evolves through correction and revision. Scientific claims encoded within…
Every Eval Ever: A Unifying Schema and Community Repository for AI Evaluation Results
Jan Batzner, Sree Harsha Nelaturu, Damian Stachura +45
AI evaluations are widely used for testing and understanding progress. However, the diverse evaluators bring with them inconsistencies that challenge analysis and comparison. First…
Can AI Be a Good Peer Reviewer? A Survey of Peer Review Process, Evaluation, and the Future
Sihong Wu, Owen Jiang, Yilun Zhao +5
Peer review is a multi-stage process involving reviews, rebuttals, meta-reviews, final decisions, and subsequent manuscript revisions. Recent advances in large language models (LLM…
SciMDR: Advancing Scientific Multimodal Document Reasoning
Ziyu Chen, Yilun Zhao, Chengye Wang +3
Constructing scientific multimodal document reasoning datasets for foundation model training involves an inherent trade-off among scale, faithfulness, and realism. To address this…
RbtAct: Rebuttal as Supervision for Actionable Review Feedback Generation
Sihong Wu, Yiling Ma, Yilun Zhao +4
Large language models (LLMs) are increasingly used across the scientific workflow, including to draft peer-review reports. However, many AI-generated reviews are superficial and in…
Teaching Language Models to Forecast Research Success Through Comparative Idea Evaluation
Srujan P Mule, Aniketh Garikaparthi, Manasi Patwardhan
As language models accelerate scientific research by automating hypothesis generation and implementation, a new bottleneck emerges: evaluating and filtering hundreds of AI-generate…