13 papers
Evidence-Informed LLM Beliefs for Continual Scientific Discovery
Dhruv Agarwal, Reece Adamson, Andrew McCallum +3
Open-ended scientific discovery with large language models (LLMs) increasingly operates as a long-horizon loop of hypothesis search and verification, where a reward signal guides w…
Process-Oriented Evaluation of AI-Assisted Scientific Writing
Patrick Queiroz Da Silva, Sanchaita Hazra, Doeun Lee +2
Bad writing hinders the publication of science. The role of artificial intelligence (AI) in generating and editing scientific texts remains unsettled. Abstracts serve as the critic…
ArtifactLinker: Linking Scientific Artifacts for Automatic State-of-the-Art Discovery
Haofei Yu, Jiaxuan You, Peter Clark +2
Scientific artifacts such as models and datasets are foundations for research. With the rapid growth of platforms like HuggingFace, researchers now have access to a large number of…
AstaBench: Rigorous Benchmarking of AI Agents with a Scientific Research Suite
Jonathan Bragg, Mike D'Arcy, Nishant Balepur +36
AI agents hold the potential to revolutionize scientific productivity by automating literature reviews, replicating experiments, analyzing data, and even proposing new directions o…
ScholarEval: Research Idea Evaluation Grounded in Literature
Hanane Nour Moussa, Patrick Queiroz Da Silva, Daniel Adu-Ampratwum +7
As AI tools become increasingly common for research ideation, robust evaluation is critical to ensure the validity and usefulness of generated ideas. We introduce ScholarEval, a re…
Understanding Usage and Engagement in AI-Powered Scientific Research Tools: The Asta Interaction Dataset
Dany Haddad, Dan Bareket, Joseph Chee Chang +19
AI-powered scientific research tools are rapidly being integrated into research workflows, yet the field lacks a clear lens into how researchers use these systems in real-world set…