8 citations · 28 across the 10 of their papers we have counts for
7 papers · 1 filter
Deep Research, Shallow Evaluation: A Case Study in Meta-Evaluation for Long-Form QA Benchmarks
Jena D. Hwang, Varsha Kishore, Amanpreet Singh +9
Recent advances have made long-form report-generating systems widely available. This has prompted evaluation frameworks that use LLM-as-judge protocols and claim verification, alon…
OpenScholar: Synthesizing Scientific Literature with Retrieval-augmented LMs
Akari Asai, Jacqueline He, Rulin Shao +22
Scientific progress depends on researchers' ability to synthesize the growing body of literature. Can large language models (LMs) assist scientists in this task? We introduce OpenS…
MARG: Multi-Agent Review Generation for Scientific Papers
Mike D'Arcy, Tom Hope, Larry Birnbaum +1
We study the ability of LLMs to generate feedback for scientific papers and develop MARG, a feedback generation approach using multiple LLM instances that engage in internal discus…
ARIES: A Corpus of Scientific Paper Edits Made in Response to Peer Reviews
Mike D'Arcy, Alexis Ross, Erin Bransom +4
We introduce the task of automatically revising scientific papers based on peer feedback and release ARIES, a dataset of review comments and their corresponding paper edits. The da…
SciRepEval: A Multi-Format Benchmark for Scientific Document Representations
Amanpreet Singh, Mike D'Arcy, Arman Cohan +2
Learned representations of scientific documents can serve as valuable input features for downstream tasks without further fine-tuning. However, existing benchmarks for evaluating t…
Embedding Recycling for Language Models
Jon Saad-Falcon, Amanpreet Singh, Luca Soldaini +3
Real-world applications of neural language models often involve running many different models over the same corpus. The high computational cost of these runs has led to interest in…
CODAH: An Adversarially Authored Question-Answer Dataset for Common Sense
Michael Chen, Mike D'Arcy, Alisa Liu +2
Commonsense reasoning is a critical AI capability, but it is difficult to construct challenging datasets that test common sense. Recent neural question answering systems, based on…