7 citations · 11 across the 13 of their papers we have counts for
5 papers · 1 filter
Old Ideas, Novel Problems: The Instability of LLM-Based Novelty Evaluation
Noy Sternlicht, Simra Shahid, Peter Jansen +3
AI-based ideation systems are often evaluated on the novelty of their ideas, and that judgment is increasingly delegated to large language models. Such judges are typically built a…
Deep Research, Shallow Evaluation: A Case Study in Meta-Evaluation for Long-Form QA Benchmarks
Jena D. Hwang, Varsha Kishore, Amanpreet Singh +9
Recent advances have made long-form report-generating systems widely available. This has prompted evaluation frameworks that use LLM-as-judge protocols and claim verification, alon…
Generating Literature-Driven Scientific Theories at Scale
Peter Jansen, Peter Clark, Doug Downey +1
Contemporary automated scientific discovery has focused on agents for generating scientific experiments, while systems that perform higher-level scientific activities such as theor…
Ai2 Scholar QA: Organized Literature Synthesis with Attribution
Amanpreet Singh, Joseph Chee Chang, Chloe Anastasiades +15
Retrieval-augmented generation is increasingly effective in answering scientific questions from literature, but many state-of-the-art systems are expensive and closed-source. We in…
ArxivDIGESTables: Synthesizing Scientific Literature into Tables using Language Models
Benjamin Newman, Yoonjoo Lee, Aakanksha Naik +6
When conducting literature reviews, scientists often create literature review tables - tables whose rows are publications and whose columns constitute a schema, a set of aspects us…