7 citations · 11 across the 13 of their papers we have counts for
3 papers · 2 filters
Old Ideas, Novel Problems: The Instability of LLM-Based Novelty Evaluation
Noy Sternlicht, Simra Shahid, Peter Jansen +3
AI-based ideation systems are often evaluated on the novelty of their ideas, and that judgment is increasingly delegated to large language models. Such judges are typically built a…
Deep Research, Shallow Evaluation: A Case Study in Meta-Evaluation for Long-Form QA Benchmarks
Jena D. Hwang, Varsha Kishore, Amanpreet Singh +9
Recent advances have made long-form report-generating systems widely available. This has prompted evaluation frameworks that use LLM-as-judge protocols and claim verification, alon…
Generating Literature-Driven Scientific Theories at Scale
Peter Jansen, Peter Clark, Doug Downey +1
Contemporary automated scientific discovery has focused on agents for generating scientific experiments, while systems that perform higher-level scientific activities such as theor…