59 citations · 82 across the 13 of their papers we have counts for
13 papers
Language Models Don't Know What You Want: Evaluating Personalization in Deep Research Needs Real Users
Nishant Balepur, Malachi Hamada, Varsha Kishore +7
Deep Research (DR) systems help researchers cope with ballooning publishing counts. Such tools synthesize scientific papers to answer research queries, but lack understanding of th…
Deep Research, Shallow Evaluation: A Case Study in Meta-Evaluation for Long-Form QA Benchmarks
Jena D. Hwang, Varsha Kishore, Amanpreet Singh +9
Recent advances have made long-form report-generating systems widely available. This has prompted evaluation frameworks that use LLM-as-judge protocols and claim verification, alon…
Understanding Usage and Engagement in AI-Powered Scientific Research Tools: The Asta Interaction Dataset
Dany Haddad, Dan Bareket, Joseph Chee Chang +19
AI-powered scientific research tools are rapidly being integrated into research workflows, yet the field lacks a clear lens into how researchers use these systems in real-world set…
PreScience: A Dataset and Benchmark for Scientific Forecasting
Anirudh Ajith, Amanpreet Singh, Jay DeYoung +7
Can AI systems trained on the existing scientific record forecast the advances that will follow? We introduce PreScience, a dataset and benchmark for scientific forecasting built a…
AstaBench: Rigorous Benchmarking of AI Agents with a Scientific Research Suite
Jonathan Bragg, Mike D'Arcy, Nishant Balepur +36
AI agents hold the potential to revolutionize scientific productivity by automating literature reviews, replicating experiments, analyzing data, and even proposing new directions o…
Ai2 Scholar QA: Organized Literature Synthesis with Attribution
Amanpreet Singh, Joseph Chee Chang, Chloe Anastasiades +15
Retrieval-augmented generation is increasingly effective in answering scientific questions from literature, but many state-of-the-art systems are expensive and closed-source. We in…
OpenScholar: Synthesizing Scientific Literature with Retrieval-augmented LMs
Akari Asai, Jacqueline He, Rulin Shao +22
Scientific progress depends on researchers' ability to synthesize the growing body of literature. Can large language models (LMs) assist scientists in this task? We introduce OpenS…
TOPICAL: TOPIC Pages AutomagicaLly
John Giorgi, Amanpreet Singh, Doug Downey +2
Topic pages aggregate useful information about an entity or concept into a single succinct and accessible article. Automated creation of topic pages would enable their rapid curati…
The Semantic Scholar Open Data Platform
Rodney Kinney, Chloe Anastasiades, Russell Authur +45
The volume of scientific output is creating an urgent need for automated tools to help scientists keep up with developments in their field. Semantic Scholar (S2) is an open data pl…
SciRepEval: A Multi-Format Benchmark for Scientific Document Representations
Amanpreet Singh, Mike D'Arcy, Arman Cohan +2
Learned representations of scientific documents can serve as valuable input features for downstream tasks without further fine-tuning. However, existing benchmarks for evaluating t…
FeedLens: Polymorphic Lenses for Personalizing Exploratory Search over Knowledge Graphs
Harmanpreet Kaur, Doug Downey, Amanpreet Singh +3
The vast scale and open-ended nature of knowledge graphs (KGs) make exploratory search over them cognitively demanding for users. We introduce a new technique, polymorphic lenses,…
Embedding Recycling for Language Models
Jon Saad-Falcon, Amanpreet Singh, Luca Soldaini +3
Real-world applications of neural language models often involve running many different models over the same corpus. The high computational cost of these runs has led to interest in…
Open4Business(O4B): An Open Access Dataset for Summarizing Business Documents
Amanpreet Singh, Niranjan Balasubramanian
A major challenge in fine-tuning deep learning models for automatic summarization is the need for large domain specific datasets. One of the barriers to curating such data from res…