Publications (51)
A Weak Supervision Approach to Detecting Visual Anomalies for Automated Testing of Graphics Units
Adi Szeskin, Lev Faivishevsky, Ashwin K Muppalla +2
We present a deep learning system for testing graphics units by detecting novel visual corruptions in videos. Unlike previous work in which manual tagging was required to collect l…
Bursting Scientific Filter Bubbles: Boosting Innovation via Novel Author Discovery
Jason Portenoy, Marissa Radensky, Jevin West +3
Isolated silos of scientific research and the growing challenge of information overload limit awareness across the literature and hinder innovation. Algorithmic curation and recomm…
A Computational Inflection for Scientific Discovery
Tom Hope, Doug Downey, Oren Etzioni +2
We stand at the foot of a significant inflection in the trajectory of scientific discovery. As society continues on its fast-paced digital transformation, so does humankind's colle…
Multi-Vector Models with Textual Guidance for Fine-Grained Scientific Document Similarity
Sheshera Mysore, Arman Cohan, Tom Hope
We present a new scientific document similarity model based on matching fine-grained aspects of texts. To train our model, we exploit a naturally-occurring source of supervision: s…
ABCD-LINK: Annotation Bootstrapping for Cross-Document Fine-Grained Links
Serwar Basch, Ilia Kuznetsov, Tom Hope +1
Understanding fine-grained links between documents is crucial for many applications, yet progress is limited by the lack of efficient methods for data curation. To address this lim…
Clustering Noisy Signals with Structured Sparsity Using Time-Frequency Representation
Tom Hope, Avishai Wagner, Or Zuk
We propose a simple and efficient time-series clustering framework particularly suited for low Signal-to-Noise Ratio (SNR), by simultaneous smoothing and dimensionality reduction a…
CHIMERA: A Knowledge Base of Scientific Idea Recombinations for Research Analysis and Ideation
Noy Sternlicht, Tom Hope
A hallmark of human innovation is recombination -- the creation of novel ideas by integrating elements from existing concepts and mechanisms. In this work, we introduce CHIMERA, th…
Beyond Good Intentions: Reporting the Research Landscape of NLP for Social Good
Fernando Gonzalez, Zhijing Jin, Bernhard Schölkopf +3
With the recent advances in natural language processing (NLP), a vast number of applications have emerged across various use cases. Among the plethora of NLP applications, many aca…
What Can Natural Language Processing Do for Peer Review?
Ilia Kuznetsov, Osama Mohammed Afzal, Koen Dercksen +21
The number of scientific articles produced every year is growing rapidly. Providing quality control over them is crucial for scientists and, ultimately, for the public good. In mod…
CascadER: Cross-Modal Cascading for Knowledge Graph Link Prediction
Tara Safavi, Doug Downey, Tom Hope
Knowledge graph (KG) link prediction is a fundamental task in artificial intelligence, with applications in natural language processing, information retrieval, and biomedicine. Rec…
Debatable Intelligence: Benchmarking LLM Judges via Debate Speech Evaluation
Noy Sternlicht, Ariel Gera, Roy Bar-Haim +2
We introduce Debate Speech Evaluation as a novel and challenging benchmark for assessing LLM judges. Evaluating debate speeches requires a deep understanding of the speech at multi…
In-depth Research Impact Summarization through Fine-Grained Temporal Citation Analysis
Hiba Arnaout, Noy Sternlicht, Tom Hope +1
Understanding the impact of scientific publications is crucial for identifying breakthroughs and guiding future research. Traditional metrics based on citation counts often miss th…
Extracting a Knowledge Base of Mechanisms from COVID-19 Papers
Tom Hope, Aida Amini, David Wadden +6
The COVID-19 pandemic has spawned a diverse body of scientific literature that is challenging to navigate, stimulating interest in automated tools to help find useful knowledge. We…
Iterate Until Retrieved: Factual Nugget Optimization for Discoverable Continual Corrections in Agentic RAG
Moshe Hazoom, Gal Patel, Alon Talmor +1
Agentic retrieval-augmented generation (RAG) systems in complex B2B (business-to-business) settings may often receive free-form response feedback. Rather than generic feedback sign…
Consistency-Driven Co-Evolution for Self-Supervised Cross-Representation Learning
Xuehang Guo, Pengyuan Li, Tom Hope +3
As chart images, tabular data, and visualization code play increasingly important roles across diverse domains, cross-representation understanding across these modalities poses fun…
On-the-fly Definition Augmentation of LLMs for Biomedical NER
Monica Munnangi, Sergey Feldman, Byron C Wallace +3
Despite their general capabilities, LLMs still struggle on biomedical NER tasks, which are difficult due to the presence of specialized terminology and lack of training data. In th…
Parameter-Efficient Fine-Tuning of LLaMA for the Clinical Domain
Aryo Pradipta Gema, Pasquale Minervini, Luke Daines +2
Adapting pretrained language models to novel domains, such as clinical applications, traditionally involves retraining their entire set of parameters. Parameter-Efficient Fine-Tuni…
Literature-Augmented Clinical Outcome Prediction
Aakanksha Naik, Sravanthi Parasa, Sergey Feldman +2
We present BEEP (Biomedical Evidence-Enhanced Predictions), a novel approach for clinical outcome prediction that retrieves patient-specific medical literature and incorporates it…
Ballpark Learning: Estimating Labels from Rough Group Comparisons
Tom Hope, Dafna Shahaf
We are interested in estimating individual labels given only coarse, aggregated signal over the data points. In our setting, we receive sets ("bags") of unlabeled instances with co…
PreScience: A Dataset and Benchmark for Scientific Forecasting
Anirudh Ajith, Amanpreet Singh, Jay DeYoung +7
Can AI systems trained on the existing scientific record forecast the advances that will follow? We introduce PreScience, a dataset and benchmark for scientific forecasting built a…
Literature-Grounded Novelty Assessment of Scientific Ideas
Simra Shahid, Marissa Radensky, Raymond Fok +3
Automated scientific idea generation systems have made remarkable progress, yet the automatic evaluation of idea novelty remains a critical and underexplored challenge. Manual eval…
CHAMP: Efficient Annotation and Consolidation of Cluster Hierarchies
Arie Cattan, Tom Hope, Doug Downey +4
Various NLP tasks require a complex hierarchical structure over nodes, where each node is a cluster of items. Examples include generating entailment graphs, hierarchical cross-docu…
Inferring Scientific Cross-Document Coreference and Hierarchy with Definition-Augmented Relational Reasoning
Lior Forer, Tom Hope
We address the fundamental task of inferring cross-document coreference and hierarchy in scientific texts, which has important applications in knowledge graph construction, search,…
Accelerating Innovation Through Analogy Mining
Tom Hope, Joel Chan, Aniket Kittur +1
The availability of large idea repositories (e.g., the U.S. patent database) could significantly accelerate innovation and discovery by providing people with inspiration from solut…
Language (Re)modelling: Towards Embodied Language Understanding
Ronen Tamari, Chen Shani, Tom Hope +3
While natural language understanding (NLU) is advancing rapidly, today's technology differs from human-like language understanding in fundamental ways, notably in its inferior effi…
Learning a faceted customer segmentation for discovering new business opportunities at Intel
Itay Lieder, Meirav Segal, Eran Avidan +2
For sales and marketing organizations within large enterprises, identifying and understanding new markets, customers and partners is a key challenge. Intel's Sales and Marketing Gr…
ARIES: A Corpus of Scientific Paper Edits Made in Response to Peer Reviews
Mike D'Arcy, Alexis Ross, Erin Bransom +4
We introduce the task of automatically revising scientific papers based on peer feedback and release ARIES, a dataset of review comments and their corresponding paper edits. The da…
MARG: Multi-Agent Review Generation for Scientific Papers
Mike D'Arcy, Tom Hope, Larry Birnbaum +1
We study the ability of LLMs to generate feedback for scientific papers and develop MARG, a feedback generation approach using multiple LLM instances that engage in internal discus…
Beyond "Not Novel Enough": Enriching Scholarly Critique with LLM-Assisted Feedback
Osama Mohammed Afzal, Preslav Nakov, Tom Hope +1
Novelty assessment is a central yet understudied aspect of peer review, particularly in high volume fields like NLP where reviewer capacity is increasingly strained. We present a s…
Anagent For Enhancing Scientific Table & Figure Analysis
Xuehang Guo, Zhiyong Lu, Tom Hope +1
In scientific research, analysis requires accurately interpreting complex multimodal knowledge, integrating evidence from different sources, and drawing inferences grounded in doma…
Ballpark Crowdsourcing: The Wisdom of Rough Group Comparisons
Tom Hope, Dafna Shahaf
Crowdsourcing has become a popular method for collecting labeled training data. However, in many practical scenarios traditional labeling can be difficult for crowdworkers (for exa…
A Search Engine for Discovery of Scientific Challenges and Directions
Dan Lahav, Jon Saad Falcon, Bailey Kuehl +8
Keeping track of scientific challenges, advances and emerging directions is a fundamental part of research. However, researchers face a flood of papers that hinders discovery of im…
SciMON: Scientific Inspiration Machines Optimized for Novelty
Qingyun Wang, Doug Downey, Heng Ji +1
We explore and enhance the ability of neural language models to generate novel scientific directions grounded in literature. Work on literature-based hypothesis generation has trad…
How do Humans and Language Models Reason About Creativity? A Comparative Analysis
Antonio Laverghetta, Tuhin Chakrabarty, Tom Hope +3
Creativity assessment in science and engineering is increasingly based on both human and AI judgment, but the cognitive processes and biases behind these evaluations remain poorly…
Scientific Language Models for Biomedical Knowledge Base Completion: An Empirical Study
Rahul Nadkarni, David Wadden, Iz Beltagy +3
Biomedical knowledge graphs (KGs) hold rich information on entities such as diseases, drugs, and genes. Predicting missing links in these graphs can boost many important applicatio…
MUSE: A Full-Text Cross-Domain Knowledge Base of Scientific Problems, Solutions, and Rationales
Tsofia Cohen, Tom Hope
Scientific papers contain fine-grained records of problem solving: authors mention technical obstacles and methods that were used to address them, often along with reasoning on why…
Increasing Textual Context Size Boosts Medical Image-Text Matching
Idan Glassberg, Tom Hope
This short technical report demonstrates a simple technique that yields state of the art results in medical image-text matching tasks. We analyze the use of OpenAI's CLIP, a genera…
LVLM-Aware Multimodal Retrieval for RAG-Based Medical Diagnosis with General-Purpose Models
Nir Mazor, Tom Hope
Retrieving visual and textual information from medical literature and hospital records can enhance diagnostic accuracy for clinical image interpretation. However, multimodal retrie…
CARE: Extracting Experimental Findings From Clinical Literature
Aakanksha Naik, Bailey Kuehl, Erin Bransom +2
Extracting fine-grained experimental findings from literature can provide dramatic utility for scientific applications. Prior work has developed annotation schemas and datasets for…
A Dataset for N-ary Relation Extraction of Drug Combinations
Aryeh Tiktinsky, Vijay Viswanathan, Danna Niezni +5
Combination therapies have become the standard of care for diseases such as cancer, tuberculosis, malaria and HIV. However, the combinatorial set of available multi-drug treatments…
Scaling Creative Inspiration with Fine-Grained Functional Aspects of Ideas
Tom Hope, Ronen Tamari, Hyeonsu Kang +4
Large repositories of products, patents and scientific papers offer an opportunity for building systems that scour millions of ideas and help users discover inspirations. However,…
SciRIFF: A Resource to Enhance Language Model Instruction-Following over Scientific Literature
David Wadden, Kejian Shi, Jacob Morrison +11
We present SciRIFF (Scientific Resource for Instruction-Following and Finetuning), a dataset of 137K instruction-following instances for training and evaluation, covering 54 tasks.…
SciCo: Hierarchical Cross-Document Coreference for Scientific Concepts
Arie Cattan, Sophie Johnson, Daniel Weld +4
Determining coreference of concept mentions across multiple documents is a fundamental task in natural language understanding. Previous work on cross-document coreference resolutio…
Augmenting Scientific Creativity with an Analogical Search Engine
Hyeonsu B. Kang, Xin Qian, Tom Hope +3
Analogies have been central to creative problem-solving throughout the history of science and technology. As the number of scientific papers continues to increase exponentially, th…
SciSight: Combining faceted navigation and research group detection for COVID-19 exploratory scientific search
Tom Hope, Jason Portenoy, Kishore Vasan +5
The COVID-19 pandemic has sparked unprecedented mobilization of scientists, generating a deluge of papers that makes it hard for researchers to keep track and explore new direction…
IdeaSynth: Iterative Research Idea Development Through Evolving and Composing Idea Facets with Literature-Grounded Feedback
Kevin Pu, K. J. Kevin Feng, Tovi Grossman +6
Research ideation involves broad exploring and deep refining ideas. Both require deep engagement with literature. Existing tools focus primarily on idea broad generation, yet offer…
SynerGPT: In-Context Learning for Personalized Drug Synergy Prediction and Drug Design
Carl Edwards, Aakanksha Naik, Tushar Khot +3
Predicting synergistic drug combinations can help accelerate discovery of cancer treatments, particularly therapies personalized to a patient's specific tumor via biopsied cells. I…
Human-LLM Compound System for Scientific Ideation through Facet Recombination and Novelty Evaluation
Marissa Radensky, Simra Shahid, Raymond Fok +3
The scientific ideation process often involves blending facets of existing papers to create new ideas. We contribute Scideator, the first human-LLM system for facet-based scientifi…
CodeScientist: End-to-End Semi-Automated Scientific Discovery with Code-based Experimentation
Peter Jansen, Oyvind Tafjord, Marissa Radensky +6
Despite the surge of interest in autonomous scientific discovery (ASD) of software artifacts (e.g., improved ML algorithms), current ASD systems face two key limitations: (1) they…
ACCoRD: A Multi-Document Approach to Generating Diverse Descriptions of Scientific Concepts
Sonia K. Murthy, Kyle Lo, Daniel King +7
Systems that can automatically define unfamiliar terms hold the promise of improving the accessibility of scientific texts, especially for readers who may lack prerequisite backgro…
WikiSTAR: A System for Shedding Light on the Hidden History of Scientific Wikipedia Articles
Omer Ehrlich, Nitzan Barzilay, Rona Aviram +1
WikiSTAR is an interactive system that uses a large language model classifier to label and visualize scientifically meaningful edits in Wikipedia articles, allowing users to trace…