Navigation-Based Candidate Expansion and Pretrained Language Models for Citation Recommendation

Abstract

Citation recommendation systems for the scientific literature, to help authors find papers that should be cited, have the potential to speed up discoveries and uncover new routes for scientific exploration. We treat this task as a ranking problem, which we tackle with a two-stage approach: candidate generation followed by re-ranking. Within this framework, we adapt to the scientific domain a proven combination based on "bag of words" retrieval followed by re-scoring with a BERT model. We experimentally show the effects of domain adaptation, both in terms of pretraining on in-domain data and exploiting in-domain vocabulary. In addition, we introduce a novel navigation-based document expansion strategy to enrich the candidate documents processed by our neural models. On three different collections from different scientific disciplines, we achieve the best-reported results in the citation recommendation task.

References in corpus (14)

• Adam: A Method for Stochastic Optimization
• BioBERT: a pre-trained biomedical language representation model for biomedical text mining
• Go for a Walk and Arrive at the Answer: Reasoning Over Paths in Knowledge Bases using Reinforcement Learning
• Passage Re-ranking with BERT
• Technical Paper Recommendation: A Study in Combining Multiple Information Sources
• Question Answering with Subgraph Embeddings
• Traversing Knowledge Graphs in Vector Space
• A Scalable Hybrid Research Paper Recommender System for Microsoft Academic
• Transfer Learning in Biomedical Natural Language Processing: An Evaluation of BERT and ELMo on Ten Benchmarking Datasets
• SciBERT: A Pretrained Language Model for Scientific Text
• Construction of the Literature Graph in Semantic Scholar
• Multi-Hop Knowledge Graph Reasoning with Reward Shaping
• End-to-End Goal-Driven Web Navigation
• Content-Based Citation Recommendation