Large language models for automated scholarly paper review: A survey
arXiv:2501.10326 · doi:10.1016/j.inffus.2025.103332
Abstract
Large language models (LLMs) have significantly impacted human society, influencing various domains. Among them, academia is not simply a domain affected by LLMs, but it is also the pivotal force in the development of LLMs. In academic publication, this phenomenon is represented during the incorporation of LLMs into the peer review mechanism for reviewing manuscripts. LLMs hold transformative potential for the full-scale implementation of automated scholarly paper review (ASPR), but they also pose new issues and challenges that need to be addressed. In this survey paper, we aim to provide a holistic view of ASPR in the era of LLMs. We begin with a survey to find out which LLMs are used to conduct ASPR. Then, we review what ASPR-related technological bottlenecks have been solved with the incorporation of LLM technology. After that, we move on to explore new methods, new datasets, new source code, and new online systems that come with LLMs for ASPR. Furthermore, we summarize the performance and issues of LLMs in ASPR, and investigate the attitudes and reactions of publishers and academia to ASPR. Lastly, we discuss the challenges and future directions associated with the development of LLMs for ASPR. This survey serves as an inspirational reference for the researchers and can promote the progress of ASPR for its actual implementation.
Please cite the version of Information Fusion
References in corpus (39)
- LLaMA: Open and Efficient Foundation Language Models
- Llama 2: Open Foundation and Fine-Tuned Chat Models
- A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions
- A Survey of Large Language Models
- Scaling Laws for Neural Language Models
- Scaling Instruction-Finetuned Language Models
- Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism
- A Survey on Multimodal Large Language Models
- Fine-Tuning Language Models from Human Preferences
- Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context
- DeepSeek-V3 Technical Report
- AI model GPT-3 (dis)informs us better than humans
- Gemma: Open Models Based on Gemini Research and Technology
- A Systematic Survey of Prompt Engineering in Large Language Models: Techniques and Applications
- Automated Paper Screening for Clinical Reviews Using Large Language Models
- ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Tools
- Hallucination is Inevitable: An Innate Limitation of Large Language Models
- Baichuan 2: Open Large-scale Language Models
- Between words and characters: A Brief History of Open-Vocabulary Modeling and Tokenization in NLP
- Parameter-Efficient Fine-Tuning for Large Models: A Comprehensive Survey
- Automated scholarly paper review: Concepts, technologies, and challenges
- A Survey on Large Language Models with some Insights on their Capabilities and Limitations
- GPT4 is Slightly Helpful for Peer-Review Assistance: A Pilot Study
- A Comparative Study on Reasoning Patterns of OpenAI's o1 Model
- The AI Review Lottery: Widespread AI-Assisted Peer Reviews Boost Paper Scores and Acceptance Rates
- MOPRD: A multidisciplinary open peer review dataset
- Are We There Yet? Revealing the Risks of Utilizing Large Language Models in Scholarly Peer Review
- MARG: Multi-Agent Review Generation for Scientific Papers
- AI-Driven Review Systems: Evaluating LLMs in Scalable and Bias-Aware Academic Reviews
- Automatic Analysis of Available Source Code of Top Artificial Intelligence Conference Papers
- What Can Natural Language Processing Do for Peer Review?
- Speculative Exploration on the Concept of Artificial Agents Conducting Autonomous Research
- Peer Review as A Multi-Turn and Long-Context Dialogue with Role-Based Interactions
- Reviewer2: Optimizing Review Generation Through Prompt Generation
- Generative Adversarial Reviews: When LLMs Become the Critic
- Evaluating and Enhancing Large Language Models for Novelty Assessment in Scholarly Publications
- Streamlining the review process: AI-generated annotations in research manuscripts
- OpenReviewer: A Specialized Large Language Model for Generating Critical Scientific Paper Reviews
- WDMoE: Wireless Distributed Mixture of Experts for Large Language Models