62 citations · 113 across the 23 of their papers we have counts for
24 papers
Beyond Human-Likeness: Mapping the Scientific Critique Profiles of LLMs and Human Reviewers
Yunhan Yang, Mike Thelwall, Guoxiu He
Large language models (LLMs) are increasingly discussed as tools for peer review, but their value is often assessed through human-likeness, perceived usefulness, or textual overlap…
Should children follow their parents' research paths? Intergenerational research continuity and divergence in academic families
Er-Te Zheng, Xiaorui Jiang, Zhichao Fang +1
How academic advantages are transmitted within families is usually studied as occupational inheritance, but it is not clear whether scholarly research orientations persist across g…
Science discussions of retracted articles on Bluesky: public scrutiny or misinformation spreading?
Er-Te Zheng, Hui-Zhen Fu, Xiaorui Jiang +2
Post-publication peer review (PPPR) has emerged as an important supplement to traditional peer review, with social media playing a growing role in publicising potential problems in…
Will AI be overconfident about academic research findings when reliant on abstracts? (v1)
Mike Thelwall
Large Language Models (LLMs) like ChatGPT, DeepSeek and Gemini seem to be increasingly used for knowledge discovery, information retrieval, and knowledge summaries, including for a…
Do Large Language Models know Which Published Articles have been Retracted?
Mike Thelwall
Large Language Models (LLMs) can be helpful for literature search and summarisation, but retracted articles can confuse them. This article asks three open weights (offline) LLMs wh…
Which stylistic features fool ChatGPT research evaluations?
Kayvan Kousha, Mike Thelwall
Large Language Models (LLMs) have the potential to be used to support research evaluation and have a moderate capability to estimate the research quality of a journal article from…
Can Large Language Models Evaluate Grant Proposal Quality? Revisiting the Wennerås and Wold Peer Review Data
Ulf Sandström, Mike Thelwall
Purpose: Despite the importance of peer review for grant funding decisions, academics are often reluctant to conduct it. This can lead to long delays between submission and the fin…
Can ChatGPT evaluate research environments? Evidence from REF2021
Kayvan Kousha, Mike Thelwall, Elizabeth Gadd
UK academic departments are evaluated partly on the statements that they write about the value of their research environments for the Research Excellence Framework (REF) periodic a…
How is science discussed on Bluesky?
Er-Te Zheng, Xiaorui Jiang, Zhichao Fang +1
Amid the migration of academics from X, the social media platform Bluesky has emerged as a potential alternative. To assess its viability and relevance for science communication, t…
Can Smaller Large Language Models Evaluate Research Quality?
Mike Thelwall
Although both Google Gemini (1.5 Flash) and ChatGPT (4o and 4o-mini) give research quality evaluation scores that correlate positively with expert scores in nearly all fields, and…
Implicit and Explicit Research Quality Score Probabilities from ChatGPT
Mike Thelwall, Yunhan Yang
The large language model (LLM) ChatGPT's quality scores for journal articles correlate more strongly with human judgements than some citation-based indicators in most fields. Avera…
Research quality evaluation by AI in the era of Large Language Models: Advantages, disadvantages, and systemic effects
Mike Thelwall
Artificial Intelligence (AI) technologies like ChatGPT now threaten bibliometrics as the primary generators of research quality indicators. They are already used in at least one re…
In which fields do ChatGPT scores align better than citations with research quality?
Mike Thelwall
Although citation-based indicators are widely used for research evaluation, they are not useful for recently published research, reflect only one of the three common dimensions of…
Can news and social media attention reduce the influence of problematic research?
Er-Te Zheng, Hui-Zhen Fu, Xiaorui Jiang +2
News and social media are widely used to disseminate science, but do they also help raise awareness of problems in research? This study investigates whether high levels of news and…
Is OpenAlex Suitable for Research Quality Evaluation and Which Citation Indicator is Best?
Mike Thelwall, Xiaorui Jiang
This article compares (1) citation analysis with OpenAlex and Scopus, testing their citation counts, document type/coverage and subject classifications and (2) three citation-based…
Estimating the quality of academic books from their descriptions with ChatGPT
Mike Thelwall, Andrew Cox
Although indicators based on scholarly citations are widely used to support the evaluation of academic journals, alternatives are needed for scholarly book acquisitions. This artic…
Journal Quality Factors from ChatGPT: More meaningful than Impact Factors?
Mike Thelwall, Kayvan Kousha
Purpose: Journal Impact Factors and other citation-based indicators are widely used and abused to help select journals to publish in or to estimate the value of a published article…
Research evaluation with ChatGPT: Is it age, country, length, or field biased?
Mike Thelwall, Zeyneb Kurt
Some research now suggests that ChatGPT can estimate the quality of journal articles from their titles and abstracts. This has created the possibility to use ChatGPT quality scores…
Evaluating the Predictive Capacity of ChatGPT for Academic Peer Review Outcomes Across Multiple Platforms
Mike Thelwall, Abdullah Yaghi
While previous studies have demonstrated that Large Language Models (LLMs) can predict peer review outcomes to some extent, this paper builds on that by introducing two new context…
Evaluating the quality of published medical research with ChatGPT
Mike Thelwall, Xiaorui Jiang, Peter A. Bath
Estimating the quality of published research is important for evaluations of departments, researchers, and job candidates. Citation-based indicators sometimes support these tasks,…
In which fields can ChatGPT detect journal article quality? An evaluation of REF2021 results
Mike Thelwall, Abdallah Yaghi
Time spent by academics on research quality assessment might be reduced if automated approaches can help. Whilst citation-based indicators have been extensively developed and evalu…
Evaluating Research Quality with Large Language Models: An Analysis of ChatGPT's Effectiveness with Different Settings and Inputs
Mike Thelwall
Evaluating the quality of academic journal articles is a time consuming but critical task for national research evaluation exercises, appointments and promotion. It is therefore im…
Quantitative Methods in Research Evaluation Citation Indicators, Altmetrics, and Artificial Intelligence
Mike Thelwall
This book critically analyses the value of citation data, altmetrics, and artificial intelligence to support the research evaluation of articles, scholars, departments, universitie…
Can ChatGPT evaluate research quality?
Mike Thelwall
Purpose: Assess whether ChatGPT 4.0 is accurate enough to perform research evaluations on journal articles to automate this time-consuming task. Design/methodology/approach: Test t…