Detection of Fake Generated Scientific Abstracts

Abstract

The widespread adoption of Large Language Models and publicly available ChatGPT has marked a significant turning point in the integration of Artificial Intelligence into people's everyday lives. The academic community has taken notice of these technological advancements and has expressed concerns regarding the difficulty of discriminating between what is real and what is artificially generated. Thus, researchers have been working on developing effective systems to identify machine-generated text. In this study, we utilize the GPT-3 model to generate scientific paper abstracts through Artificial Intelligence and explore various text representation methods when combined with Machine Learning models with the aim of identifying machine-written text. We analyze the models' performance and address several research questions that rise during the analysis of the results. By conducting this research, we shed light on the capabilities and limitations of Artificial Intelligence generated text.

References in corpus (8)

• Efficient Estimation of Word Representations in Vector Space
• word2vec Explained: deriving Mikolov et al.'s negative-sampling word-embedding method
• ScispaCy: Fast and Robust Models for Biomedical Natural Language Processing
• How Close is ChatGPT to Human Experts? Comparison Corpus, Evaluation, and Detection
• Release Strategies and the Social Impacts of Language Models
• DetectGPT: Zero-Shot Machine-Generated Text Detection using Probability Curvature
• AI vs. Human -- Differentiation Analysis of Scientific Content Generation
• CoCo: Coherence-Enhanced Machine-Generated Text Detection Under Data Limitation With Contrastive Learning