Ai-generated Research Papers Published On Arxiv Post Chatgpt Launch

We did a comprehensive analysis on 13,000 research papers sourced from ARXIV, Papers. This study delves into the question of How Artificial Intelligence is Shaping Research Papers and by extension Academic Writing.

Summary/key Findings:

1. Since chat-GPT's launch, a 72%  boost in papers potentially written by AI.
2. A significant increase in AI usage in computer science could be seen when comparing 2 mathematics and physics.

Study Method And Data

About Arxiv

ARXIV is a free-to-access repository that hosts a vast collection of research papers across various disciplines. Researchers worldwide use ARXIV to share and discover cutting-edge scientific findings, fostering collaboration and open access to knowledge within the global scientific community.

PAPER SUBMITTED BY CATEGORIES ON ARXIV AFTER YEAR 2019 - Source

Papers of top 3 categories published after the year 2019.

The Originality.ai AI detector was used to determine if the content was AI or human-generated. The accuracy of the detector has been validated in this study here and here. It has been shown to have a false positive rate of approximately ~2.5% which is evident in the graphs below. In the months in 2019 before Generative AI existed the tool identified ~3% of the papers as AI generated which are false positives.

Study

The surge in paper publication on Arxiv suggests a hypothesis that: “The widespread adoption of a writing tool, such as Chat-GPT, is a driving force”. To explore this, we downloaded 60,000 papers from ARXIV using ARXIV APIs, selecting the top 150 papers monthly from the year 2019 in computer science physics and mathematics. We created a data set of 13,000 papers. Leveraging Originality.AI’s AI detection tool, we obtained a score from 13,000 papers, showcasing A notable rise in AI used for writing. This approach sheds light on the increased impact of AI tools in the academic writing landscape.

Originality.AI’s AI detector analyzed texts, distinguishing between human-generated and AI-generated content. We computed the percentage of papers that were AI-generated every month, tracking the dynamic shifts in the percentage of AI-generated text over time. Notably, the introduction of Chat-GPT in November 2022 triggered a significant surge in AI-generated papers, escalating the number of papers identified as AI in November 2022 from 3.61% to 6.22% in Nov 2023. 

Building upon this observation, it becomes evident that the hypothesis suggesting the widespread adoption of a writing tool, such as ChatGPT, is a driving force, holds true. The positive surge in papers featuring higher AI scores post the ChatGPT launch serves as compelling evidence supporting the notion that innovative AI tools are indeed instrumental in shaping the trajectory of academic writing.

This trend raises a second question: “Does usage of AI writing affect papers of all categories?”

To explore the impact of AI writing on papers across categories, computer science mathematics and physics we conducted A detailed analysis of AI scores.

This comprehensive analysis sheds light on the prevalence of AI influence in diverse academic domains. Specifically, in the realm of computer science, the influence of AI-generated content stands out prominently. AI writing has made a notable impact within the computer science domain especially after the launch of ChatGPT. This observation provides compelling evidence supporting the hypothesis that the usage of AI writing does affect papers across diverse categories, with computer science emerging as a prominent domain where this influences particularly highest.

Summary

The growing prevalence of AI in research papers raises concern about the potential impact on the authenticity and originality of research. As more papers rely on AI-driven content generation, questions arise about the ability to maintain academic integrity and the potential for automated systems to inadvertently introduce biases or inaccuracies.