Researchers at UPenn, University College London, King’s College London, and Carnegie Mellon University recently completed the most complete study yet to evaluate AI detector efficacy - RAID: A Shared Benchmark for Robust Evaluation of Machine-Generated Text Detectors (study).
Originality.ai’s AI detector was the most accurate detector in the study!
The study looked at 12 AI detectors, 11 different text-generating LLMs (like ChatGPT), and 11 types of adversarial attacks (like paraphrasing), resulting in a dataset of over 6 million text records. This is the most robust evaluation of AI text detectors to date, and our own AI detection accuracy study is likely the second most in-depth study.
Below is a summary of the study, including the key findings related to Originality.ai’s industry-leading performance and a couple of our weaknesses identified in the study that we are excited to address.
Study Details:
Study: https://arxiv.org/abs/2405.07940
It is important to note these accuracy scores reflect a 5% false positive threshold, more on what that means below.
Originality.ai’s model 2.0 Standard was used for these results, however we would expect model 3.0 Turbo (which was released 1 month after the authors used Originlaity.ai) to outperform 2.0.
11 AI models were used to generate a non-adversarial dataset and it was evaluated against all AI detectors. These included the most popular AI models such as ChatGPT, Llama, Mistral and GPT-4.
Originality.ai was the most accurate across the test achieving 98.2% accuracy on ChatGPT content and an average of 85% across all 11 AI models.
Of the 11 adversarial (adversarial means trying to make undetectable AI content) techniques used Originality.ai was the most accurate of the 12 detectors in 9 of the 11 tests, 2nd in 1, and performed poorly in 2 rarely used bypassing techniques.
“Originality.ai Rank” shows how we performed vs the 12 AI detectors for each of the Adversarial Bypassing Techniques.
Clearly, Originality.ai’s AI Detection needs to improve on the Homoglyph and Zero-Width AI bypassing strategies, but was the leading AI detector on the other more common bypassing techniques.
The authors looked at how AI detectors performed on different types of content (news articles vs poems etc).
Originality.ai was the most accurate across 5 of the 8 types of content “domains” and the 2nd most accurate in the other 3 domains.
Important to note is that the study did not include the domain/type of content we at Originality.ai focus most on, i.e. web content/marketing content.
The most common method used in trying to make AI content undetectable is “Paraphrase Plagiarism”. The strategy uses a paraphrasing AI like Quillbot to change words in an attempt to bypass AI detection tools. This is the strategy used by common tools such as Undetectable.AI.
Originality.ai performed uniquely well on this common adversarial bypassing strategy achieving 96.7% accuracy while the average accuracy from other detectors was 59%.
In the context of text detection, a false positive occurs when a detector incorrectly labels a piece of human-written text as machine-generated. False positives can occur for several reasons listed here. The False Positive Rate (FPR) is the percentage of these incorrect labels out of all the human-written texts evaluated. For instance, if a detector examines 100 human-written texts and incorrectly labels 5 of them as machine-generated, the FPR would be 5%.
A 5% FPR threshold is a benchmark used to ensure a balanced evaluation of text detectors. It was selected by the authors of this study.
Here’s why it matters for this study:
This is the best study by far that looks to address if AI detectors work and what their limitations are.
I would have several comments for the authors if the study is continued moving forward or for future studies …
Cover More Relevant Domains:
Most users of AI detectors fall into 2 types of content… Web Content and Academia.
The domains that were selected for this study included essentially no web content (Abstracts, Books, News, Poetry, Recipes, Reddit, Reviews and Wiki). Our detector is specifically trained on web content and it would have been great to see that domain included. The societal importance of being able to detect AI-generated poems is lower than the importance of being able to detect AI-generated news.
False Positive Rate Threshold Makes Sense But Incomplete for Users:
This study standardizes on a FPR threshold of 5%. However, there is use cases where a VERY low FPR rate is required and use cases where a higher false positive rate is acceptable if it means very very little AI can get past the detector. Ultimately the users of AI detectors will be the ones who want to make a decision on what the right trade-off between correctly identifying AI as AI and correctly identifying human as human is. We ask to consider other methods of displaying detector efficacy instead of just accuracy with an FPR threshold including confusion matrix and corresponding efficacy scores (F1, TPR, FPR, etc).
AI Edited Dataset:
One dataset that is missing that will become more prevalent is AI-edited content (whether that be lightly AI-edited or heavily AI edited). With AI editing tools increasing the amount of AI they inject into an edited article, the ability to identify lightly edited AI content is important. For a small analysis here is the impact of Grammarly editing on AI detection.
Our AI detector performed poorly on 2 bypassing strategies. Homoglphyh and Zero-Width-Space attacks. These two strategies work by creating text that seems readable to humans but is less readable to machines. We strongly recommend against using this strategy to try and trick Google’s AI detection.
Originality.ai focuses on providing an accurate AI detector for web publishers, so focusing on Homoglyph bypassing and Zero-Width Space had not been a priority for us (these aren’t machine-readable and therefore not a good strategy for web publishing). However, in order to ensure we are the most accurate across all bypassing strategies, we look forward to having our AI research team close these 2 smaller weaknesses in our detector!
We will shortly have 3 models for users to choose from depending on their desired tradeoff between false positives (calling human text AI) and false negatives (calling AI content human).
This is well represented on the graph Figure 4 from the study and overlaying our target ranges for our 3 AI detection models.
Our model showed leading performance for a given false positive rate until it got near 1%. Our Lite model will have industry leading low false positive rates.
It is great to see a high-quality efficacy study after some flawed early studies provided misinformation about AI detectors and their efficacy. Thank you, researchers - this study will help people!
It is most exciting to see our approach with our Blue Team (building detectors) and Red Team (trying to beat our detector) is working. Our entire AI research team should be very proud of the industry leading results they are producing.
If anyone is interested in running a study that includes Originality.ai, we are happy to make credits available and have our own open source tool available for evaluating detector efficacy for free.
We believe that it is crucial for AI content detectors reported accuracy to be open, transparent, and accountable. The reality is, each person seeking AI-detection services deserves to know which detector is the most accurate for their specific use case.