AI Studies
Are AI Checkers Biased Against Non-Native English Speakers? A Response to a Flawed Stanford Study
Flawed study claims AI checkers are biased? Originality.AI debunks claims and offers solutions.
This study looks at a flawed Stanford Study that incorrectly showed that AI checkers are biased against non-native english speakers.
AI content generation is, without a doubt disrupting and revolutionizing entire industries online. Many experts, professors and publishers rely on AI content checkers in order to distinguish between what is human-written and what’s written by AI both online and academically.
With the launch and ongoing development of groundbreaking tools like Originality.AI, there are going to be numerous case studies conducted to evaluate the efficacy of these types of tools – as there should be.
In fact, we welcome any and all AI detection accuracy tests to further help educate users on the limitations of AI checkers and how to properly use them. However, the process in which they are conducted can sometimes be questionable.
We have in fact made and open-sourced an AI detector efficacy research tool to let anyone evaluate AI checkers efficacy on their own dataset.
A study examining the possible biases of AI content detectors against non-native English speakers was published by Stanford scholars.
After reviewing the results, we noticed a number of flaws that needed to be addressed - especially because this study was being referenced by multiple sources to falsely claim that AI checkers are biased.
We decided to conduct our own study with a larger and more thorough publicly available dataset. Below are our findings.
In July 2023, a Stanford paper was released discussing the rapid growth of generative language models like ChatGPT and their potential risks, including the spread of fake content and cheating. This was in response to many educators being concerned about their ability to detect AI in students' work due to accuracy issues.
In the study, the authors stated that AI checkers exhibit bias against non-native English speakers, misclassifying their writing as AI-generated. Their data showed that while detectors were able to accurately classify US student essays, they incorrectly labeled more than half of TOEFL essays as AI-generated, with an average false positive rate of 61.3%.
This bias may lead to unintended consequences, like the marginalization of non-native speakers. Most AI content detectors rely on measures like text perplexity, which can be influenced by linguistic variability.
To mitigate bias, the authors enhanced the vocabulary of non-native writing samples. They also discovered that AI checkers could be easily bypassed by manipulating prompts, which also raised questions about their effectiveness.
The implications of bias in AI checkers include potential discrimination on social media, limitations for non-native researchers, and false accusations in education. The study suggests caution in using these detectors in evaluative settings, comprehensive evaluation with diverse samples, and inclusive conversations to define acceptable use.
Overall, the Stanford study calls for developing more robust and fair AI checkers, emphasizing inclusivity and trust, and engaging all stakeholders in defining ethical AI usage in various contexts (Source). These are worthy ideals that Originality strives for with our content detection tool.
The Stanford study received an overwhelming reaction by the education, writing, and AI community - that being that AI content detectors were not accurate and therefore could not be trusted to detect AI content.
We reviewed the data ourselves and found multiple flaws with the study, which ultimately put their findings into question. Let's go over these one at a time.
The first problem in the paper concerns the small sample size of 91 TOEFL essays that were taken from a student forum. There may be questions regarding the findings' generalizability if a small sample size was used.
The range of non-native English writing may not be sufficiently reflected in TOEFL essays from a student forum. It's crucial to have a larger and more diverse dataset to draw more robust conclusions about the performance of AI checkers on non-native English writing.
The second flaw is the comparison against 8th-grade US essays. This introduces a significant confounding variable, as the age group and educational level of individuals completing TOEFL exams differ substantially from those in 8th grade. The writing styles, vocabulary, and linguistic complexity can vary widely between these two groups.
This comparison might not accurately reflect the nuances of non-native English writing, as TOEFL essays are expected to adhere to a more advanced linguistic standard.
The third flaw involves the misclassification of "GPT-4 polished articles" as human-written content. This error calls into doubt the validity of the assessment procedure.
The validity of the study's conclusions is called into question if the detectors are unable to reliably distinguish between information that is generated by GPT and content that is authored by humans. The validity of assertions about bias against non-native English speakers is affected by sample misclassification.
The fourth flaw is the absence of updated information on the current performance of detectors. In light of the swift progress being made in AI technology, particularly in language models, it is imperative to present current findings to guarantee the pertinence and precision of the research outcomes.
The Stanford study does not appropriately depict the present limitations and capabilities of AI checkers in identifying AI-generated content in the absence of up-to-date data. Below is an example of why this is true.
The Stanford case study was based on version 1.1 of Originality.AI. We have made significant changes to our AI checker. See version history and increases in accuracy here - https://originality.ai/blog/ai-content-detection-accuracy
We ran our tool against this exact data set and feel that the case study should be modified to reflect that the latest Originality.AI Model (1.4) detected AI-Written Content at 100% for all AI data sets.
Had our latest model been used for the study, the charts would have looked MUCH different:
The identified flaws highlight potential limitations in the methodology and execution of the study. While the paper claims bias against non-native English authors based on misclassification rates, these flaws suggest the need for a more rigorous and comprehensive analysis. AI checker bias is a serious issue that has to be carefully considered and researched, particularly in regard to bias against non-native English speakers.
Additional research needs to be done to address the points raised in order to guarantee the validity and dependability of its conclusions and successfully address this problem. Additionally, a more inclusive approach to dataset selection, accurate detector evaluation, and ongoing updates on performance can contribute to a more nuanced understanding of bias in AI checkers.
To address the flaws in the Stanford study, we conducted our own analysis, taking into consideration all of the points made above. Originality.AI's evaluation significantly outperforms the study conducted by the Stanford scholars in several key aspects, rendering the latter's findings invalid. Let's break down our approach and findings.
Unlike the Stanford study, which used a small sample size of only 91 TOEFL essays from a student forum, Originality.AI utilized a much larger dataset of over 1,500 essay samples collected from Kaggle and other sources. This extensive dataset ensures a more comprehensive representation of non-native English writing, enhancing the reliability and generalizability of the findings.
The table below shows the datasets used by Originality.AI with brief information about each.
The Stanford study compared TOEFL essays against 8th-grade US essays, introducing a confounding variable due to differences in age, education level, and writing standards. In contrast, Originality.AI ensured a fair comparison by using similar IELTS essays. This approach allows for a more accurate assessment of detector performance across different writing styles and linguistic complexities.
Originality.AI incorporated an AI content scoring system to analyze the combined IELTS essays. Here's a breakdown of the steps involved:
This process provided insights into the authenticity of the essays using AI content scoring and visualizing the performance of the classification model as confusion matrix
The application of a confusion matrix to determine the originality of text is known as confusion matrix analysis. Originality.AI's assessment provides numerical insights into how well its detector performs. The detector demonstrates exceptional accuracy in discerning between AI-generated and human-written content.
Of the total essays analyzed, the Originality.AI detector accurately identified 1,526 as human-written and incorrectly labeled only 81 as AI-generated. This shows a True Negative Value of 94.96% and a False Positive Value of 5.04%.
This result of a 5.04% false positive rate is significantly lower than Stanford’s average false positive rate of 61.3%. From this we can determine that:
Overall, Originality.AI's approach addresses the flaws identified in the Stanford study, offering a more convincing and explanatory assessment of bias in AI checkers.
By utilizing a larger dataset and addressing the other flaws of the Stanford study, Originality.AI provides more robust evidence and reliable results, effectively invalidating the findings of the Stanford scholars' article.
To read more about Originality.AI’s accuracy ratings, you can check out our in-depth Detection Score Accuracy article here.
There have been additional studies exploring if there is bias against non-native english speakers.
Specifically: https://www.sciencedirect.com/science/article/abs/pii/S0360131524000848?
Detecting ChatGPT-generated essays in a large-scale writing assessment: Is there a bias against non-native English speakers?
Their findings were similar to our own showing no bias in AI detectors against non native english speakers.
"Results showed that our carefully constructed detectors not only achieved near-perfect detection accuracy, but also showed no evidence of bias disadvantaging non-native English speakers."
It’s important to note the differences in false positives in terms of marketing versus academic use. We have repeatedly emphasized that Originality.AI is not for academic use.
The Data that our AI has been trained on is closely tied to online content for the purpose of ranking on search engines- NOT academic papers. On our signup page it specifically says that it is built for publishers, agencies and writers, not for students….If we deploy an academic focused solution we will train our AI detection on more TOEFL essays to avoid this problem. But for now our stance is that we are not for academic use and are built specifically for serious content marketers and SEOs.
As one of the most popular AI writing detection services, we are continually developing our services to have the lowest false positive detection rate of any AI content detector. With our most recent update, we now have this number at less than 2.5%. See the study and comparison with other tools along with our latest GPT-4-trained detection model by clicking here.
When it comes to non-native English speaker texts being erroneously flagged as false positives, we’re looking deeply into the causes and correlations to find the underlying causes. One of the theses we are investigating is what we call “cyborg writing”.
Cyborg writing happens when a writer uses too many writing assistant tools (many of which are powered by AI). For example, if a writer uses autocorrect while relying extensively on a grammar tool and runs their content through an outlining or content optimization tool, all of these tools leverage AI to some extent and that could be the underlying reason for these false positives.
Even if the student or content creator writes the words themselves, submitting it through different degrees of filtering and assistance can leave tell-tale AI “tracks” that systems trained to detect these tracks (like Originality.AI) will pick up.
But is this more of an issue among non-native English speakers? Or is it simply a more nuanced question of “What level of computer-aided assistance is allowed before something no longer becomes a writer’s original work?” We believe there is no easy, one-size-fits-all solution to this and that the answer will depend on the situation.
For this reason, even over the long-term, AI checkers will never be able to provide a perfectly clean 100% solid track record with zero false positives. However, being able to combine AI detection with the ability to visualize the content creation process is one of the main reasons why we built our free Google Chrome AI Detection Extension. This allows someone to see the creation of a Google Doc in order to prove that the writer did indeed create the content, rather than copying and pasting it from ChatGPT or another AI writing service.
With all of these points in mind, and considering that version 2.0 of Originality.AI accurately predicted AI-written (and human-written) content at nearly 100% across all data sets, we would like to invite the authors of this paper to rerun their data on our updated version and see the results firsthand for themselves.
Have you seen a thought leadership LinkedIn post and wondered if it was AI-generated or human-written? In this study, we looked at the impact of ChatGPT and generative AI tools on the volume of AI content that is being published on LinkedIn. These are our findings.
We believe that it is crucial for AI content detectors reported accuracy to be open, transparent, and accountable. The reality is, each person seeking AI-detection services deserves to know which detector is the most accurate for their specific use case.