GPTZero is an artificial intelligence AI content detection tool created by Princeton University student Edward Tian. He developed it for teachers after seeing the increased cases of “AI plagiarism.” after the release of ChatGPT
AI plagiarism refers to the use of an AI tool like Chat-GPT to create similar content based on existing information without proper attribution. Many educators and university lecturers have expressed concerns over AI content creation tools like ChatGPT and are worried students would use them on exams.
Tian positions GPTZero as the solution to this problem as it analyzes texts based on their perplexity and burstiness. Perplexity refers to the randomness of a text to a model or how well a language model likes a text. Meanwhile, burstiness refers to how uniform or constant the perplexity is over time compared to varied human-written text.
The app is currently in beta mode, with the creator planning to release a paper with accuracy stats based on data from student journalism pieces.
Because AI content creation is becoming more common, it is up to apps like GPTZero to regulate AI use and protect the integrity of student work. Let’s learn more about this AI detection tool’s features and see if it is a viable solution to AI plagiarism.
Features
To see if GPTZero’s accuracy holds up, we conducted a test using seven text samples generated by Jasper.ai, a popular AI writing tool. For comparison, we’ll show GPTZero’s results alongside Originality.ai’s. But before we show them, it’s important to explain how GPTZero presents results.
GPTZero doesn’t use the probability format like many other AI content detection tools. It just provides a general statement of whether or not AI likely generates the text. Then, the app shows the perplexity score and the burstiness score.
On the other hand, Originality.ai presents the results with percentages. It evaluates how much of a text is original and what percentage is most likely created by AI. Unlike GPTZero, Originality.ai is trained on several language models, namely GPT-3, GPT-3.5, GPT-4 and ChatGPT.
For comparison, we’ll show GPTZero’s perplexity score and statements. Here are the results.
Off the bat, you can tell that GPTZero’s results are quite confusing. The concept of perplexity, while defined clearly by the creator, is difficult to grasp in terms of the numbers shown. We can only go off from the statement that the higher the perplexity, the more likely the text is human-generated.
From our testing, we found that the perplexity score can go as high as 30,000. Aside from the general statements given by GPTZero, this is where we will base our analysis.
GPTZero could not catch 100% detection from any of the samples. The app would show this statement if it did: “Your text is likely to be written entirely by AI.” Meanwhile, Originality.ai showed 100% detection on samples #1 and #2. It also detects more than 90% on three more samples.
On the positive side, GPTZero does highlight phrases that are likely made by AI. Here’s an example:
Overall, GPTZero can detect instances of AI-generated content. However, the presentation of scores should be further improved for educators to truly grasp the gravity of a text’s AI plagiarism. In terms of accuracy and clarity, Originality.ai is the winner in this test.
Since the tool is still in beta mode, it’s unfair to point out its drawbacks at this time. Instead, we can focus on where GPTZero is likely to go. For starters, the app is currently free, which makes it highly accessible to teachers.
The API is also available for those who request it. This means other developers can build on it and integrate it with their own systems.
Edward Tian said from the beginning that the tool was built with academics in mind, unlike many AI content detection tools on the market that are designed for blog posts and marketing copy.
As the team tests GPTZero on more data from student work, the app’s accuracy is likely to increase. Combined with online plagiarism checkers, it can help improve the integrity of student essays and theses.
For now, educators should use the GPTZero sparingly while it is still being developed. As the app reminds users on its page, AI detection is just “one of the many pieces of a holistic assessment of student work.” You can also use plagiarism detection tools that scan documents against billions of public content online.
GPTZero is a free and decent tool for catching AI plagiarism. However, it has a long way to go to catch up with competitors like Originality.ai AI Checker, which detects not only AI content but also plagiarism.
However, it is important to note that GPTZero is still in beta mode. As the Princeton team continues to work on the app, we can expect its accuracy and features to improve. GPTZero’s metrics provide a decent starting point for teachers looking to detect AI plagiarism in the meantime.
