As AI content generation continues to advance, more brands are adding AI-based features and AI tools to their offerings.
One brand that’s recently gone down this avenue is Grammarly. In August 2024, Grammarly released an announcement that it’s adding an AI content detector to its suite of tools. This news comes despite Grammarly’s previous stance that AI detection tools are ‘unreliable.’
In this review, we’ll test the Grammarly AI Content Detector to see if it’s reliable.
Grammarly initially launched into the mainstream as a grammar-checking tool, offering users a free plugin that helped them ensure their grammar was correct, their word usage was as effective as possible, and the structure of their content matched their intent.
Since then, the tool has become a go-to option for marketers, small business owners, and writers.
Despite their apparent distrust of AI content detectors as ‘unreliable’ (see the above screenshot), Grammarly has decided to create their own tool. Hopefully, this is a sign that the company is coming to an understanding of the importance of AI detection.
There is no doubt that AI content generation has its benefits. It can be an incredible tool for creating content briefs or helping with content ideation.
However, AI text also creates problems, including plagiarism, a lack of transparency, and problems with potentially getting marked as spam by Google.
With that in mind, industry leaders, business owners, and marketing experts are looking for a reliable AI detector to gain clarity for content production and content consumption.
In our evaluation of the Grammarly AI Detector using the RAID AI detection accuracy study dataset, it is clear that the detector is highly susceptible to bypassing techniques, especially when tested with adversarial attacks.
The RAID dataset, designed to test the robustness of AI content detectors, showed critical weakness in Grammarly’s AI detection capabilities.
The RAID dataset (Robust AI Detection) is a benchmark specifically designed to evaluate the robustness of AI-generated text detectors.
It includes a wide variety of adversarial attacks and multiple AI models making it an effective tool to test a detector's ability to identify manipulated AI-generated content. The dataset's diversity in text manipulations and generative models helps expose weaknesses in AI detection tools, highlighting their susceptibility to adversarial techniques.
In conclusion, Grammarly’s AI detection might be able to catch obvious instances of AI-generated text. However, it’s easy to bypass with minimal effort. On the other hand, Originality.ai’s AI detector demonstrated a strong, robust performance as a more reliable AI content detection tool.
As part of this review, we tested Grammarly’s AI detector with three different forms of AI content.
Step 1: We started by building out detailed prompts for each test before posting them into ChatGPT, one of the most frequently used AI content generators.
Step 2: From there, we took the exact copy output from ChatGPT for each test and pasted the results into Grammarly’s AI content detector (without making any changes) to help show whether or not the tool can detect AI-generated content.
Step 3: For comparison, we also took that exact same copy and pasted it into the Originality.ai AI content detector, allowing us to see how each tool compares with the same content input.
The three types of content used in this test include:
Prompt: "Write a concise blog post on the benefits of investing in a summerhouse as a garden office, targeting professionals and business owners.
Key points:
Keep the tone professional, clear, and persuasive."
Originality.ai AI Detector: 99% AI & 1% Human-written
Grammarly AI Score: 99% AI & 1% Human-written
Prompt: "Write a research paper exploring the benefits of using a summerhouse as a garden office, targeting a scholarly audience.
Focus Areas:
Structure the paper to provide a clear, data-backed analysis of each point."
Originality.ai AI Detector : 100% AI & 0% Human-written
Prompt: "Craft a promotional email for a summerhouse company offering a 10% discount on first orders over $4,000. The email should emphasize the benefits of adding a summerhouse to a garden, such as creating a functional and stylish space for work or leisure. Highlight the limited-time nature of the offer and encourage immediate action. The tone should be warm, persuasive, and focused on the value and quality of the summerhouses, with a clear call to action to use the discount code at checkout."
Originality.ai AI Detector : 100% AI & 0% Human-written
Grammarly AI Score: 71%% AI & 29% Human-written
From the results of this test, the Grammarly AI content detector demonstrated strong capabilities when it was faced with some of the most basic AI content detection, such as marketing content for blog posts.
However, when it came to slightly more complex content, such as promotional emails, where language patterns differ, and the text is shorter, Grammarly’s AI detector struggled compared to Originality.ai.
Searching for more AI detector reviews? Check out our guide on the Best AI Content Detection Tools Reviewed.
Want some additional insight into the efficacy of Originaltiy.ai’s AI detector, check out our meta-analysis of third-party studies.
Looking for further insight into Grammarly? Check out our collection of Grammarly resources:
Grammarly’s AI content detector is free, and you don’t need a Grammarly account to use it. However, there is a word count limit for each submission.
Like other AI content detection tools, Grammarly’s AI detector is designed to identify AI-generated content and help users understand what content is human-written text and what isn’t.
When used sparingly, in the right context, and with transparency, AI writing tools have the power to be extremely beneficial, helping improve productivity and efficiency. However, when used to try and pass off AI work as your own, ethics comes into question.
