AI Writing
Fictitious.AI Detector Review — Does It Work?
As part of our AI detector review series, we tested the Fictitious.AI AI content detector to see whether it could accurately identify AI text. These are our results.
Ever since the launch of ChatGPT, there have been concerns about the impact of AI on the future (and present) of content creation.
After all, if you can't tell the difference between human-written text and AI-generated content, it can lead to issues across countless industries, from content marketing to education.
That's where Fictitious claims it can help by providing an AI detector designed to identify AI-generated academic content.
So, as part of our AI detector review series, we’re reviewing the Fictitious.AI AI detector to see just how accurate it is.
Fictitious.AI is a plagiarism detection platform that offers a tool designed to detect AI-written content within academic assignments.
The company claims to identify AI-generated content produced by AI tools, such as ChatGPT and GPT-4, to prevent AI writing from impacting academic integrity.
While many AI content detection tools focus on detecting AI content in marketing collateral, Fictitious.AI takes a stance against AI-generated text in academic content.
Understandably, there are significant concerns over the future of academic integrity, especially if humans aren’t as good as they think they are at identifying AI-generated content.
If left unchecked, artificial intelligence could generate content that leads to incorrect academic results, which could have serious effects.
Not to mention the fact that it could lead to factually incorrect papers getting published with AI hallucinations.
As a result, many within the academic world are searching for an AI content detection tool that can identify human-generated content and AI-generated text to help maintain academic integrity in an increasingly digital world.
With that in mind, we tested Fictitious.AI to see how it compared to the Originality.ai AI detector.
Step 1: We created academic-style content in ChatGPT and then pasted it directly into both AI detectors without making any amends to see whether or not they could tell that it was created by an AI tool.
Step 2: We chose ChatGPT 3.5 as it’s one of the most used tools for AI-generated content. Additionally, we opted for academic content as it’s what Fictitious.AI claims their detector is designed to tackle.
The three forms of academic content we focused on included:
Prompt: "Please write me a detailed academic essay on the importance of using solar power for the long-term impact on our climate.
This academic essay should be written from the perspective of someone who has studied the subject for decades and is writing a study for their peers, who are also highly experienced in the field.
The essay should go into great detail, leaving no stone unturned, and should seem like a real human expert has written it."
Originality.ai AI Detector — 2.0.1 Standard Score: 100% AI & 0% Human-written
Fictitious.AI Score: 67.11% AI & 33.89% Human-written*
*Note: Fictitious.AI detects AI in sections. For comparison purposes, we noted the average percentage across the entire piece.
Prompt: "Now use the essay you have generated above and the persona you have taken to write it, and create a research paper in a similar style. This research paper should be even more detailed, and should focus on a proposal that every new home should be built with solar panels automatically installed."
Originality.ai AI Detector — 2.0.1 Standard Score: 100% AI & 0% Human-written
Fictitious.AI Score: 38.84% AI & 61.16% Human-written*
*Note: Fictitious.AI detects AI in sections. For comparison purposes, we noted the average percentage across the entire piece.
Prompt: "Now tweak this content into a real-world case study, assuming that the research paper was reviewed and approved. Provide a detailed account of how the project has gone, including specific findings and results."
Originality.ai AI Detector — 2.0.1 Standard Score: 100% AI & 0% Human-written
Fictitious.AI Score: 55.77% AI & 44.23% Human-written*
*Note: Fictitious.AI detects AI in sections. For comparison purposes, we noted the average percentage across the entire piece.
At first glance, Fictitious.AI doesn’t seem as effective at accurately identifying AI content based on the averages of its scores across the entire piece.
For transparency, there are some mitigating factors to be aware of:
It is also worth noting that these tests were executed on the tech demo version of the software, as the more detailed version of the tool is locked behind a protective digital wall.
As a result, it is difficult to conclusively identify the accuracy of the Fictitious.AI detection tool and further testing is needed.
Interested in learning more about AI detection and AI detection tools? Check out Originality.ai’s AI detector accuracy review. Then, read a meta-analysis of third-party studies.
You can use the Tech Demo Mode of Fictitious.AI for free, but you need to request full access to get the best version of the tool.
Fictitious.AI is an AI detector that focuses on identifying the use of AI content in academic writing.
In some instances, AI-generated content is not an issue. However, when it comes to academic work, it’s essential to clarify whether an assignment permits AI before using it.