In the constantly evolving realm of AI generated content the veracity of information is of utmost importance. With a couple of fact checking solutions available, discerning their efficacy becomes crucial. Originality.ai, revered for its transparency and accuracy in AI content detection had recently ventured into the domain of fact checking but how does our solution stack up against well established giants like ChatGPT or emerging contenders like Llama-2? This study aims to answer this question.
Our text compare tool is a fantastic, lightweight tool that provides plagiarism checks between two documents. Whether you are a student, blogger or publisher, this tool offers a great solution to detect and compare similarities between any two pieces of text. In this article, I will discuss the different ways to use the tool, the primary features of the tool and who this tool is for. There is an FAQ at the bottom if you run into any issues when trying to use the tool.
What makes Originality.ai’s text comparison tool stand out?
Keyword density helper – This tool comes with a built-in keyword density helper in some ways similar to the likes of SurferSEO or MarketMuse the difference being, ours is free! This feature shows the user the frequency of single or two word keywords in a document, meaning you can easily compare an article you have written against a competitor to see the major differences in keyword densities. This is especially useful for SEO’s who are looking to optimize their blog content for search engines and improve the blog’s visibility.
Ways to compare
File compare – Text comparison between files is a breeze with our tool. Simply select the files you would like to compare, hit “Upload” and our tool will automatically insert the content into the text area, then simply hit “Compare” and let our tool show you where the differences in the text are. By uploading a file, you can still check the keyword density in your content.
URL compare
Comparing text between URLs is effortless with our tool. Simply paste the URL you would like to get the content from (in our example we use a fantastic blog post by Sherice Jacob found here) hit “Submit URL” and our tool will automatically retrieve the contents of the page and paste it into the text area, then simply click “Compare” and let our tool highlight the difference between the URLs. This feature is especially useful for checking keyword density between pages!
Simple text compare
You can also easily compare text by copying and pasting it into each field, as demonstrated below.
Features of Originality.ai’s Text Compare Tool
Ease of use
Our text compare tool is created with the user in mind, it is designed to be accessible to everyone. Our tool allows users to upload files or enter a URL to extract text, this along with the lightweight design ensures a seamless experience. The interface is simple and straightforward, making it easy for users to compare text and detect the diff.
Multiple text file format support
Our tool provides support for a variety of different text files and microsoft word formats including pdf file, .docx, .odt, .doc, and .txt, giving users the ability to compare text from different sources with ease. This makes it a great solution for students, bloggers, and publishers who are looking for file comparison in different formats.
Protects intellectual property
Our text comparison tool helps you protect your intellectual property and helps prevent plagiarism. This tool provides an accurate comparison of texts, making it easy to ensure that your work is original and not copied from other sources. Our tool is a valuable resource for anyone looking to maintain the originality of their content.
User Data Privacy
Our text compare tool is secure and protects user data privacy. No data is ever saved to the tool, the users’ text is only scanned and pasted into the tool’s text area. This makes certain that users can use our tool with confidence, knowing their data is safe and secure.
Compatibility
Our text comparison tool is designed to work seamlessly across all size devices, ensuring maximum compatibility no matter your screen size. Whether you are using a large desktop monitor, a small laptop, a tablet or a smartphone, this tool adjusts to your screen size. This means that users can compare texts and detect the diff anywhere without the need for specialized hardware or software. This level of accessibility makes it an ideal solution for students or bloggers who value the originality of their work and need to compare text online anywhere at any time.
In the constantly evolving realm of AI generated content the veracity of information is of utmost importance. With a couple of fact checking solutions available, discerning their efficacy becomes crucial. Originality.ai, revered for its transparency and accuracy in AI content detection had recently ventured into the domain of fact checking but how does our solution stack up against well established giants like ChatGPT or emerging contenders like Llama-2? This study aims to answer this question.
Summary of Findings:
The article compares 6 AI models for fact checking: Originality.ai, GPT-4, GPT-3.5, CodeLlama-34b, Llama-2-13b, and Llama-2-70b.
A dataset of 120 recent facts (60 true, 60 false) was created and fed to each model to test accuracy.
Originality.ai’s Fact Checking Aid has the highest accuracy at 72.3%. GPT-4 was second at 64.9%.
GPT-4 and CodeLlama-34b had high rates of returning "unknown", making them unreliable for fact checking.
Originality.ai stood out with 0% "unknowns" and low error rate, indicating reliability.
When fact checking a generated article, Originality.ai caught more false facts than GPT-4 and Llama-70b.
Originality.ai's strength is fact checking for content creators, providing sources and explanations.
Originality.ai performed well on recent general knowledge facts and aims to continue improving its fact checking capabilities.
What is Fact-checking?
Fact-checking is the rigorous process of verifying the accuracy and authenticity of information presented in various forms of content whether that be news articles, blogs, speeches or even social media posts. In an age where information can be shared at lightning speed the spread of misinformation can have profound consequences, from influencing public opinion to endangering public health and safety.
The importance of fact checking is multifold:
Upholding trust: Reliable information forms the basic foundation of trust between content creator, news organizations and their audience. Fact-checking ensures that the information shared is accurate maintaining the credibility of these entities.
Informed decision making: Accurate and up to date information enables individuals to make informed decisions about life based off of the content found across the media.
Preventing misinformation spread: Fact-checking acts as a filter preventing the spread of false narratives and misconceptions that can potentially have an overwhelming negative impact on the public.
Promoting accountability: It holds public figures, journalists, and content creators accountable for their statements, ensuring they are responsible in their communications and discouraging the spread of false information.
LLM Hallucinations
When we discuss fact checking using LLMs we would be amiss to not discuss hallucinations which are defined as “a confident response by an AI that does not seem to be justified by its training data.”1
Here are the hallucination rates for some of the popular LLMs found from Anyscale
Model
Accuracy (%)
Llama-2-13b
58.7
Llama-2-70b
81.8
GPT-3.5-turbo
67.0
GPT-4
85.5
The Methodology
Crafting the dataset: At the heart of this study lies the dataset, a carefully selected collection of 120 different facts. Let’s break down the composition:
True facts (60 items): These statements grounded in reality were gathered from trustworthy and reliable sources.
False facts (60 items): These statements were crafted to oppose the true facts, theses statements while seeming credible are fundamentally flawed.
It’s worth noting that ALL the facts contained within this data set are facts from 2022-onwards.
Each of these 60 facts was fed into the AI-powered fact checking solutions, Originality.ai, GPT-3.5-turbo, GPT-4, Llama-2-13b, Codellama-34b and Llama-2-70b.
The main goal?
To measure the accuracy of each tool in distinguishing fact from fiction
Gathering the Responses
For each fact checked we logged the response from the tool. The responses fell into one of 3 categories
True: The tool marked the fact as true
False: The tool marked the fact as false
Unknown: The tool was unable to verify the fact true or false
Error: The tool did not provide a response
In the scenario involving Originality.ai, when the provided explanation was at odds with the isTrueClaim value, the explanation was regarded as the accurate source of information.
Analysis
After collecting the data the primary metrics under the microscope were:
Accuracy: The ration of facts correctly identified as true or false.
False positives: Occasions when False facts were incorrectly identified as True.
False negatives: Situations where True facts were incorrectly identifies as False.
Results & diagrams:
Model
Accuracy(%)
Unknowns
Errors
Originality.ai
72.3
0
2
GPT-4
64.9
43
0
GPT-3.5
58.6
4
0
CodeLlama-34b
58.6
33
0
Llama-2-70b
55.2
0
4
Llama-2-13b
55.0
11
0
Accuracy = (Number of correct predictions/Total number of predictions) x 100
Unknown = The model return the result ‘unknown’
Error = The model returned an error
Analysis of results:
In an analysis of the 6 AI models on a dataset of 120 facts, Originality.ai achieved the highest accuracy at 72.3%. GPT-4 followed with a respectable 64.9%.
GPT-4 had the highest unknown rate at 34.2% followed by CodeLlama-34b with 27.5% making them both unreliable for the purposes of fact checking. Both Llama2-70b and Originality.ai had the lowest unknown rate at 0%.
When it comes to error rates Llama-2-70b had the highest at 3.3% while Originality had a modest 1.7%. Notably the rest of the models did not return any errors.
Lower unknown rates are preferable for reliable fact checking. Originality.ai and Llama-2-70b stand out in this regard with 0% unknowns. High error rates or unknowns such as that in Llama-2-70b or GPT-4 could pose a challenge in the real world as their use cannot be relied upon.
Use Originality.ai’s Fact Checking Aid to Help Fact Check AI or Human Text
This testing however is not where Originality.ai shines, Our fact checking tool was built with content editors in mind which is why we include sources and an explanation when using our graphical interface.
We used ChatGPT-3.5 to generate a short article about a very recent news event, Iwe then passed that information to Originality.ai’s tool.
Originality.ai
Originality.ai’s fact checking feature in action
Conclusion:
The Originality.ai model performed well with a performance of 72.3% accuracy rate. Impressively it did not produce any ‘unknown’ outcomes suggesting a level of reliability in regard to being able to produce a result. It is worth noting that all of the facts used in this dataset show that the Originality.ai model is particularly strong in handling recent general knowledge questions. Our fact checking tool was built with content editors in mind which is reflected by the inclusion of sources and explanations when using our graphical interface. This thoughtful feature empowers content editors, enriching the fact-checking process and underscoring the unique value proposition of Originality.ai.
Originality.ai Fact Checking Aid Beta Version Released
Originality.ai is deeply committed to enhancing this fact checking feature to deliver even more accurate and reliable results. Recognizing the critical role that factual accuracy plays in today’s information rich landscape the team is committed to continuously refining the algorithms. Our initial version already performs commendably in the “Recent General Knowledge” category but the pursuit of excellence is an ongoing journey and Originality.ai aims to set new benchmarks in AI assisted fact checking, ensuring users can rely on the platform for precise and trustworthy information.
Founder / CEO of Originality.ai I have been involved in the SEO and Content Marketing world for over a decade. My career started with a portfolio of content sites, recently I sold 2 content marketing agencies and I am the Co-Founder of MotionInvest.com, the leading place to buy and sell content websites. Through these experiences I understand what web publishers need when it comes to verifying content is original. I am not For or Against AI content, I think it has a place in everyones content strategy. However, I believe you as the publisher should be the one making the decision on when to use AI content. Our Originality checking tool has been built with serious web publishers in mind!
When making the tool we had to make each prompt as general as possible to be able to include every kind of input. Not to worry though ChatGPT is smart and will still understand the prompt.
In The Press
Originality.ai has been featured for its accurate ability to detect GPT-3, Chat GPT and GPT-4 generated content. See some of the coverage below…
Originality.ai did a fantastic job on all three prompts, precisely detecting them as AI-written. Additionally, after I checked with actual human-written textual content, it did determine it as 100% human-generated, which is important.
Vahan Petrosyan
searchenginejournal.com
I use this tool most frequently to check for AI content personally. My most frequent use-case is checking content submitted by freelance writers we work with for AI and plagiarism.
Tom Demers
searchengineland.com
After extensive research and testing, we determined Originality.ai to be the most accurate technology.
Rock Content Team
rockcontent.com
Jon Gillham, Founder of Originality.ai came up with a tool to detect whether the content is written by humans or AI tools. It’s built on such technology that can specifically detect content by ChatGPT-3 — by giving you a spam score of 0-100, with an accuracy of 94%.
Felix Rose-Collins
ranktracker.com
ChatGPT lacks empathy and originality. It’s also recognized as AI-generated content most of the time by plagiarism and AI detectors like Originality.ai
For web publishers, Originality.ai will enable you to scan your content seamlessly, see who has checked it previously, and detect if an AI-powered tool was implored.
Tools for conducting a plagiarism check between two documents online are important as it helps to ensure the originality and authenticity of written work. Plagiarism undermines the value of professional and educational institutions, as well as the integrity of the authors who write articles. By checking for plagiarism, you can ensure the work that you produce is original or properly attributed to the original author. This helps prevent the distribution of copied and misrepresented information.
Text comparison is the process of taking two or more pieces of text and comparing them to see if there are any similarities, differences and/or plagiarism. The objective of a text comparison is to see if one of the texts has been copied or paraphrased from another text. This text compare tool for plagiarism check between two documents has been built to help you streamline that process by finding the discrepancies with ease.
Text comparison tools work by analyzing and comparing the contents of two or more text documents to find similarities and differences between them. This is typically done by breaking the texts down into smaller units such as sentences or phrases, and then calculating a similarity score based on the number of identical or nearly identical units. The comparison may be based on the exact wording of the text, or it may take into account synonyms and other variations in language. The results of the comparison are usually presented in the form of a report or visual representation, highlighting the similarities and differences between the texts.
String comparison is a fundamental operation in text comparison tools that involves comparing two sequences of characters to determine if they are identical or not. This comparison can be done at the character level or at a higher level, such as the word or sentence level.
The most basic form of string comparison is the equality test, where the two strings are compared character by character and a Boolean result indicating whether they are equal or not is returned. More sophisticated string comparison algorithms use heuristics and statistical models to determine the similarity between two strings, even if they are not exactly the same. These algorithms often use techniques such as edit distance, which measures the minimum number of operations (such as insertions, deletions, and substitutions) required to transform one string into another.
Another common technique for string comparison is n-gram analysis, where the strings are divided into overlapping sequences of characters (n-grams) and the frequency of each n-gram is compared between the two strings. This allows for a more nuanced comparison that takes into account partial similarities, rather than just exact matches.
String comparison is a crucial component of text comparison tools, as it forms the basis for determining the similarities and differences between texts. The results of the string comparison can then be used to generate a report or visual representation of the similarities and differences between the texts.
Syntax highlighting is a feature of text editors and integrated development environments (IDEs) that helps to visually distinguish different elements of a code or markup language. It does this by coloring different elements of the code, such as keywords, variables, functions, and operators, based on a predefined set of rules.
The purpose of syntax highlighting is to make the code easier to read and understand, by drawing attention to the different elements and their structure. For example, keywords may be colored in a different hue to emphasize their importance, while comments or strings may be colored differently to distinguish them from the code itself. This helps to make the code more readable, reducing the cognitive load of the reader and making it easier to identify potential syntax errors.
With our tool it’s easy, just enter or upload some text, click on the button “Compare text” and the tool will automatically display the diff between the two texts.
Using text comparison tools is much easier, more efficient, and more reliable than proofreading a piece of text by hand. Eliminate the risk of human error by using a tool to detect and display the text difference within seconds.
There is never any data saved by the tool, when you hit “Upload” we are just scanning the text and pasting it into our text area so with our text compare tool, no data ever enters our servers.
Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:
Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.
Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS “AS IS” AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
