Did DeepSeek Copy ChatGPT and is it Detectable?

DeepSeek has introduced DeepSeek Chat — a cutting-edge conversational AI designed for enhanced contextual understanding and long-form coherence. 

This model focuses on improved reasoning, multilingual capabilities, and efficient response generation.

This study analyzes 150 DeepSeek-Chat generated text samples to determine whether they can be detected by the Originality.ai Turbo and Lite AI content detectors. 

Additionally, we compare the detection accuracy of our models against two other tools — GPTZero and RapidAPI’s Trending Content Detection Tool (AI Content Detector | AI/GPT).

Then, we also look at whether DeepSeek Chat could potentially be a distilled version of OpenAI’s LLMs.

Key Takeaways (TL;DR)

Is DeepSeek Chat AI Content Detectable?

• Yes — DeepSeek-Chat text is detectable with 99.3% accuracy using our 3.0.1 Turbo model and 99.3% accuracy with our Lite 1.0.0 model. 
• Both models, developed by Originality.ai, outperform GPTZero and RapidAPI, which achieved 97.3% and 80.7% accuracy respectively in detecting content generated by the DeepSeek-Chat model.

Is DeepSeek Chat a Distilled Version of OpenAI Technology?

• Each time a new LLM comes out, we run a test to evaluate our AI detector's efficacy and until today we typically see a slight drop off in accuracy when a new model is released. 
• However, with DeepSeek we are not seeing that dip in accuracy. Both of our models were able to detect DeepSeek content with 99%+ accuracy. 
• So, based on our research, it is possible that DeepSeek could be a distilled version of ChatGPT.

Try our AI Detector here.

Dataset

In order to evaluate the detectability of DeepSeek Chat, we prepared a dataset of 150 DeepSeek-Chat-generated text samples.

AI-Generated Text Data

For AI-text generation, we used DeepSeek-Chat based on three approaches given below:

1. Rewrite prompts: Generating the content by providing the model with a customized prompt along with some articles (probably generated by LLMs) as a reference to rewrite from. (50 Samples)
   
2. Rewrite human-written text: Generating the content considering the provided prompt to bypass the AI Detection tool by rewriting the human-written text which we fetched from an open-source dataset (50 Samples)
   
   1. One-Class Learning for AI-Generated Essay Detection
      
      1. Paper: https://www.mdpi.com/2076-3417/13/13/7901
      2. Dataset: https://github.com/rcorizzo/one-class-essay-detection
         
3. Write articles from scratch: Generating the articles from scratch based on the given topics ranging from fictional and non-fictional diverse domains such as history, medicine, mental health, content marketing, social media, literature, robots, future etc. (50 Samples)

Evaluation

To evaluate the efficacy we used the Open Source AI Detection Efficacy tool that we have released:

• AI Detector Used in this Study
• AI Detector Efficacy Testing Article
• Open Source AI Detection Efficacy Testing Tool

Originality.ai has two models namely Model 3.0.1 Turbo and Model 1.0.0 Lite for the purpose of AI Text Detection.

• Use Version 3.0.1 Turbo - If your risk tolerance for AI is ZERO! It is designed to identify any use of AI even light AI.
• Version 1.0.0 Lite - If you are okay with slight use of AI (i.e. AI editing)

The open-source testing tool returns a variety of metrics for each detector you test, each of which reports on a different aspect of that detector’s performance, including:

• Sensitivity (True Positive Rate): The percentage of the time the detector identifies AI correctly.
• Specificity (True Negative Rate): The percentage of the time the detector identifies humans correctly.
• Accuracy: The percentage of the detector’s predictions that were correct.
• F1: The harmonic mean of Specificity and Precision, often used as an agglomerating metric when ranking the performance of multiple detectors.

If you'd like a detailed discussion of these metrics, what they mean, how they're calculated, and why we chose them, check out our blog post on AI detector evaluation. For a succinct snapshot, though, we think the confusion matrix is an excellent representation of a model's performance.

Below is an evaluation of both the models on the above dataset. 

Confusion Matrix:

Evaluation Metrics:

For this smaller test to be able to identify the ability of Originality.ai’s AI detector to identify DeepSeek-Chat content we look at True Positive Rate or the % of the time that the model correctly identified AI text as AI out of a 150 sample DeepSeek-Chat content. 

Model 1.0.0 Lite:

• Recall (True Positive Rate) = 99.3%

Model 3.0.1 Turbo:

• Recall (True Positive Rate) =  99.3%

GPTZero:

• Recall (True Positive Rate) =  97.3%

Results — DeepSeek Chat Text is Detectable

Our study confirms that DeepSeek-Chat AI-generated text is highly detectable using our AI content detectors. 

Both Model 3.0.1 Turbo and Model 1.0.0 Lite achieved an impressive 99.3% recall (true positive rate), significantly outperforming GPTZero (97.3%) and RapidAPI’s AI Content Detector (80.7%). 

These results highlight the effectiveness of the Originality.ai AI detection models in accurately identifying DeepSeek-Chat-generated content.

Could DeepSeek Chat Be a Distilled Version of OpenAI’s ChatGPT?

Every time a new LLM comes out, we run a test to evaluate our AI detector's efficacy. 

Until today we typically saw a drop off in accuracy when a new model was released. 

Through extensive testing as documented in our AI Detection Accuracy Study, our AI detection models have an overall 99%+ accuracy rate (Turbo 3.0.1) and 98% accuracy rate (Lite).

Although results can vary, following a new model release we typically see a slight drop-off in accuracy. Then, our machine learning engineers train our models to get accuracy back up.

However, with DeepSeek we are not seeing a drop off in accuracy. 

Could this mean that DeepSeek is potentially a distilled version of OpenAI’s ChatGPT and existing LLMs? 

Based on our research, yes, it is our hypothesis that DeepSeek could be a distilled version of ChatGPT.

Further, Bloomberg and the BBC are reporting that OpenAI and Microsoft are investigating if OpenAI technology was used or obtained in an unauthorized way in relation to DeepSeek.

Final Thoughts

It is clear that DeepSeek has caused immense disruption in the AI industry. 

Yet, the text that DeepSeek Chat produces is still detectable by the industry-leading Originality.ai AI Checker. 

Further, the exceptional accuracy rates (1.0.0 Lite: 99.3% and Turbo: 99.3%) for such a new and disruptive model, pose the question of whether DeepSeek is simply a distilled version of OpenAI’s LLMs.

This is notable considering the historical context of a slight drop-off in AI detection accuracy when new AI models are released.

To learn more about AI detection read our AI detection accuracy study and a meta-analysis of AI detection studies conducted by third parties.

Then, get insight into what DeepSeek is and how it’s impacted the AI industry in our DeepSeek guide.