Is Claude 4 Sonnet and Opus Content Detectable?

On May 22, 2025, Anthropic announced Claude 4, including the Claude Sonnet 4 and Claude Opus 4 models.

Claude Sonnet 4 is the latest upgrade in Anthropic’s Sonnet series, which offers a significant boost in coding accuracy, reasoning, and instruction. 

Then, Anthropic describes Claude Opus 4 as “the world’s best coding model” and is designed to excel in complex coding and agent workflows. Opus 4 also delivers sustained performance over long-running tasks. Further, it leads industry benchmarks like SWE-bench and Terminal-bench. 

Following this release, we analyzed Claude Sonnet 4 and Claude Opus 4 to assess how the advancements of these models impact the performance of our AI detector.

To test how well AI-generated text can be detected, we ran two tests:

1. We created 1,000 samples of text using Claude 4 Sonnet and ran them through Originality.ai’s AI detector on the Turbo model. 
2. Then, to test how well AI-generated text can be detected, we created 1,000 samples of text using Claude 4 Opus and ran them through Originality.ai’s AI detector on the Turbo model. 

The study’s tests aimed to check how accurate our tool is at detecting AI-written content.

Is Claude 4 Sonnet and Claude 4 Opus AI Content Detectable?

• Yes — Claude 4 Sonnet text is detectable with 98.4% accuracy for the Originality.ai Turbo 3.0.1 model.

• Yes — Claude 4 Opus text is detectable with 98.4% accuracy for the Originality.ai Turbo 3.0.1 model.

It was interesting to note that despite being different models, Claude 4 Sonnet and Claude 4 Opus performed identically when it came to detectability.

Try our AI Detector.

The Datasets

Claude 4 Sonnet: Dataset

To evaluate the detectability of Claude 4 Sonnet, we prepared a dataset of 1000 Claude 4 Sonnet-generated text samples.

AI-Generated Text Data

For AI-text generation, we used Claude 4 Sonnet based on three approaches given below:

1. Rewrite prompts: We generated the content by providing the model with a customized prompt along with some articles (probably generated by LLMs) as a reference to rewrite the prompts. (450 Samples)
   
   
2. Rewrite human-written text: For the second method, we generated the content by attempting to use the provided prompt to bypass the AI Detection tool. To accomplish this, we asked Claude 4 Sonnet to rewrite the human-written text, which we fetched from an open-source dataset (325 Samples)
   
   1. One-Class Learning for AI-Generated Essay Detection
      
      1. Paper: https://www.mdpi.com/2076-3417/13/13/7901
      2. Dataset: https://github.com/rcorizzo/one-class-essay-detection
         
         
3. Write articles from scratch: Finally, for the third approach, we generated the articles from scratch based on a set of topics (fiction and nonfiction) such as history, medicine, mental health, content marketing, social media, literature, robots, the future, etc. (225 Samples).

Claude 4 Opus: Dataset 

To evaluate the detectability of Claude 4 Opus, we prepared a dataset of 1000 Claude 4 Opus-generated text samples.

AI-Generated Text Data

For AI-text generation, we used Claude 4 Opus based on three approaches given below:

1. Rewrite prompts: We generated the content by providing the model with a customized prompt along with some articles (probably generated by LLMs) as a reference to rewrite the prompts. (450 Samples)
   
   
2. Rewrite human-written text: For the second method, we generated the content by attempting to use the provided prompt to bypass the AI Detection tool. To accomplish this, we asked Claude 4 Opus to rewrite the human-written text, which we fetched from an open-source dataset (325 Samples)
   
   1. One-Class Learning for AI-Generated Essay Detection
      
      1. Paper: https://www.mdpi.com/2076-3417/13/13/7901
      2. Dataset: https://github.com/rcorizzo/one-class-essay-detection
         
         
3. Write articles from scratch: Finally, for the third approach, we generated the articles from scratch based on a set of topics (fiction and nonfiction) such as history, medicine, mental health, content marketing, social media, literature, robots, the future, etc. (225 Samples).

The Evaluations

To evaluate the efficacy, we used our Open Source AI Detection Efficacy tool:

• AI Detector Used in this Study
• AI Detector Efficacy Testing Article
• Open Source AI Detection Efficacy Testing Tool

Originality.ai has three models — Model 3.0.1 Turbo, 1.0.0 Lite, and Multi Language for the purpose of AI text detection.

For this test, we evaluated Claude 4 Sonnet & Opus with the Turbo 3.0.1 model.

• Version 3.0.1 Turbo — If your risk tolerance for AI is ZERO! It is designed to identify any use of AI, even light AI.
• Version 1.0.0 Lite — If you are okay with slight use of AI (i.e., AI editing).
• Multi Language — Detects AI content across 30 languages.

Learn more about which AI detection model is best for you and your use case.

The open-source testing tool returns a variety of metrics, each of which reports on a different aspect of performance, including:

• Sensitivity (True Positive Rate): The percentage of the time the detector identifies AI text correctly.
• Specificity (True Negative Rate): The percentage of the time the detector identifies human-written text correctly.
  Accuracy: The percentage of the detector’s predictions that were correct.
• F1: The harmonic mean of Specificity and Precision, often used as an agglomerating metric when ranking the performance of multiple detectors (a performance measurement that combines recall and precision to evaluate models).

If you'd like a detailed discussion of these metrics, what they mean, how they're calculated, and why we chose them, check out our blog post on AI detector evaluation. For a succinct snapshot,  the confusion matrix is an excellent representation of a model's performance.

Below is an evaluation of the Turbo 3.0.1 model on the above datasets. 

Confusion Matrix

Claude Sonnet 4: Confusion Matrix

Claude Sonnet 4: Evaluation Metrics

For this small test to reflect the Originality.ai AI detector’s ability to identify Claude 4 Sonnet content, we looked at the True Positive Rate or the percentage of the time the model correctly identified AI text as AI out of a 1000-sample of Claude 4 Sonnet content. 

Model 3.0.1 Turbo:

• Recall (True Positive Rate) = 98.4%

Claude Opus 4: Confusion Matrix

Claude Opus 4: Evaluation Metrics

For this small test to reflect the Originality.ai detector’s ability to identify Claude 4 Opus content, we looked at the True Positive Rate or the percentage of the time the model correctly identified AI text as AI out of a 1000-sample of Claude 4 Opus content. 

Model 3.0.1 Turbo:

• Recall (True Positive Rate) = 98.4%

Conclusion

Our study confirms that the content generated by Claude 4 Sonnet and Opus is highly detectable with our AI detector. 

The Originality.ai model 3.0.1 Turbo excelled with 98.4% accuracy in detecting both Claude 4 Sonnet and Claude 4 Opus-generated text in our tests.

These results highlight the effectiveness of the Originality.ai AI detector in identifying AI-generated content, ensuring reliable detection across various text generation approaches.

Interested in learning more about AI detection? Check out our guides:

• AI Content Detector Accuracy Review
• A Meta-Analysis of AI Detection Studies by third-party researchers and academics.
• How Does AI Content Detection Work?