Is Qwen2.5-Max Content Detectable?

With the release of Alibaba’s Qwen2.5-Max, a cutting-edge Mixture-of-Experts (MoE) AI model, the boundaries of language generation and understanding continue to evolve. This development is particularly exciting for industries that rely on nuanced language understanding, like translation services, customer support, and content creation.

As AI-generated content becomes more advanced, the question arises — can we still detect it? 

This study examines 1,000 samples from Qwen2.5-Max to assess their detectability using our Turbo and Lite AI content detectors, alongside GPTZero and RapidAPI’s AI Content Detector. 

Is Qwen2.5-Max AI Content Detectable?

1. Yes — Qwen2.5-Max text is detectable by Originality.ai with 99.8% accuracy using our 3.0.1 Turbo model and 96.9% accuracy with our Lite 1.0.0 model. 
2. Turbo 3.0.1, developed by Originality.ai, outperformed GPTZero and RapidAPI, which achieved 97.9% and 29.2% accuracy respectively in detecting content generated by the Qwen2.5-Max model.

Try our AI Detector here.

Dataset

In order to evaluate the detectability of Qwen2.5-Max, we prepared a dataset of 1000 Qwen2.5-Max generated text samples.

AI-Generated Text Data

For AI-text generation, we used Qwen2.5-Max based on three approaches given below:

1. Rewrite prompts: Generating the content by providing the model with a customized prompt along with some articles (probably generated by LLMs) as a reference to rewrite from. (450 Samples)
   
   
2. Rewrite human-written text: Generating the content considering the provided prompt to bypass the AI Detection tool by rewriting the human-written text which we fetched from an open-source dataset (350 Samples)
   
   1. One-Class Learning for AI-Generated Essay Detection
      
      1. Paper: https://www.mdpi.com/2076-3417/13/13/7901
      2. Dataset: https://github.com/rcorizzo/one-class-essay-detection
         
3. Write articles from scratch: Generating the articles from scratch based on the given topics ranging from fictional and non-fictional diverse domains such as history, medicine, mental health, content marketing, social media, literature, robots, future etc. (200 Samples)

Evaluation

To evaluate the efficacy we used the Open Source AI Detection Efficacy tool that we have released:

• AI Detector Used in this Study
• AI Detector Efficacy Testing Article
• Open Source AI Detection Efficacy Testing Tool

Originality.ai has two models namely Model 3.0.1 Turbo and Model 1.0.0 Lite for the purpose of AI Text Detection.

• Use Version 3.0.1 Turbo - If your risk tolerance for AI is ZERO! It is designed to identify any use of AI even light AI
• Version 1.0.0 Lite - If you are okay with slight use of AI (i.e. AI editing)

The open-source testing tool returns a variety of metrics for each detector you test, each of which reports on a different aspect of that detector’s performance, including:

• Sensitivity (True Positive Rate): The percentage of the time the detector identifies AI correctly.
• Specificity (True Negative Rate): The percentage of the time the detector identifies human-written content correctly.
• Accuracy: The percentage of the detector’s predictions that were correct.
• F1: The harmonic mean of Specificity and Precision, often used as an agglomerating metric when ranking the performance of multiple detectors.

If you'd like a detailed discussion of these metrics, what they mean, how they're calculated, and why we chose them, check out our blog post on AI detector evaluation. For a succinct snapshot, though, we think the confusion matrix is an excellent representation of a model's performance.

Below is an evaluation of both the models on the above dataset. 

Confusion Matrix:

Evaluation Metrics:

For this smaller test to be able to identify the ability of Originality.ai’s AI detector to identify Qwen2.5-Max content we look at True Positive Rate or the % of the time that the model correctly identified AI text as AI out of a 1000 Qwen2.5-Max content samples. 

Model 1.0.0 Lite:

• Recall (True Positive Rate) = 96.9%

Model 3.0.1 Turbo:

• Recall (True Positive Rate) =  99.8%

GPTZero:

• Recall (True Positive Rate) =  97.9%

RapidAPI (AI Content Detector | AI/GPT):

• Recall (True Positive Rate) =  29.2%

Conclusion

Our study confirms that Qwen2.5-Max AI-generated text is highly detectable using our AI content detectors. Model 3.0.1 Turbo achieved an outstanding 99.8% recall, while Model 1.0.0 Lite followed closely with 96.9% recall.

Turbo 3.0.1, developed by Originality.ai surpassed GPTZero (97.9%) and significantly outperformed RapidAPI’s AI Content Detector (29.2%). 

These results demonstrate the superior accuracy and reliability of our AI detection models in identifying AI-generated content, reinforcing their effectiveness as industry-leading tools for AI content detection.

To learn more about AI detection and its efficacy read our AI detection accuracy study and a meta-analysis of AI detection studies conducted by third parties.