At Originality.ai we looked to use our AI detector to identify if Hollywood writers were or were not already using generative AI (such as ChatGPT) to help write scripts.
The transcripts of over 3800 USA TV show episodes that aired from 2020 to 2023 were analyzed for this study. Findings were consistent: these scripts were not produced by AI text-generators or other LLM(large language model)-based tools.
A dataset of approximately 6000 episode transcripts from several genres was collected, cleaned, preprocessed and filtered through the date range of January 2020 to August 2023. Approximately 3800 scripts were then fed into Originality.ai’s API. The AI detector analyzed the text for AI Content, returning a probability score in the range of 0 to 1. To review the accuracy of the AI detector see this AI detector accuracy study.
The date a script was written was not able to be easily identified.
It was important to try to identify the date the script was written and to do so we made the assumption that due to the short writing time to production time this study focused on 2 categories of shows:
We acknowledge that this scope does not represent the entirety of the genres of Hollywood-produced TV shows. However, its findings are still relevant and can provide an interesting starting point for further studies on more diverse data.
We look forward to continuing this study as writers get back to work after the strike ends.
Almost 100% original content was found on average across genres, and years. More specifically, there is approximately 1.9% average probability of AI Content with a standard deviation of 0.05 across the scripts.
This is in contrast to almost all other studies we have conducted that are showing significant increases in the rate of AI usage:
Scripts from Soap Opera’s in 2020 showed 1.3% were suspected of being AI generated. This % is in line with our tested AI detection false positive rate. However, the % of scripts that were suspected of being AI generated has increased every year to 4.5% in 2023. This is still a relatively small number and given the detectors not perfect accuracy we would caution against drawing significant conclusions from this increase. However, it shows it is definitely worth continuing to monitor.
Hollywood has provided over a century of entertainment to the general public and shaped our culture. But the people directly responsible for creating these beloved stories, writing unforgettable lines, and spawning memes went on strike in March 2023. The Writers’s Guild of America commenced their industrial strike action against their studio bosses. Their demands included that they would not be replaced by AI.
In the 1984 film The Terminator, a robot from the future says: “I’ll be back.” This led to an argument between the robot’s actor and the screenplay writer in what has become an uncanny metaphor of our present dilemma. The actor, Arnold Schwarzenegger, argued that a robot would be more grammatically correct: “I will be back.” The screenplay writer, James Cameron said: “I don’t tell you how to act. Don’t tell me how to write.” (paraphrased). The line stayed and the rest is history.
The use of Technology in media has always been a double-edged sword. It has enabled the most otherworldly stories to be translated out of the imaginations of creators onto the screen, and shared with the world. But that Hollywood magic can also be a curse to the creators and the other people behind the scenes.
The advent of CGI in visual effects negatively impacted the model-builders, the artists and the designers. The VFX artists who replaced them are not better off. Less than a year after Rhythm & Hues Studios won an Oscar for Visual Effects in 2013’s Life of Pi, the VFX company filed for bankruptcy.
The advent of streaming affected distribution, then season to episode count, then the structure of writers’ rooms. Each of these changes altered the profit sharing structure of media to the detriment of writers. The most popular shows in syndication are raking in millions for the streaming companies and studios, while the writers get pennies – if they’re lucky.
Now with the advent of LLM text generators, the WGA fears that the studios will replace them with AI and their fears are not unfounded.
Writers are paid for ideas and for their script. What happens when the AI generates the idea? We know what the studios want – more profit. But if an AI is writing the scripts, where will this leave the writers? On August 18th 2023 a DC District Court Judge Beryl A. Howell ruled that AI-generated content is not protected by copyright laws. But it doesn’t take an army of lawyers to realize that by hiring a writer who has been forced to accept a ‘co-writing’ credit, that ruling can be easily bypassed.
A wealth of stories make up the training corpus of Large Language Models. Few, if any, creators gave permission for their art to be used, in fact OpenAI is facing several lawsuits. Fewer still got any compensation for this. Some might argue that a bulk of this text was already in the public domain and therefore free for use. But free to whom? Public domain has evolved from, amongst many cultures, the ancient Roman law of res publicae and res communes – that things that should be free so that they could be commonly enjoyed by mankind.
Almost 40 years after The Terminator, the same writer weighed in on the AI debate: “I don’t believe that a disembodied mind, regurgitating what other embodied minds have said — about the life that they’ve had, about love, about lying, about fear, about mortality — and put into a word salad and then regurgitate it. I don’t believe that’s going to move an audience.” (paraphrased)
We believe that it is crucial for AI content detectors reported accuracy to be open, transparent and accountable. The reality is, each person seeking AI-detection services deserves to know which detector is the most accurate for their specific use case.