I always get a random message from a webmaster or a client about how a content piece is AI. No surprise, they plugged the content into either ZeroGPT or GPTZero and concluded that this content is AI. The bottom line is that AI checker tools are far from reliable; research shows simple prompting can evade AI content detection.
Here, I’ll give you a sense of how these tools work and demonstrate how simple it is to evade the most popular AI content checker tools.
How AI Content Checkers Work
AI Content Checkers uses a combination of linguistic analyses and machine learning techniques to evaluate the likelihood that the given text is AI-generated.
- Predictability Check: Looks at how predictable the text is. AI writing tends to be more predictable than human writing.
- Sentence Variety: Checks if sentence lengths and structures vary. Human writing usually has more variety.
- Word Patterns: Examines word combinations. Certain patterns can reveal if AI wrote the text.
- Writing Style: Analyzes unique features like word choice and punctuation to spot AI-generated content.
- Deep Learning Models: Uses advanced tools to compare the text in detail, distinguishing AI from human writing.
However, according to Wikipedia, a 2023 study by Weber-Wulff et al. evaluated 14 detection tools, including Turnitin and GPT Zero, revealing that none achieved over 80% accuracy, with only five exceeding 70% accuracy. So, the lack of accuracy is a known fact. However, the brilliant marketing of these tools has buried the flaws deep.
Where and how do these tools fail?
It’s simple: if you write content in clear and concise language, it will most likely be flagged as AI. For this example, I asked ChatGPT to “write a 100-word piece about Starfish,” and this is what it gave me:
Then, I copied the content directly from the chat window and pasted it into ZeroGPT. The result was a 63% probability of AI content.
Now, let’s add this simple prompt to the same chat window: “make this really ornate and unnecessarily complicated.” Below are the results.
Guess what? ZeroGPT says my content is 100% human because of the ridiculous words and over-complicated language.
There is always speculation about the word count needed to measure AI content likeliness accurately. Of course, any algorithm needs sufficient data to parse accurately. So, to be fair, I will generate four 500+ word article using my custom prompt with a defined role, and no, I will not be sharing the prompt or the custom role. These articles will be developed to bypass Writer.com’s AI detection tool.
Below are the results.
Now, let’s test an article that is not AI. But how do we know for sure? Let’s pick an article that predates 2018, look up the cache, and stick it back into the popular AI detection tool, ZeroGPT.
For this example, we will use CNN’s archives to pick up a URL that has been cached. In this case, we are going to use this one:
https://money.cnn.com/interactive/technology/computing-power-timeline
Plus, we’re going to double check it is really 2011 content by using Wayback Machine for the content: https://web.archive.org/web/20121030012315/https://money.cnn.com/interactive/technology/computing-power-timeline/
Now that we have established that the content really predates LLM’s, let’s see the results on ZeroGPT.
ZeroGPT returned a 2011 article with an 85% probability that it was AI-generated. Now, let’s address the concern that this maybe a one-off situation. So, let’s use another URL at random from CNN: https://money.cnn.com/2012/02/22/real_estate/home_sales/index.htm
Not surprisingly, ZeroGPT gave it an 87% probability that the content is AI.
Now, to test the most popular AI checker, GPTZero, I will generate a 500+ word article about Poker. Just to be sure, I will use their “Deep Scan” on a free account. The topic is “Poker’s Popularity Spikes After Key Movie Releases: A Historical Look” – Below are the results.
Let’s take it a step further. We will plug the same piece of content that I generated with GPT prompting and role creation into CopyLeaks, Scribbr, Undetectable, Writer, and Quillbot AI checkers for their results.
Where do we go from here?
Just like how large language models like ChatGPT are trained on data, so are these AI content checkers. But, since these tools’ bottom line is to pick up AI content, they are trained on a very limited source of truth. This creates “bias.” The inflexibility and bias of these tools’ algorithms are the ultimate driving point of their inaccuracy.
So, in short, if you write formally or in simple language with on-point grammar, you will be flagged as AI content. However, if you write in a lot of first-person, include a few random numbers, add people’s names, and mix up sentence structures, your content will most likely pass the test. This is because the majority of these tools have been trained on news reports and other press releases as a single source of truth.
Now, if you’re wondering how you can detect AI content without relying on these tools, here are a few tips without me over-indulging in my secrets:
- High-level Content: Content that says a lot without saying a lot. If you come across a content piece that is 1,000 words and contains content you have seen 100 times, it’s most likely due to AI. Large language models are trained on given data sets and can’t think “out of the box” to add unique talking points or opinions.
- Odd Words: If you see content that uses really unusual words that you or anyone you know may never use in a regular conversation, that will be a red flag. Think about it: how many times have you used the words: “realm, daunting, or allure” in a conversation? So, if most words seem unnatural, it most likely is.
- Flowery Language: Finally, flowery or ornate language is a dead giveaway. If a content piece uses language that leans towards prose or language that is hyperbolically ornate, it’s most likely AI. Here’s an example “The soft glow of the evening sun bathed the garden in warm light, casting long shadows that swayed gently with the breeze.”
So, in the end, rely on your judgment if you come from a content background, and if you don’t, use the tips above to judge for yourself if a content piece is AI, or not.
