AI Detectors Don't Work - Unmasking AI Detector Flaws

Capstone

Recast 1

Recast 2

Reflection

Explore the complexities and limitations of AI detection technologies.

AI, Artificial Intelligence, keyboard, machine learning, natural language processing, chatbots, virtual assistants, automation, robotics, computer vision, deep learning, neural networks, language models, human-computer interaction, cognitive computing, data analytics, innovation, technology advancements, futuristic systems, intelligent systems, smart devices, IoT, cybernetics, algorithms, data science, predictive modeling, pattern recognition, computer science, software engineering, information technology, digital intelligence, autonomous systems, IA, Inteligencia Artificial,

Unmasking AI Detector Flaws

Glucometer Mockups, Download PSD Mockup Files from https://mockupfree.net

Explore the depths of AI detection!

<h4>(Blog Entry #5)</h4>You’re probably wondering, “How is it a coin flip?” Let’s fill you in. 1. AI detectors are based on guesses of patterns. 2. They were never reliable. 3. Many creators shut down their detectors due to inaccuracy.First off, the GENIUSES that created OpenAI…you know the ChatGPT everyone knows…shut down their own AI classifier after just 9 months. When you have a feeling about something, it’s not proof. These detectors are just a second feeling that your professor hopes will back up theirs. The detectors don’t detect anything; they just look for common patterns in your writing and in AI’s writing, and look to see if it matches it at all.Using any sort of detectors is literally a coin flip, and schools are letting these algorithms decide if a student is guilty or not. A lot of studies are showing that these detectors are actually biased, misclassifying non-native English speakers as “AI” more often than native writers. Also the best part, the law is actually on your side here; the U.S. Copyright Office says a human is still the author even if they use a computer as an “assisting instrument”. &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Basically, the bottom line is that we have got to stop pretending that these tools are truth-finders when they are really just pattern-matchers that get it wrong most of the time.

An artist’s illustration of artificial intelligence (AI). This image explores how AI can be used to solve fundamental problems, unlocking solutions to many more. It was created by Jesper Lindborg as part of the Visualising AI project launched by Google DeepMind.

Why Your Professor’s AI Detector is Actually a $20 Coin Flip

Frequently Asked Questions

Students should not be forced to gamble their career because of a broken algorithm. For over 800 years, society has fought for fairness and individuals have fought to be innocent unless proven guilty. Yet, across the country, students have been accused of academic dishonesty, not by a human, but by a machine, a machine so flawed that even its own creators declared it as having a “low rate of accuracy”, it is a digital coin flip. The issue is that ever since the start of AI society always has to wonder now, “is this AI?” Educational institutions across the world have convinced themselves that they have found a solution, when in reality there really is no way to have 100% certainty if something is AI or not. Institutions have been using AI detectors to practically “catch” students who are using AI in their work, when these "detectors" are not reliable at all. They do not “detect” anything; they only attempt to find common patterns and see if it maybe matches. This is why they output a percentage as the result, it is not a guarantee. All the detectors do is guess; there will never be 100% accuracy. Even OpenAI shut down its own detector for its "low rate of accuracy." For artists, writers, and really anyone that publishes their work, there is a legal way to use AI work under the “Human Contribution” exception, but if the only tool available to enforce this is broken, then the rule is useless.&nbsp;First, let us understand how AI content is handled by the law. What most people do not realize is that AI-generated content is actually considered “public domain.” Public domain means that the content is available to the public. For example, if an individual gives AI a prompt and then copies the output word for word, the text that was just copied is considered public domain.&nbsp;Amanda Robert, an accredited social justice worker and graduate from Saint Louis University states that, "The Copyright Office and the District of Columbia agreed that only works created by human beings are eligible for copyright" (Robert). This source supports the claim that one cannot claim AI content as their own. As a result, “Any pure AI output is not eligible for copyright protection.” This is the biggest rule for using AI content.&nbsp;However, there is a way to claim work that contains AI. This is called the Human Contribution exception. For example, someone could have AI generate ideas for an invention, but they cannot just steal the idea. The main rule is that the person who wants to patent an invention created with the assistance of AI, the person must have had a very significant contribution in the work. The AI must only be an assistant. As the U.S. Copyright Office (2023) explains, the key question is "whether the 'work' is basically one of human authorship, with the computer... merely being an assisting instrument." The point of the document is that pure AI output cannot be claimed, but a human can be the author if their own creative contribution is the main part of the work.One thing that is insanely concerning about checking for AI content, is the AI&nbsp;“detectors”. The name itself is misleading. They do not actually detect anything. These detectors actually work by storing millions of outputs from different types of AI language models, and then they use those to discover a common pattern. Then they provide a percentage based on how closely the patterns match up. They are pattern-matchers, not truth-finders.&nbsp;Now, we would not have trouble worrying about whether something is AI or not, if the detectors worked correctly. These pattern-matching tools are not accurate. Their lack of certainty is revealed in their results; if they had 100% accuracy, they would not have a percent probability as the result. Many detectors will not necessarily give you a percentage, but they will usually sound like this, “We are highly confident that this text is human content.” A university study found that AI detectors, "consistently misclassify non-native English writing samples as AI-generated, whereas native writing samples are robustly identified as human-written" (Liang et al.). This proves that AI detectors are not just unreliable, but are also mistaking text from writers whose first language is not English as AI.&nbsp;On top of that, even the makers of these detectors are guilty! OpenAI, the company that created ChatGPT (the industry leader in generative AI) admitted its own tool was a complete failure, so bad that they decided to fully shut it down after only nine months from the first public release of ChatGPT. This article states, “As of July 20, 2023, the AI classifier is no longer available due to its low rate of accuracy” (OpenAI). When these detectors say, “We are highly confident that this text is human content” they use the word confidence. Confidence means “the feeling or belief that one can rely on someone or something.” Keywords: feeling, belief. That does not mean 100%. If the geniuses that released the industry changing generative AI chat-bot could not make a reliable detector for their own creation, then how can any other small third-party website do it? Also, this is Elon Musk we are talking about, the person that society believes to be the smartest guy in the world failed at creating an AI detector for his own AI.Relying on any of these detectors could putting individuals, universities, and businesses in a position of false accusation. These tools are literally just making an educated guess, there is no real certainty. This article states that, “AI detection software is far from foolproof—in fact, it has high error rates and can lead instructors to falsely accuse students of misconduct” (MIT). If one claims it as their own, they might get in trouble, but by whom? Exactly, nobody. This uncertainty makes the entire issue impossible to manage fairly. If someone’s own work is flagged as AI by one of these unreliable detectors, then that person is being accused and nobody knows who to trust.&nbsp;Basically, the law allows for individuals to claim partially AI-generated content considered Human Contribution, the tools used to enforce plagiarism are based on "belief" and&nbsp;"patterns," not certainty. People need to stop relying on AI detectors to determine if something is&nbsp;AI or not. The reliance on these detectors leaves society with this unanswerable question: “is it AI?” One can never be certain that something is AI, or if something is not, like the text on the screen right now… but everybody can trust that the detector says it is not. Right?&nbsp; <h4>Works Cited&nbsp;</h4>“AI Detectors Don’t Work. Here’s What to Do Instead.” MIT Sloan Teaching &amp;&nbsp;&nbsp;Learning Technologies, 2024, <a target="_blank" rel="noopener noreferrer nofollow" href="http://mitsloanedtech.mit.edu/ai/teach/ai-detectors-dont-work/">mitsloanedtech.mit.edu/ai/teach/ai-detectors-dont-work/</a>.&nbsp;&nbsp;Liang, Weixin, et al. “GPT Detectors Are Biased against Non-Native English Writers.”&nbsp; ArXiv (Cornell University), 5 Apr. 2023, <a target="_blank" rel="noopener noreferrer nofollow" href="https://doi.org/10.48550/arxiv.2304.02819">https://doi.org/10.48550/arxiv.2304.02819.</a>&nbsp;&nbsp;OpenAI. “New AI Classifier for Indicating AI-Written Text.” <a target="_blank" rel="noopener noreferrer nofollow" href="http://Openai.com">Openai.com</a>, 2023,&nbsp; <a target="_blank" rel="noopener noreferrer nofollow" href="http://openai.com/index/new-ai-classifier-for-indicating-ai-written-text/">openai.com/index/new-ai-classifier-for-indicating-ai-written-text/</a>.&nbsp;&nbsp;Robert, Amanda. “Art Generated by AI Can’t Be Copyrighted, DC Court Says.” ABA Journal,&nbsp;&nbsp;25 Aug. 2023, <a target="_blank" rel="noopener noreferrer nofollow" href="http://www.abajournal.com/news/article/art-generated-by-ai-cant-be-copyrighted-dc-court-says">www.abajournal.com/news/article/art-generated-by-ai-cant-be-copyrighted-dc-court-says</a>.&nbsp;Accessed 26 Feb. 2026.&nbsp;&nbsp;U.S. Copyright Office. “Copyright Registration Guidance: Works Containing Material Generated&nbsp; by Artificial Intelligence.” U.S. Copyright Office, U.S. Copyright Office, 13 Mar. 2023,&nbsp; <a target="_blank" rel="noopener noreferrer nofollow" href="http://www.copyright.gov/ai/ai_policy_guidance.pdf">www.copyright.gov/ai/ai_policy_guidance.pdf</a>. Accessed 26 Feb. 2026.&nbsp;

A somewhat commonly-used artistic trope that can be used to illustrate both pro-AI and anti-AI sentiment. Modeled and rendered with Blender Cycles 5.0.1

AI Detectors Are Broken

“AI Detectors Don’t Work. Here’s What to Do Instead.” MIT Sloan Teaching &amp;&nbsp;Learning Technologies, 2024, <a target="_blank" rel="noopener noreferrer nofollow" href="http://mitsloanedtech.mit.edu/ai/teach/ai-detectors-dont-work/">mitsloanedtech.mit.edu/ai/teach/ai-detectors-dont-work/</a>.MIT Sloan Teaching &amp; Learning Technologies states that, “AI detection software is far from foolproof—in fact, it has high error rates and can lead instructors to falsely accuse students of misconduct” (MIT). This is an article published by MIT. I will use this to support my claim that AI detectors don’t work, and my action for the reader, that teachers shouldn’t be relying on these detectors entirely. This is different from other sources because they don’t usually talk about how teachers should stop using them, they only talk about how they are unreliable. I also love this source because they have so many sources at the bottom and I can use any of those to include in my writing, it makes it much easier to find good sources. Liang, Weixin, et al. “GPT Detectors Are Biased against Non-Native English Writers.”&nbsp;ArXiv (Cornell University), 5 Apr. 2023,<a target="_blank" rel="noopener noreferrer nofollow" href="https://doi.org/10.48550/arxiv.2304.02819">https://doi.org/10.48550/arxiv.2304.02819.</a>This study–conducted by Weixin Liang, Mert Yuksekgonul, Yining Mao, Eric Wu, and James Zou–found that AI detectors ”consistently misclassify non-native English writing samples as AI-generated, whereas native writing samples are robustly identified as human-written" (Liang et al.). This is a university study. It proves that AI detectors are not just unreliable, but are also mistaking text from writers whose first language is not English as AI. This is more evidence that could support my claim. It supports my argument that AI detectors are unreliable and are not 100% accurate. This source is different from the OpenAI article. Instead of just sharing OpenAI’s admission of guilt, this is third-party, academic proof of why the detectors are unreliable and flawed. OpenAI. “New AI Classifier for Indicating AI-Written Text.” <a target="_blank" rel="noopener noreferrer nofollow" href="http://Openai.com">Openai.com</a>, 2023,&nbsp;<a target="_blank" rel="noopener noreferrer nofollow" href="http://openai.com/index/new-ai-classifier-for-indicating-ai-written-text/">openai.com/index/new-ai-classifier-for-indicating-ai-written-text/</a>.OpenAI states, “As of July 20, 2023, the AI classifier is no longer available due to its low rate of accuracy.” (OpenAI) This is direct evidence to support my claim that OpenAI discontinued their AI detector. I will include this with any paragraph that mentions OpenAI and use it to support my claim that AI detectors are unreliable. This is way better than any source because the source is directly supporting my claim, it is a primary source to my topic. Robert, Amanda. “Art Generated by AI Can’t Be Copyrighted, DC Court Says.” ABA Journal,&nbsp;25 Aug. 2023, <a target="_blank" rel="noopener noreferrer nofollow" href="http://www.abajournal.com/news/article/art-generated-by-ai-cant-be-copyrighted-dc-court-says">www.abajournal.com/news/article/art-generated-by-ai-cant-be-copyrighted-dc-court-says</a>. Accessed 26 Feb. 2026.Amanda Robert states that, "The Copyright Office and the District of Columbia agreed that only works created by human beings are eligible for copyright.” This source will help to support my claim that you cannot claim AI content as your own. I must keep in mind that this is only referring to AI art, not writing. This source will help support any part of my essay that I want to talk about art. This source is different from most only because of how direct the information was. U.S. Copyright Office. “Copyright Registration Guidance: Works Containing Material Generated&nbsp;by Artificial Intelligence.” U.S. Copyright Office, U.S. Copyright Office, 13 Mar. 2023,&nbsp;<a target="_blank" rel="noopener noreferrer nofollow" href="http://www.copyright.gov/ai/ai_policy_guidance.pdf">www.copyright.gov/ai/ai_policy_guidance.pdf</a>. Accessed 26 Feb. 2026.As the U.S. Copyright Office explains, the key question is "whether the 'work' is basically one of human authorship, with the computer... merely being an assisting instrument." This is the official document from the U.S. Copyright Office. The point of the document is that pure AI output cannot be claimed, but a human can be the author if their own creative contribution is the main part of the work. I can use this to explain the legality behind using AI in public works. This source is much different than any other source because it’s not an argument or an opinion, it's the real legal document from the government that my entire argument is based on.

Annotated Bibliography

The goal of this campaign was to, as you said, “push my argument off the page and into action,” by moving from a not so pretty to look at, dense, academic essay, to visually pleasing posters and pamphlets. My core argument is that using AI detectors is a literal coin flip that unfairly forces students to gamble their career on a stupid algorithm. The “Your Career is Not a Coin Flip” poster specifically targets the students to spark an immediate sense of urgency regarding the safety of their own careers. I’m going to be honest, this was way overkill and it’s really not that serious but I did it so that it would stand out more. The “If the Geniuses Failed” poster changes the target to educators and instructors, challenging the logic of guessing with student lives when the tech’s own creators couldn’t make it work. Next, the pamphlet acts as a tactical guide for students who have already been flagged and need help with defense. I would put the posters in high traffic campus hallways and faculty offices. On the other hand, I would distribute the pamphlets in student centers and labs.	I used Pathos in the student directed poster by using the visual of a coin to emphasize the randomness and fear of losing an 800 year old right to fairness. I used Logos in the educator directed poster by highlighting the cold, hard fact that OpenAI shut down their classifier on July 20, 2023, due to a “low rate of accuracy.” The digital aesthetic across all of my pieces is a deliberate visual appeal meant to symbolize the broken and unreliable nature of the detectors. I chose the&nbsp; contrast color palette of deep blue, black, and “system error” red to get attention to people passing by. I also used the stop hand icon as a symbol to drop any previous assumptions about AI detection. In the pamphlet, I included “audience-appropriate evidence” by citing the documented bias against non-native English writers to add social justice layer to my argument. The phrase “pattern-matchers, not truth-finders” was repeated to strip away the “credibility” of the tools instructors are currently using.	Visual rhetoric is a far more effective medium for this specific argument because most people will never read a 5-page research paper, but they will look at a coin flip poster for 10 seconds. The transition from the written argument was achieved by identifying my best points (like the OpenAI failure) and turning them into punchy, high-impact headlines. While the academic paper was turned towards scholarly readers these re-casts are designed for a real life context where decisions happen fast. Next, I moved away from explaining the law in my bulk writing to applying it as a practical tool for students through the Human Contribution exception. My use of bold font makes sure that my call to action is the first thing you’ll see on the page.	I feel that the biggest challenge for me was condensing the complex legal part of it (the Human Contribution exception) into a simple three page pamphlet. A major reward was seeing how my technical skepticism could be translated into a visual campaign that actually feels like a “full-bodied experience” for the viewer. This project revealed that even the most broken algorithm can be successfully challenged when you take the argument off the page and put it into the hands of the people affected. The transition taught me that a professional level project isn’t just about good grammar…it’s about matching the medium to the needs and values of the audience. Overall, creating these recasts showed me that visual arguments have a global impact that traditional papers rarely achieve.

An artist’s illustration of artificial intelligence (AI). This image explores how AI can be used advance biology and protein folding. It was created by Twistedpoly as part of the Visualising AI project launched by Google DeepMind.

Project Reflection

Posters (Recast #1)

Pamphlet (Recast #2)

A face of a humanoid robot, side view on black background