The Anxious Culture of AI Detection

In the middle of last spring semester, the TikTok algorithm suddenly keyed into my role as a teacher and starting surfacing two parallel streams of videos. First, I was seeing professors like those on #ProfessorTok complaining about students—these professors often claim to be offering students insight into how college actually works, but their collective output amounts to a chorus of “Read the Syllabus.” Alongside the professors ran a parallel stream of students—if “Read the Syllabus” is the call, “please be reasonable!” seems to be the most characteristic response as students complain about rigid enforcement of late policies where a single second makes the difference, their (sometimes correct) impression that professors want them to fail, their frustration with professors phoning it in in lectures and just reading off the slides, and their eye-rolling at professors who are insensitive and/or passive-aggressive in emails.

In the midst of this stream of complaining students appeared Caroline Mincks, whose story about a professor’s irresponsible use of AI detection appeared in my feed in medias res¹:

@carolinemincks

♬ original sound - Caroline Mincks

At this point in the saga, Mincks had been accused twice of using generative AI on discussion-board posts in her online class. In this particular video, she has made the rather bold choice to run the professor’s emails through an AI checker and found percentage scores similar to the ones with which it had tagged her own work. The professor does not love that as a student strategy, but nevertheless is a bit cowed and withholds final judgment on grades for a time. By the end of the video, both student and teacher remain in limbo.

Mincks’s saga unfolds over weeks—I count at least fourteen videos documenting it (1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14). From the start, Mincks vigorously denies AI use—in fact, long before the interactions with her professor began, Mincks had regularly been posting anti-AI screeds, such as this one involving an interaction with a classmate who scoffs that ChatGPT had arrived at a different answer than she had:

@carolinemincks

The universe thinks it’s soooooo funny with its timing and its baiting me to be an asshole specifically

♬ L.Boccherini, Minuet from String Quartet No.5 in F major - AllMusicGallery

My understanding of events as they unfold:

• Mincks’s professor marks her grade down on a discussion post based on a 35% AI score from an unnamed AI detector.
• Mincks is horrified by the accusation and reaches out to her teacher over email. As she describes it, “I, of course, immediately contacted my teacher, as in about one minute after she posted my grade, and said, hi, uh, there's been a mistake. I am extremely anti AI. I don't touch it. I didn't so much as open another tab to do this opinion based discussion post, let alone do it with AI.”
• Mincks reconstructs a Google Docs Version History with a newly rewritten version of the post (before I move on with the summary, I’ll briefly observe that many teachers, including me, would treat this choice as suspicious, or at the very least as wildly unconvincing evidence to prove the student’s innocence). She shares that Google Doc with her professor. Mincks expresses her fears that she’ll always have to work this way going forward.
• The teacher marks Mincks down for detector-driven AI suspicion on a second discussion-post assignment.
• Mincks’s request for a more detailed accounting of what specifically about the posts is raising suspicions go unanswered.
• The professor invokes a second AI detector, apparently believing a second dubious AI tool will offset the flaws of the first.
• The professor expresses a preference for results that show 100% human-written, a score achieved by some classmates.
• Mincks runs the professors’ emails through an AI detector that finds 82.1% and 47.1% AI scores.
• The professor pauses entering grades but also writes, "I would suggest that grading an instructor's writing is usually not a great way to win them over to your side of an issue.”
• Eventually, the professor offers the entire class a new option for discussion posts, to make a video instead of writing a post.
• Mincks escalates the issue to the department level and undergoes a slow resolution process.
• And so on. I haven’t been able to find a video that documents the resolution of the departmental process which must have happened before the end of the school year.

I hope Mincks was not found responsible for AI use on her assignment, because whatever mistakes she made in the process (I’m counting “fabricated Version History” as an obvious mistake and forgiving “passively aggressively put the teacher’s emails in an AI detector” as a reasonable choice, given the circumstances), there just seems to be no evidence beyond an utterly opaque score from unnamed AI detectors that she did in fact cheat on the assignment. What captivated me about the experience, though, is the utterly preposterous and contentious battle between teacher and student that the belief AI detectors can stop the deluge of student AI use has produced.

The professor’s Learning Management System seems to show them an AI score without a specific request, and when that number creeps up, the professor believes it surely must mean something—if it weren’t good enough to stick in the Learning Management System, surely administrators wouldn’t stick it there (some teachers and professors sincerely seem to believe this, even after they’ve used these notoriously janky systems for years). And if one detector isn’t offering confidence, surely multiple detectors will, so the professor pastes the student’s work into more online systems to produce more dubious numbers, as if checking the results of one electronic Ouija Board against another electronic Ouija Board will turn either one into something other than an electronic Ouija Board. The professor is unable to offer any particular information to the student about the deficiencies in her work that pointed to AI use, for the simple reason that they have no idea—the black-box algorithm has told them nothing about it beyond the scary-looking number, which nevertheless, as said before, must mean something. The professor doesn’t want to be unjust to students, but they also don’t want to let the AI use they know runs rampant in higher education to go unchecked—so something must be done with those numbers until teacher and student alike find themselves in a weeks-long formal resolution process that produces no satisfying resolution for either party.

Mincks’s position, of course, is even less enviable than the professor’s—the professor at least has a modicum of power in the situation. She knows the number produced by the AI detector is finally utterly dubious—to her, the detector is just one more AI system to hate because of its notorious unreliability. Her initial arguments prove unconvincing to the teacher: "One of my school assignments got flagged as 35% AI. Now, here's the problem with that. I am me, so that can't be true.” The fact that so polls show so many students having disdain and hatred for AI even as polls show so many students using AI, of course, do point to a significant set of students who claim to hate AI at the same time they use it regularly. I don’t think Mincks is in that group, though—her account of actually writing the thing seems utterly convincing to me. But what other arguments can she really make? She’s written the post in the text box of the Learning Management system, so she doesn’t have a Google Docs Version History. But, perhaps as she looks through online fora focused on AI accusations, she finds that the Version History is important—so she produces one, however dubious it is. The dubious document proves nothing, so she finally pulls the trick of showing the professor that their own work would produce (presumably) false positives in the notoriously janky AI detectors—but at the cost of escalating the confrontation with the professor. I don’t know Mincks, and perhaps I’m naive to assume her innocence in the situation. I’ve seen no reliable evidence that would prove her guilt, though—and, unless we accept the numerical vibe-prophesies of an electronic Ouija Board as a source of reliable truth in the world, neither has her professor.

Mincks is not alone among the falsely accused, and a series of Reddit communities has developed around AI detection and AI-use accusations, and spending time in these communities points to the toll the AI-detection wars are taking. r/AccusedOfUsingAI, for example, doesn’t start with judgment—it bills itself as “A space for students who’ve been called out, flagged, accused, or questioned for using AI in their essays, whether the accusation was false or true.” Some of the posts come from students who did indeed use generative AI, were caught, and didn’t know what to do. When one user admits “I know I was dumb to use AI” and doesn’t want a lecture but does want advice, the commenters generally give advice that most professors would approve of. Several college professors and teachers pipe in to suggest telling the truth. A self-identified English professor gets the most-upvoted comment: “It is so much better to tell the truth. Just own up to it, say you felt stressed, and made a mistake. That will have the best outcome. Don’t dig a deeper hole.” Those voices dominate, but there are some who caution against doing so: “You should carefully consider if you want to admit to academic misconduct. I personally would not.” I’d clearly advise the student to be honest, but there are also enough severe potential consequences to doing so that I can’t wholly dismiss that advice.

Most posts, though, come from students who express frustration with having been falsely accused. As they navigate Kafkaesque processes similar to the interactions Mincks describes, they’re advised repeatedly: be sure you’ve got a fleshed-out Google Docs Version History or have Track Changes turned on in word; respond to your instructor calmly and patiently, and don’t panic; show other samples of your work to provide evidence of your preexisting style. All reasonable suggestions, I suppose, but they don’t work for every circumstance, especially for students who weren’t using Version History or Track Changes already. And the problem with all the advice is that they simply don’t prevent false accusations from professors trusting the confident-looking detector numbers over other evidence. The palpable panic of the falsely accused emerges over and over again in the forum.

Another ecosystem of subreddits has emerged to preemptively check student work against AI detectors. Groups including r/CheckTurnitin point students to Discord servers where they can upload their work. A user with access to Turnitin’s tools gives them back the otherwise mysterious percentage. Students alter their work until the percentage seems acceptable to them, a phenomenon Dadland Maye has written about in The Chronicle of Higher Education. In posts and comments, folk theories about what’s driving the algorithm abound: “i think it really hates starting sentences with therefore or having too many long ones back to back,” “Anytime you use a colon of any type seems to trigger it. I also found using just normal words like “mirrors” will trigger it,” “Well crafted, formal writing will always be flagged for potential AI.” Many believe they must dumb down their work to reduce their percentage scores: “now im second guessing if i need to make stuff sound more casual or leave in a couple minor awkward phrases to not get flagged,” “
seriously though this thing hates normal college writing now i swear,” “I need to write like a dumb to pass AI detection.” The paranoia doesn’t just circulate within these communities but is adopted by college influencers. Harlan Cohen, for example, whose insights reach 1.7 million across different platforms, argues not that students may be accused of AI cheating but that they will be:

@helpmeharlan College Tip 1585: If you’re a student writing a paper assume you will be accused of using AI and cheating #cheating #professor #students ♬ original sound - Harlan Cohen

If you're a student writing a paper, assume your teacher will accuse you of using AI. They will accuse you of cheating. And if you are an extraordinary writer, you are even more at risk. Because when they read extraordinary writing, they think this must be AI. And the problem is, you're guilty until you're proven innocent.

Wise advice, perhaps, as a self-protective measure for students—but so incredibly bleak for those of us who care about classrooms and policies that inspire students’ curiosity rather more than their compliance.

For professors puzzling over AI detector results and communities of students striving to avoid either the detection of AI use or false accusations of AI use, the numerical result of the detector takes on a talismanic power: teachers and students alike worship it and fear it, and it, rather than skills and content, becomes the center of the educational experience.

In my department’s first discussion after a faculty-meeting presentation about ChatGPT in early spring 2023, the first questions surrounded AI detection. Turnitin has existed to detect plagiarism for years, and we’d just seen a demo in which an algorithm could write an argument. Surely some digital tool would be able to tell us what was made by machines and what was made by people. The market for such a product existed, and competitors arose to sell it: GPTZero, ZeroGPT, CopyLeaks’s AI Detector, Turnitin’s AI Detection, launched in April 2023, Pangram, Grammarly AI Detection, Quillbot AI Detection, and others. Most of these companies also hawked wares to students looking to “humanize” their AI output, though, so the same algorithmic tool that produces the mysterious number that prompts accusations can be used to finesse AI output into an undetectable form. Given the two directions, I’ve had colleagues become convinced that a 100% human score is itself suspect, as implicit proof of machine authorship paired with humanization.

So far as I can tell, these products have thus far produced virtually no good outcomes. Students quickly picked up on the true—but often repeated in suspiciously verbatim manner—sentence, “AI detectors are notoriously unreliable,” a sentence I have heard in multiple disciplinary hearings. Some conversations were started, I suppose, and at institutions that put blind faith in AI detectors, some AI offenders were punished (as, surely, many innocent of AI use were). I’ve seen colleagues spin themselves into contortions of AI numerology and meeting with dozens of students in a week in attempts to crack down on use. I’ve heard students talk about losing trust in teachers after mass accusations, and describe in detail how impossible they find the idea of talking openly with adults about AI, given the taboos established around its use. What I haven’t seen is any clear evidence that the electronic Ouija Boards are having any particular deterrent effect on students’ AI use, any salutary effect on classroom communities, or any production of certainty in pursuit of AI accusations.

That original dream, though—if only we had the right AI detector, then everything would be just like it was and fine—stays alive. Recently, Pangram has been receiving more attention for taking a different approach to AI detection. Roughly, where other approaches emphasize a formal sentence-level metric called “perplexity,” Pangram uses machine learning on a large data set of human writing and AI writing to make what it calls “An AI Detector that actually works.” The company’s CEO acquitted himself well on a recent episode of the Vergecast, and the Authors Guild found that Pangram produced the fewest false positives of the AI Detectors it recently tested. The company’s claims feature impressive numbers for low false negatives and low false positives. Perhaps they’ve cracked the code, and the one true AI detector has actually emerged. If it finally proves reliable, though—and call me skeptical, given the continuing changes in generative the AI technologies themselves and what common practice counts as acceptable use—we’ll still be left with a mysterious, unsourced number as the only piece of evidence. Teachers and administrators basing claims on Pangram results will still have to say, “Trust AI Company Pangram (TM)—we do!” And when a student is falsely accused—even the most optimistic projections point to many students still getting falsely accused at scale—that will have to be where the evidence stops because of the nature of a black-box system. That number may be enough evidence to satisfy some teachers and some administrators, but it wouldn’t be enough to satisfy me, and I suspect it won’t be enough to satisfy the lawyers the affronted students who can afford it will hire to loom in disciplinary meetings.

AI Detectors also assume a completely black or white approach, and many of my colleagues in the humanities believe in a black or white approach. They write long missives to students in the form of AI syllabus statements about the evils of turning learning or writing over to an algorithm, hoping their moral appeals will finally win over students primed to be skeptical of AI companies. Any work tainted by proximity to the algorithm was never work in the first place, and the righteous work will be marked as such once the just and right detector comes along. The reality of AI use at present, though, features a whole lot of grey area. If a student triggers a Google AI summary in the course of their research, have they violated an all-or-nothing AI policy? If they ask the new Siri for a definition of a word, have they violated it? If they brushed up on background materials in conversation with an LLM before they sat down to write something themselves, have they violated it? If they hit tab to autocomplete a sentence in Google Docs, like their professors and teachers routinely do while using Google Docs comments, have they violated it? A confident-looking number from a black-box AI company is unlikely to resolve any of those questions in a satisfactory way.

Pick your metaphor—cat and mouse, Wile E. Coyote and Roadrunner, arms race—whatever it is we’re doing right now, it’s not working, and the path to a better future for K-12 and Higher Education is not paved with more AI detectors or better AI detectors. Assessments completed out of class have not been the same since November 2022, and that’s a real shame—but doubling down on policing in search of some irrecoverable past is only creating an environment in which students and teachers alike are filled with anxiety and distrust around classwork.

The solutions so far aren’t fully satisfying. There were good reasons writing teachers were skeptical of in-class assessment for so long, and few would pretend that an in-class essay does anything similar to the work of an out-of-class essay circa 2015. But that past is not coming back, and we need to be looking toward futures more interesting and hopeful than the continuing AI-policing moment.

It’s not perfect, but the Two-Lane Approach to Assessment proposed by Danny Liu and Adam Bridgeman is more promising than either the policing approach or the purely in-class assessment model proposed in, for example, The Atlantic. Briefly, Liu and Bridgeman propose two lanes in any given class. In one, students complete assessments in a secure environment to guarantee essential student learning. In the other, out-of-class assignments are designed with the assumption that students might use generative AI. I’ve yet to fully adopt this logic, but I really don’t want to experience again one of the AI-cop spirals that so many of us have enacted and experienced in the last 4 years. Students can’t just get credit for AI slop from which they’ve learned nothing, but the more we focus our energies on catching the slop, the less likely we are to come up with something better than the old ways that just no longer work.

¹ I was thinking about Mincks’s case while I was writing about the “Trojan Horse” prompt approach some instructors have used in my last post, Against Shady AI Policing in Education. Mincks’s professor seems to be acting in fully good faith—there’s no intent to trick or deceive here. But I think the professor’s stark misuse of AI detectors points to the claim I gestured toward in my conclusion—I’m against specifically shady AI policing, but in the end, I’m starting to think AI policing is generally pretty shady, and I think Mincks’s case shows that, too.