Security· 16 min read

Who Wrote This? What AI-Text Detection Can and Can't Tell You

Whether a text was written by AI is really three questions. Watermarks can sometimes answer one of them, detectors guess at another, and the third is about who takes responsibility.

A printed report with blurred text and charts on a wooden desk, a brass jeweller's loupe resting on the page. A fountain pen, a nearly empty coffee cup, a plant, a leather notebook and an out-of-focus laptop sit around it in warm window light.

Most offices have had the moment by now. A colleague who usually sends three terse lines suddenly delivers a two-page status update, with headings, a balanced risks section, and the word “robust” twice. Nobody says anything. A few people quietly wonder whether they finally took that writing course, or took a shortcut. Asking would be awkward. Pasting the text into a free online detector feels much less awkward, which is a small preview of the problem.

For a status update it hardly matters. It’s either accurate or it isn’t, and Thursday’s steering meeting will tell. The question gets serious elsewhere. A work sample can decide a hire. An answer to a regulator or an auditor has to reflect what the institution actually knows. And someone can end up in a disciplinary process because a tool gave their writing a high score. In cases like these, knowing whether and how AI was involved changes a decision about a person or a contract.

That’s when people want a yes-or-no answer to “was this written by AI?”, and that’s where it goes wrong. It sounds like one question. It isn’t, and most of the confusion around detection tools and the new EU rules comes from treating it as one.

Something did change this year, though. Google, Anthropic and, inside the EU, OpenAI now say they put an invisible statistical watermark into the text their assistants generate. For the first time there’s a mechanism, at scale, that can sometimes say “this came from our model”. That’s real progress, but it establishes less than most product pages suggest.

Three questions hiding in one

Origin: did a specific system produce these words? It can only be answered with the cooperation of whoever ran that process, for example through a watermark or a log.

Resemblance: does this text look like what language models produce? This is a statistical question. Detectors answer it with a probability, and that probability describes the text, not how it came to be.

Accountability: who stands behind this text and answers for it? That’s not a technical question at all. A contract drafted by a lawyer with heavy AI help is still the lawyer’s contract.

The three come apart all the time, and most real documents are now mixed: a human outline, a model draft, a human rewrite, a model proofread. For a text like that, “human or AI?” has no correct answer. The useful questions are how much of it a model wrote, and who signed it off.

Only a watermark hit is strong evidence, and only for one questionA grid of three questions, origin, resemblance and accountability, against what each asks, how it is answered, what a yes tells you and what a no tells you. Only the origin column's yes, this model was involved, is highlighted.Only a watermark hit is strong evidence, and only for one questionThree questions hide in “was this written by AI?”. They are answered differently, and a no means little.OriginResemblanceAccountabilityAsksDid this systemproduce these words?Does it lookmachine-made?Who stands behind itand answers for it?Answered byA watermark or log,held by the providerA detector score onthe finished textReview, sign-off,and recordsA yes tells youThis model was involved(given enough text)The style is predictable,for whatever reasonA named personowns the textA no tells youAlmost nothing: otherkeys, edits, short textNot much either:editing hides the styleNo one owns it, whichis the real problemMost real documents are mixed, so the useful questions are how much a model wrote, and who signed it off.
Only a watermark hit is strong evidence, and only for one of the three questions.

How a model writes, and why that leaves a trace

A language model doesn’t write sentences. It predicts a probability distribution over the next token (a word or part of a word), picks one, appends it, and repeats.

We read words. A model reads tokens.The English sentence "The steering committee decided to postpone." split into nine tokens, with steering and postpone each cut into two pieces. The German sentence "Der Lenkungsausschuss vertagte den Entscheid." split into ten tokens, with the compound Lenkungsausschuss cut into four pieces. The split is illustrative.We read words. A model reads tokens.Illustrative split: every model's tokenizer cuts a little differently. ␣ marks a space that belongs to the token.We readThe steering committee decided to postpone.The model readsThe␣steering␣committee␣decided␣to␣postpone.9 tokens for 6 words. Common words are one token; tinted ones are a word cut into pieces.Same in GermanDer␣Lenkungsausschuss␣vertagte␣den␣Entscheid.Long compounds such as Lenkungsausschuss break into several pieces.At every step the model scores every token in its vocabulary and picks one.Detectors and watermarks both work on those picks, not on words.
Illustrative split. Every model’s tokenizer cuts text slightly differently.

Take “The steering committee decided to ___”. The model might give “postpone” 30%, “approve” 25%, “reject” 10%, and spread the rest over thousands of options. Which one gets picked depends on a random draw, shaped by settings like temperature.

Two things follow from this. Models tend to pick words that are likely in context, so their output is, on average, less surprising to another language model than human writing is. Most detectors measure some version of that surprise, or train a classifier on it. And the random draw is a place where you can hide information without changing what the text says. That’s where watermarks live.

Watermarks: the model signs its own work

I’ll admit my first mental model was naive. I pictured a watermark as some kind of hidden formatting, invisible characters or metadata riding along with the text, and assumed that pasting it into a plain text file would wash it out. That’s how it works for a Word document’s author field. A text watermark lives somewhere else entirely: in which words the model chose. Anything that keeps the words keeps the mark, whether that’s copy and paste, a .txt file, a new font, or an email client that strips every bit of formatting.

The basic idea was published by Kirchenbauer and colleagues in 2023.1 Before each word is chosen, a secret key and the preceding tokens are used to split the vocabulary pseudo-randomly into a “green” and a “red” list, and the model gets a small push towards green. A human writer knows nothing about the key, so about half of their words land on green by chance. Watermarked text lands on green noticeably more often. The detector recomputes the lists with the key and tests whether green turns up more often than chance allows.

A watermark only works where the model has a real choiceLeft panel: for The steering committee decided to, postpone has 30 percent, approve 25 percent and reject 10 percent; the key favours approve and reject, which get a small extra push. Right panel: for The capital of Switzerland is, Bern has almost 100 percent, so there is nothing to nudge. Illustrative.A watermark only works where the model has a real choiceIllustrative probabilities. Green marks the words the secret key favours in this context; dashed is the small extra push.Real choiceThe steering committee decided to ___postpone30%approve25% + pushreject10% + push… and thousands of other tokens with tiny sharesThe pick tilts slightly towards green words.The sentence still reads normally.No real choiceThe capital of Switzerland is ___Bernalmost 100%ZurichtinyOne right answer. Nudging it would changethe meaning, so nothing gets watermarked.The detector recomputes the green lists with the key and counts how often the text picked green.Human text lands on green about half the time; watermarked text noticeably more, over hundreds of tokens.
Illustrative probabilities.

Google DeepMind’s SynthID-Text refines this. It was published in Nature in 2024, deployed in Gemini, and open-sourced.2 Anthropic’s watermark, announced in August 2026, is a version of SynthID-Text. It applies worldwide to Claude models launched from 2 August 2026, and Anthropic extended it to several older models on 3 November.34 OpenAI built a text watermark back in 2024 and held it back, reportedly partly because a share of users said they’d use ChatGPT less.5 In October 2026 it started watermarking ChatGPT and Codex text in the EU only, with watermarking in the API opt-in worldwide.6

It’s the strongest tool available, and a positive result does mean something. On enough text, the chance that unwatermarked writing scores high is small and can be quantified. No style-based detector has that property.

A negative result means very little. OpenAI says plainly that a missing watermark doesn’t prove human authorship.6 The text might be too short or edited. It might come from another provider with a different key, from a model released before August 2026, or from an open-weight model running on someone’s laptop, where nobody has to switch a watermark on.

The signal is also thinner than the headlines suggest.

  • It needs length. Each word carries only a little evidence. OpenAI reports about 80% detection at 200 tokens and about 95% at 400 tokens on answers to psychology questions, at a 1% false-positive target, with worse results on maths.7 The EU Code of Practice only expects watermarking of free-form text longer than 200 tokens.8 Plenty of emails are shorter than that.
  • It needs choice. Where only one word is correct, there’s nothing to nudge. Anthropic says factual passages and code where exact output matters generally aren’t watermarked.3
  • It fades with editing. In OpenAI’s tests on 400-token passages, swapping 10% of words for synonyms dropped detection from about 92% to 66%, and swapping 25% dropped it to 17%.7 A full rewrite, by a person or by another model, removes it.
  • Only the key holder can check. Every provider has its own key. As of November 2026, Anthropic’s detection API is in private preview for selected organisations such as regulators and researchers, and OpenAI’s detector is limited to approved researchers.46 Google’s public SynthID Detector, opened in October, checks images, video, and audio, but not text.9 Until then, an ordinary organisation that wants to check a document mostly can’t.
  • Nobody outside can verify the claims. A September 2026 preprint argues that deployed configurations and error rates can’t be checked from outside, and that this unverifiability, more than the watermarking itself, is the real governance gap.10

There’s a theoretical ceiling as well. Zhang and colleagues showed that under assumptions which get easier to meet as models improve, an attacker with a way to judge quality and a way to make small changes can wash out a watermark without noticeably hurting the text.11 So watermarks raise the cost of laundering text without making it impossible.

Detectors: reading the tea leaves

Detectors work without anyone’s cooperation. They look at a finished text and estimate whether it resembles model output. This is the kind of tool sold to schools and HR departments, and its history is sobering.

OpenAI launched its own classifier in January 2023. It caught 26% of AI-written text and labelled 9% of human text as AI. Six months later OpenAI withdrew it, citing its low accuracy.12 A Stanford study the same year ran seven popular detectors over essays that non-native speakers wrote for the TOEFL exam. On average 61% of these human essays were flagged as AI-generated, while essays by US eighth-graders were classified almost perfectly.13 A limited vocabulary and simple sentences look unsurprising, and unsurprising is what the detectors learned to flag. In a country where much of the business English is written by people whose first language is German, French, or Italian, that’s not a side note.

It would be wrong to stop there. Binoculars, a zero-shot method from 2024, caught over 90% of ChatGPT text at a 0.01% false-positive rate on its benchmarks.14 A 2025 study found that five people who use ChatGPT a lot for writing, voting together, misclassified only one of 300 articles, which beat most commercial detectors.15 On long English prose from known models, under friendly conditions, detection can be very good.

The conditions that matter are rarely friendly. The RAID benchmark tested twelve detectors across 11 generator models, 8 domains, and 11 adversarial attacks. The detectors were easily fooled by paraphrasing, different sampling settings, repetition penalties, and models they hadn’t seen.16 A detector’s published accuracy describes its own test set, which tells you little about your documents or about someone actively trying to get past it.

The arithmetic nobody puts on the slide

Even a good detector runs into base rates. Say you screen 1,000 documents, and 20 of them really were machine-written. Your detector catches 95% of those and wrongly flags 1% of the human ones. That sounds excellent. It gives you 19 correct flags and about 10 wrong ones, so a third of the people you’d confront did nothing wrong.

If the share of AI text is lower or the false-positive rate is higher, false flags soon outnumber true ones. OpenAI made the same point about its own watermark in 2024, noting that a low false-positive rate per text still adds up to many misidentifications at volume.5

A third of the flagged documents were written by peopleA grid of 1,000 dots. 19 machine-written documents are caught, 1 is missed, 10 human-written documents are wrongly flagged and 970 are not flagged. 29 flags in total, 10 of them wrong.A third of the flagged documents were written by people1,000 documents, 20 machine-written. A detector that catches 95% and wrongly flags 1% of human text.19 machine texts caught1 machine text missed10 human texts flagged970 human texts, not flagged29 flags, 10 of them wrongFewer machine texts or a higherfalse-positive rate, and the wrongflags soon outnumber the right ones.
Dot positions are spaced evenly for readability.

Where detection does work: counting, not convicting

There’s one setting where resemblance methods hold up well, and that’s measuring prevalence across a large body of text rather than judging single documents. Kobak and colleagues analysed over 15 million PubMed abstracts and tracked words whose frequency jumped after ChatGPT appeared, “delves” being the best-known one. They estimated that at least 13.5% of 2024 biomedical abstracts were processed with an LLM.17 The method never says anything about a single abstract, because the signal only exists in the aggregate.

In an organisation, if you want to know roughly how much of a team’s output is AI-drafted, or whether a supplier’s submissions changed character after a certain date, statistical methods can give a useful answer. If you want to decide whether one specific person cheated, they can’t.

What the EU actually asks for

The regulation reflects these limits better than it gets credit for. Article 50(2) of the AI Act requires providers of generative systems to mark their output, text included, in a machine-readable and detectable way. It applies from 2 August 2026, and the Digital Omnibus (Regulation (EU) 2026/1744) gave systems already on the market until 2 December 2026.18

Article 50(4) is the more interesting part for anyone who publishes. Deployers who publish AI-generated text to inform the public on matters of public interest must disclose it, unless the text went through human review and someone holds editorial responsibility for it.19 For text, then, the law doesn’t lean on detection. It asks who is accountable, which is the third question from the top of this post.

Switzerland has no equivalent of the AI Act. The Federal Council plans to implement the Council of Europe’s AI Convention, with a consultation draft due around the turn of the year.20 The effects reach Swiss organisations anyway: the assistants staff use follow EU rules, and anything published into the EU falls under Article 50(4).

What this means in an organisation

Don’t base decisions about individuals on a detector score. Not in hiring, not in disciplinary cases, not in supplier evaluation. A score is a probability about style, and used as evidence it will hit people writing in their second language hardest. If something matters enough to act on, look for process evidence, such as version history and whether the person can explain and defend the work.

Treat a watermark hit as a lead, and a missing watermark as nothing. If you get access to a provider’s detector, a strong positive on a long passage is useful evidence that a particular model was involved. It still can’t tell you how much a person contributed, and it can’t tell “Claude wrote this” apart from “Claude heavily edited this”.3

Know what your own tools mark. Microsoft documents watermarks for images, audio, and video made with Microsoft 365 Copilot, but I couldn’t find anything on text, and with automatic model routing you may not know which provider wrote a given paragraph.21 If origin matters for your own output, the more reliable record is the one you already control, such as retained prompts and responses and a record of who approved what.

Put accountability where the law puts it. For anything published or sent outside, name the responsible editor and write down what “human review” means in practice. That covers Article 50(4), and it answers the question people outside the organisation actually care about.

Ask vendors the boring questions. For any detection product, ask for the measured false-positive rate by population and language (native and non-native writers, German and French as well as English), the minimum text length, robustness to paraphrasing, which models it was tested against, and how often that’s repeated. If the answer is a single accuracy figure above 99%, the RAID authors have already described how far such claims tend to travel.16


  1. Kirchenbauer et al., A Watermark for Large Language Models, ICML 2023. ↩︎

  2. Dathathri et al., Scalable watermarking for identifying large language model outputs, Nature 634, 818–823 (2024). Implementation notes: Introducing SynthID Text. ↩︎

  3. Anthropic, How Claude’s text watermark works, 14 August 2026. ↩︎ ↩︎ ↩︎

  4. Anthropic Help Center, How Claude marks AI-generated content. The 3 November extension is from Anthropic’s release notes, as reported by Let’s Data Science. ↩︎ ↩︎

  5. First reported by the Wall Street Journal in August 2024. Summary with OpenAI’s statement: The Decoder. ↩︎ ↩︎

  6. OpenAI, Our approach to EU text provenance rules, 5 October 2026. Coverage: TechCrunch. ↩︎ ↩︎ ↩︎

  7. Figures from OpenAI’s post, as reported by BleepingComputer. The textGrain technical report covers the method and its theoretical calibration. ↩︎ ↩︎

  8. European Commission, Code of Practice on Transparency of AI-Generated Content. Summary of the text provisions: Freshfields. ↩︎

  9. eWeek, Google’s SynthID Detector Goes Public, 8 October 2026. ↩︎

  10. Nemecek, Chaudhary, Ayday, Watermarks Without Verification: AI Text Watermarking After the EU AI Act, arXiv preprint, September 2026. Not peer-reviewed. ↩︎

  11. Zhang et al., Watermarks in the Sand: Impossibility of Strong Watermarking for Generative Models, ICML 2024. ↩︎

  12. OpenAI, New AI classifier for indicating AI-written text, with the July 2023 update. ↩︎

  13. Liang et al., GPT detectors are biased against non-native English writers, Patterns, 2023. ↩︎

  14. Hans et al., Spotting LLMs With Binoculars, ICML 2024. ↩︎

  15. Russell, Karpinska, Iyyer, People who frequently use ChatGPT for writing tasks are accurate and robust detectors of AI-generated text, ACL 2025. ↩︎

  16. Dugan et al., RAID: A Shared Benchmark for Robust Evaluation of Machine-Generated Text Detectors, ACL 2024. ↩︎ ↩︎

  17. Kobak et al., Delving into LLM-assisted writing in biomedical publications through excess vocabulary, Science Advances 11(27), 2025. ↩︎

  18. Regulation (EU) 2026/1744 (Digital Omnibus on AI), published in the Official Journal on 24 July 2026. Summary: Cooley. ↩︎

  19. Regulation (EU) 2024/1689, Article 50(4). Summary of the Code’s deployer section: Jones Day. ↩︎

  20. OFCOM, Artificial intelligence. The SME portal says early 2027: KMU portal. ↩︎

  21. Microsoft Learn, Add watermarks to content generated or altered by using AI in Microsoft 365. Covers video, audio, and images, checked 19 November 2026. ↩︎