Why perplexity-based AI detection stopped working

For about two years, perplexity was a genuinely good way to tell whether a machine had written a sentence. Feed a piece of text into a language model and ask it how surprised the model is, token by token, at the choices the writer made. Average that surprise across the text and you get a single number: perplexity. Text a model finds predictable scores low. Text that wanders, contradicts itself, or reaches for an odd word scores high. Early detectors set a threshold on that number and, against GPT-2 and the first wave of GPT-3 output, it worked remarkably well.

The intuition behind it was sound. A language model trained to maximize the likelihood of its own output will, if sampled conservatively, produce the most probable next token more often than a human would. Human writing carries the residue of a mind doing something other than optimizing for fluency — tangents, hedges, specific and slightly inefficient phrasing, the occasional typo. Low temperature, low top-p generations from 2019-era models were close to the statistical mode of their own distribution. That gap between "predictable to the model" and "predictable to a human" was wide enough to build a classifier on. It has been closing ever since, and by most measures it is now too narrow to trust on its own.

A metric measuring its own blind spot

The flaw was there from the start, it just didn't matter yet. Perplexity does not measure whether a human wrote a sentence. It measures how closely that sentence matches the distribution of a specific reference model. Those are only the same question when the space of "how a machine might write" is narrow — one model family, one decoding strategy, low temperature, no instruction tuning. As soon as that space widens, the metric stops tracking authorship and starts tracking decoding configuration.

It widened fast. Instruction tuning and RLHF trained models to sound less like a raw next-token predictor and more like a person explaining something — hedging, varying sentence length, using discourse markers like "that said" and "to be fair." That shift wasn't aimed at detectors, it was aimed at human preference raters who liked text that read naturally. But the side effect was the same either way: self-perplexity went up. Add higher sampling temperature, nucleus sampling, repetition penalties, and a prompt that says "vary your sentence structure," and a model can raise its own perplexity almost on demand, without changing what the text says or how coherent it is.

Perplexity never measured whether a human wrote the sentence. It measured whether the sentence looked like something a particular model would write, sampled a particular way. — from an internal review of first-generation detectors

The gap compressed in about eighteen months

Track detection accuracy of a perplexity-only classifier across model generations and the trend line is steep. It's not one dramatic failure, it's a steady erosion, generation over generation, as base perplexity for "typical" synthetic text crept toward the human range from both directions — machine text got more varied, and detectors calibrated on older models fell further out of date with each release.

Paraphrasing made a weak signal collapse

Even the residual gap in the chart above assumes an adversary who isn't trying. A far simpler attack finishes the job: run the output through a paraphraser, even a small, cheap one, and perplexity moves toward the human range almost immediately, because paraphrasing is precisely the operation of trading the most probable phrasing for an equally valid, less predictable one. Meaning survives. The statistical fingerprint doesn't. A cottage industry of "humanizer" tools now exists to do exactly this on purpose, and none of them need to know anything about how the underlying detector works — they just need to raise perplexity, which is a much easier target than "write like a human."

Here's roughly what that looks like in practice, comparing self-perplexity of the same underlying content before and after one paraphrase pass:

# perplexity under a reference model, before/after paraphrase
# illustrative figures from a single held-out sample

human_text_baseline        = 38.2
model_output_raw           = 11.4   # cleanly separable in 2019
model_output_paraphrased   = 33.6   # inside the human range

def perplexity(logprobs):
    n = len(logprobs)
    avg_nll = -sum(logprobs) / n
    return exp(avg_nll)

None of this means perplexity is useless. It's still a cheap, informative feature — it just stopped being a sufficient one somewhere around the point instruction-tuned models became the default way people generated text. Treating it as a standalone classifier is the part that broke.