Can you fool it? We tried nine ways to disguise AI text

The obvious way to sneak AI text past a detector is to rewrite it: soften the phrasing, paraphrase it, run it through another language and back. We spent a while doing exactly that to our own detector, nine different ways, and kept only the rewrites that still said the same thing. Against a single one of those meaning-preserving rewrites, the detector still caught 94% of the AI text. That number is most of the story. The rest of this post is how we got it, and where it breaks.

Why attack your own model

Accuracy on clean, unedited AI text is the easy exam. It tells you how a detector does against someone who isn't trying. The interesting question is the adversarial one: what happens when a motivated person actively tries to disguise the text? If a thirty-second rewrite makes AI writing invisible, a headline accuracy number is close to meaningless.

So we built a second benchmark whose only job was to make our own model fail. We'd rather find the hole ourselves than have someone find it for us.

The method, honestly

At a high level: we took AI text the detector reliably flags and asked other models to rewrite it nine ways, paraphrase, casual register, compression, back-translation, restructuring, and a handful more. The important step is what we threw away. Any rewrite that changed the meaning went in the bin, because a rewrite that turns the argument into a different argument isn't a disguise, it's a different document. We kept only the rewrites a careful reader would call the same text, said differently, and then measured how many of those now slipped through.

We're describing this at the level of a robustness report, not an evasion recipe. The point isn't to teach the trick; it's to tell you whether the trick works.

What held, and what bent

Mostly, it held. Paraphrase, restructuring, and back-translation each evaded the detector roughly 0% of the time, rephrasing the sentences, shuffling the structure, or bouncing through another language left the underlying signal intact.

The attacks that bit hardest were the ones that lowered the register: casual, compressed, or hurried rewrites. Those got about 15–20% of the AI text past the detector. That's a real dent, and worth being honest about, but it also means four out of five of even the best-working rewrites were still caught.

We also tried stacking attacks: rewriting a rewrite of a rewrite. Even three chained together, depth-3, still left the detector catching about 94%. Piling on transformations didn't compound into an escape; it mostly just produced worse writing that we still flagged.

The one thing that surprised us

Here's the finding we didn't expect, and the one we'd defend. Look at which attacks worked. The rewrites that slipped through weren't the clever ones, they were the ones that genuinely changed the text's character, clipping it shorter, rougher, less careful. The rewrites that faithfully preserved what the text said, and how deliberately it said it, we caught almost every time.

Put plainly: to make AI text read as human, you mostly can't just change how it's written. You have to change what it says, and once you've done that, it's arguably not the same text anymore. Meaning-preserving evasion, the thing everyone assumes is easy, is the thing that mostly doesn't work.

The failure that actually matters

None of this is the failure that actually worries us. A detector that occasionally misses a disguised AI paragraph is a manageable problem. A detector that wrongly accuses a human is not, and that's the direction where ours is genuinely weak.

We assembled a set of the most AI-looking genuine human writing we could find, the real documents most likely to trip an alarm, and pointed the detector at them. At a loose threshold, it wrongly flagged as many as 44% of them. At our strict operating point, that fell to about 13%. Either way: these are real people, writing in their own words, being told a machine thinks they didn't.

That asymmetry is the whole reason we frame every catch rate against its false-positive cost. Missing a disguised AI document is a miss. Flagging a real person's honest writing is an accusation. The second failure is more expensive, and it's the one we spend most of our time trying to shrink.

Limitations

This is one detector, ours, measured on our own benchmark. "Meaning-preserving" was judged by a mix of models and heuristics, not a perfect oracle, so some borderline rewrites surely leaked either way. Nine families of automated rewrite is not every possible attack, and an automated rewrite is not the same adversary as a patient human editor with intent. And attacks evolve: this is a snapshot from today's models, not a permanent guarantee. We treat all of these numbers as a signal to keep testing, never as proof of anything.

Try it on your own text, and read the full attack-by-attack breakdown, at Verdict.