Turnitin's AI false positive rate
What Turnitin publishes about how often its AI detector is wrong about human writing, and how that compares to a detector calibrated for a low false positive rate.
Turnitin publishes two figures for how often its AI detector is wrong about human writing: under 1% of documents, and around 4% of sentences. Both come from Turnitin.
At institutional volume those rates stop being abstract. Ten thousand submissions a term at 1% is around a hundred documents where the indicator is wrong. The rate is knowable. Which specific document it applies to is not.
Why the rate is the number that matters
Detectors advertise accuracy. Accuracy blends two errors that have nothing in common: missing generated text, and flagging writing somebody wrote themselves. Only one of those lands on a person.
A detector can post an excellent accuracy figure and still produce a large absolute number of wrong flags, because the volume is large. The rate on human writing is the number worth asking about, and it is the one most tools do not lead with.
Where wrong flags land
They are not spread evenly. Turnitin's own analysis of falsely flagged human sentences found 54% sit directly beside a sentence the model scored as AI written, and another 26% sit two sentences away.
Four in five occur next to a genuine detection. Mixed documents, where drafted and written passages alternate, are the hardest case for every detector, and they are becoming the normal case as drafting tools get built into ordinary writing software.
Some writing sits nearer the boundary whoever produced it: regular sentence construction, a formal register, a narrow vocabulary range, repeated technical structures. Research has repeatedly found higher false positive rates for non-native English writers for those reasons. See why non-native English writing gets flagged.
What we measure
We calibrate backwards from the error that matters. Rather than picking the threshold with the best headline accuracy, we fix a tolerable false positive rate and accept whatever detection rate follows. The threshold is the score only 1% of known human documents exceed.
Measured on held-out pre-LLM academic writing:
| operating point | wrong about human writing | AI academic text caught |
|---|---|---|
| strict | 0.4% of documents, 99.6% specificity | 88.3% |
| public default | 0.8% of documents | 88.3% |
Ten times fewer wrong flags on real academic writing, on a corpus of 15,900 documents built for exactly this measurement. We publish the number rather than a single blended accuracy figure. The method is in how the unslop AI detector works.
Run any text through it. Free, unlimited, no account.
Sources
- Turnitin, Understanding the false positive rate for sentences of our AI writing detection capability
- Turnitin, AI writing detection update from Turnitin's Chief Product Officer
- Turnitin, AI writing detection capabilities FAQ
- Liang, Yuksekgonul, Mao, Wu and Zou (2023), GPT detectors are biased against non-native English writers
- unslop, How the unslop AI detector works: our own figures, their sample sizes and how they were measured