Writing

Field notes on how we measure the detector, and where it breaks. Every post leads with a real number.

• Introducing Verdict: an AI detector that publishes its own error rate. Detection is a screening signal, not proof, so we publish our real held-out numbers, including the places we're weak.
• Inside our benchmark: 15,899 documents, 30 generators, and the number we almost got wrong. How we measure whether an AI-text detector actually works, and how we caught ourselves inflating our own headline number.
• The surrogate gap: why a tiny model spots ChatGPT but misses everything else. One zero-shot detector, about 0.99 against clean ChatGPT and about 0.55 across thirty other generators, which is why a single demo number tells you almost nothing.
• Can you fool it? We tried nine ways to disguise AI text. We built a benchmark whose only job was to break our own detector. It mostly held, and the real weakness wasn't the one anyone worries about.