OpenAI will watermark ChatGPT text in the EU, and says 25% synonym swaps cut detection to 17%

OpenAI will watermark ChatGPT text in the EU, and says 25% synonym swaps cut detection to 17%

OpenAI will add an invisible watermark to ChatGPT and Codex text in the European Union over the coming weeks, but its own evaluation says a round of synonym swaps strips most of the signal straight back out. Replacing 10% of the words in a 400-token passage cut detection from about 92% to 66%, and replacing 25% of them took it down to 17%, according to the approach OpenAI published on October 5.

The ChatGPT logo and wordmark on OpenAI's blue and orange gradient

The watermark reaches eligible ChatGPT and Codex users on all plans in the EU, and OpenAI has published no date for it. Image: OpenAI

The watermark is called textGrain, and OpenAI describes it as "an invisible statistical signal" added to the model's word choices. Length changes the odds as well: at a target false positive rate of 1%, OpenAI says its detector found watermarks in about 80% of 200-token passages against about 95% of 400-token passages for content such as psychology, and that detection was "substantially lower" for mathematics, "where there is less flexibility in word choice".

Every one of those figures is OpenAI's own, measured by OpenAI on its own models, and nobody outside the company can reproduce them yet. The detector is not public: applications opened on October 5, and OpenAI says access "will initially be limited to approved researchers and expert organizations", granted case by case "in accordance with the Code of Practice", citing the risk of missed watermarks and false positives.

The API gets an opt-in and the EU gets the default

From October 5, API customers anywhere in the world can opt in to watermarked text for "select models", and OpenAI says watermarking "will remain off by default in the API". The ChatGPT and Codex rollout works the other way round and covers less ground, reaching eligible users "across all plans in the EU only", with the company stating plainly that it is "not making text watermarking a global default at launch".

Nothing has shipped yet. The EU rollout is "over the coming weeks" with no date attached, and the models eligible for the API opt-in are not named anywhere.

OpenAI's own benchmark run shows no quality cost

OpenAI published eight paired benchmark results for Astra at maximum reasoning, watermarked against unwatermarked, and the gaps run in both directions. GPQA Diamond slipped from 94.44% to 93.94% and DeepSWE v1.1 from 72.80% to 71.68%, while Terminal-Bench Science 0.1 climbed from 56.90% to 60.00% and the Artificial Analysis Intelligence Index from 49.57 to 49.76 points.

OpenAI reads that spread as evidence it does "not see meaningful performance differences with and without watermarking". It also says textGrain "matched or exceeded" the other approaches it tested, among them SynthID for text, the lineage behind Google DeepMind's watermark for AI-designed proteins, while warning that "strong performance under ideal conditions does not guarantee reliable detection in everyday use".

Five things OpenAI says a watermark cannot tell you

By the company's own listing, a detection result "does not measure human contribution", "does not establish ownership or responsibility", "does not identify the user" and "does not verify accuracy". The fifth cuts against the exercise itself: "the absence of a detected watermark does not prove human authorship", because text may be too short, edited or translated, or may come from an unsupported model.

That caveat lands awkwardly beside the 17% figure, since an editor running a passage through a thesaurus arrives at the same result as a writer who never used ChatGPT at all. The maths behind textGrain sits in a technical report OpenAI published alongside the post, which the company says "will be updated with additional details in the coming weeks", and OpenAI plans to release the technology in open source.

Image and audio checking is unaffected and stays open to anyone, through OpenAI's own verify tool and the Content Provenance API. Text is the part being held back, and Codex, which OpenAI has been pushing into automated repository work, is in the first group to carry it. All figures here were read from OpenAI's post at 22:38 UTC on October 6.

Leave a Comment

Your email address will not be published. Required fields are marked *