OpenAI Faces €15M Fines: Hidden 'textGrain' Watermarks Are Now Mandatory

OpenAI is deploying a new technology called textGrain to embed invisible watermarks into text generated by ChatGPT and Codex. This system subtly tweaks word statistics, allowing a detector to sniff out AI origins without human eyes ever noticing a difference. While this currently targets users within the EU, it follows the EU AI Act, which mandates machine-readable labeling for synthetic content to avoid fines reaching up to 15 million euros.

The detection mechanism relies on statistical analysis of token sequences, which works surprisingly well for long-form prose but struggles with specialized content like mathematics. However, the system is fragile: rephrasing or swapping just a quarter of the words in a piece of text drops detection accuracy to as low as 7-15%. OpenAI is rolling this out as a pilot, initially providing access only to select experts before opening it to wider API usage.

Despite OpenAI claiming that these watermarks don't impact model performance—and even jokingly suggesting that watermarked models might perform slightly better—it remains clear that this is a regulatory band-aid. As long as a simple synonym swap can bypass these 'security' measures, the cat-and-mouse game between AI-generated slop and detection tools will remain a stalemate. Whether this serves as a genuine deterrent or just another performative compliance checkbox in the era of automated content is a debate that will define the next cycle of AI governance.

Source: OpenAI