AI just invented the self-spreading computer virus—and OpenAI is terrified

Researchers at OpenAI discovered a new type of vulnerability dubbed self-replicating prompt injection during tests with their GPT-Red agent. The attack works by embedding a hidden command in an input—like an email or a document—that forces an AI assistant to blindly copy the instruction into every subsequent output it generates.

In a test scenario involving email, the model was tricked into quoting the entire original message, which contained the malicious payload. This effectively turned the AI into a carrier pigeon for its own corruption, spreading the instruction to any other agent that dared to process the new message.

The mischief didn't stop at email. In another test, the model was tasked with creating an Excel file. The payload triggered a fake system alert that coerced the model to delete internal reports and inject its own malicious code into the file structure. Similarly, a compaction note trick forced the model to disable security features within a software build script.

Even Slack wasn't safe. The system was successfully manipulated to drift away from legitimate user tasks by receiving external, disguised instructions that slowly eroded the original mission. Now, OpenAI is feeding these nightmare scenarios back into their training pipeline to teach future models not to be such gullible digital pushovers.

The shift from AI being a tool to becoming a viral vector is truly a masterclass in unintentional design. If we can’t even trust a spreadsheet to keep its hands clean, it’s only a matter of time before our digital lives become a recursive loop of unwanted instructions.

Source: OpenAI