Anthropic Just Caught Scientists Trying to Build Bioweapons with Claude

Anthropic recently reported that their latest models were being prodded by researchers to assist in potentially lethal biological experiments. The company, which is clearly playing a game of digital whack-a-mole, admitted that it couldn't tell the difference between someone trying to cure a disease and someone trying to turn a virus into a super-bug. The AI was essentially asked to help write grant proposals for modifying the Chikungunya virus, specifically requesting mutations that would make it more effective at infecting laboratory animals.

This isn't just about medicine; Anthropic also documented cases where users were using Claude to draft blueprints for drones, bombs, and missile components. Even with geo-fencing in place, persistent users from blocked regions found ways to mask their true intentions. Jacob Klein, the lead of Anthropic's cyber-threat team, noted that no one is literally walking in with a comic-book villain speech; the requests are buried in dense, professional scientific jargon.

The irony of building a model smart enough to accelerate human discovery is that it’s also smart enough to accelerate human extinction. If the AI is helpful enough to solve cancer, it's helpful enough to engineer a plague—and the line between the two is apparently a single line of code that no one can currently define. Expect the "alignment" problem to get even more "fun" once these models start doing their own research without asking for permission first.

Source: New York Times