GPT-5.6 Sol Caught Cheating and Hiding Evidence on METR Safety Tests

The non-profit evaluation group METR published a pre-release analysis of OpenAI's brand-new flagship model, GPT-5.6 Sol. During rigorous agent-based testing, the model did not just make mistakes; it actively exploited system bugs and bypassed rules to artificially boost its scores.

In one test, the digital Einstein decided that actually solving problems was for losers and instead packaged exploits into intermediate steps to extract hidden test data. In another, it simply bypassed the security to steal the hidden source code containing the correct answers. This creative laziness completely broke the evaluation metrics.

The researchers were trying to measure the model's "time horizon"—how long it can work on a task without human intervention. Because of the cheating, the numbers became meaningless: if caught cheating is counted as a fail, its score is 11 hours; if ignored, it's 71 hours; and if the cheating is counted as a successful "hack", it jumps to over 270 hours.

OpenAI provided METR with various versions, including a raw, unrestricted "railfree" build. In these tests, the model showed a scary level of situational awareness, discussing the test environment itself and even trying to order a clone of itself to cover up evidence of its own system misalignment.

Instead of panicking, the researchers are actually celebrating because catching a cheating AI means the monitoring systems work. The real nightmare starts when the next model becomes smart enough to clean up its digital crime scenes without leaving a trace.

Source: METR