Taking Away GPT-5 Leaves Users Worse at Solving Problems Solo

Relying on artificial intelligence to solve tricky problems might feel efficient, but taking the assistant away appears to leave people worse off than if they had worked alone from the start.

In a paper presented at the Conference on Language Modeling (COLM 2026), researchers from Carnegie Mellon University, UC Berkeley, MIT, the University of Oxford, and UCLA tested how people performed after access to an automated assistant was abruptly cut off. The trials involved 1,222 participants across three experiments focusing on fraction arithmetic and SAT reading comprehension.

During the sessions, one group had access to GPT-5 via a sidebar that could deliver direct answers, while a control group worked entirely unassisted. Once the assistant was removed, participants who had relied on it struggled significantly more on subsequent questions than the control group.

In the first arithmetic test, the former AI users averaged 57% correct answers compared to 73% in the control group, while skipping twice as many problems (20% versus 11%). A second math trial showed a smaller gap—71% accuracy versus 77%—with negligible differences in skip rates. However, the reading comprehension test mirrored the initial drop: unassisted controls achieved 89% accuracy with only a 1% skip rate, whereas the cut-off AI group scored 76% and skipped 8% of the questions.

The authors suggest the problem lies in the erosion of "productive struggle"—the mental effort required to work through friction when learning. When users leaned on instant answers, their persistence dropped sharply once independent effort was required. Notably, the decline was far more pronounced among those who asked the model for finished solutions rather than step-by-step hints.

Source: habr

Study: arXiv