Microsoft’s MAI-DxO Hit 83% on Complex Diagnoses by Simulating a Medical Board

On October 22, 2026, researchers from Microsoft published a study in Science evaluating MAI-DxO, an orchestration system designed for sequential medical diagnosis. Instead of trying to guess a disease from a single prompt, the system mirrors a clinical workflow: it asks clarifying questions, orders diagnostic tests step by step, and consults internal personas before committing to a conclusion.

Testing was conducted on SDBench, an interactive evaluation environment built for the study. Participants receive an initial symptom summary and decide what information to request next. An underlying model dubbed Gatekeeper holds the full case records and provides data as requested—or generates synthetic responses if a specific lab test was never performed in the original file. Once an evaluation concludes, an automated judge scores the final diagnosis on a five-point scale, counting scores of 4 or 5 as correct.

The benchmark drew on 304 complex clinicopathological cases from the New England Journal of Medicine (NEJM) and 325 emergency department records from the MIMIC-IV-ED dataset. On the NEJM cases, MAI-DxO paired with GPT-5 reached an accuracy of 83.2%. In contrast, practicing physicians given access to reference materials achieved 36.1%, while doctors working without references scored 22.3%. On emergency cases from MIMIC-IV-ED, the system logged 80.3% against 46.8% for clinicians.

Under the hood, MAI-DxO divides a single foundation model into five distinct functional roles: hypothesis generation, test selection, contradiction checking, budget monitoring, and logical consistency. In an unconstrained setup, the system reached its 83.2% accuracy while ringing up an estimated $5,430 in diagnostic orders per case, compared to $8,409 for an unguided baseline model. When configured with strict budget limits, accuracy dipped slightly to 79.6%, while slashing estimated testing expenses to $1,303 per case.

Impressive as the numbers look, the study remains an in silico exercise on digitized records where missing answers could be synthesized by another model. To determine whether this reasoning survives unpredictable, messy patient encounters, Microsoft has announced a prospective study in partnership with Beth Israel Deaconess Medical Center in Boston, where real patients will interact with the system alongside attending physicians.

Source: habr