Microsoft's AI 'diagnostic orchestrator' beats generalist doctors on hard cases in a Science study — on a text-only benchmark

A Microsoft team has published in Science a study of the Diagnostic Orchestrator (DxO), an AI system that works through a medical case step by step — deciding which questions to ask and which tests to order — instead of receiving a complete case summary at once.

The results

• On 304 New England Journal of Medicine clinicopathological cases, DxO reached the correct diagnosis in 83.2% of cases. Generalist physicians scored 22.3% without reference tools and 36.1% with them.
• On 325 emergency-department admissions from the MIMIC-IV-ED database, DxO was correct in 80.3% of cases versus 46.8% for physicians, Medical Economics reported.
• DxO runs on a commercial large language model (GPT-5 in the main results) and simulates a panel of virtual doctors who challenge each other’s hypotheses and weigh the cost of each test.
• Without budget limits, DxO’s average testing costs were higher than the doctors’ ($5,431 vs $4,725 per case on NEJM cases). With budget limits it cut costs, at some loss of accuracy.

Limitations

The authors acknowledge that the comparison was with individual generalists, not specialists or teams; the benchmark was text-only (no imaging or pathology slides); every case had a disease, so healthy and benign visits were excluded; and the study measured only diagnosis, not treatment, safety or patient outcomes. All authors are Microsoft employees holding company equity, and the study had no outside funding. It does not show that the system is safe to use with patients.