Microsoft's Diagnostic AI Claims an 83% Accuracy Rate — With Some Caveats
Microsoft published a study in the journal Science this week claiming its AI system, the Diagnostic Orchestrator, can diagnose medical conditions more accurately than human doctors, according to The Verge.
The system works like a panel of specialists. Multiple AI agents (software programs that can reason through a problem and take actions) examine symptoms and medical histories, decide which tests to order, and arrive at a diagnosis together. It is powered by OpenAI's GPT-5.
The numbers in the study are striking. On medical cases drawn from the New England Journal of Medicine, the system hit 83% diagnostic accuracy. Doctors using reference materials scored 36%; those working without scored 22%. On emergency department data from a Boston hospital, the AI scored 80% compared to an average of 47% for physicians.
Those are Microsoft's own numbers, running on Microsoft's own benchmark — called SDBench — which the company designed for this study. No independent research group has verified the results yet, and the benchmark does not include newer frontier AI models for comparison.
More importantly, every test case was historical. The system has never been run in a live clinical environment. Microsoft AI CEO Mustafa Suleyman told The Verge it "needs clinical validation" before wider deployment, and the company plans to begin hospital trials with Boston's Beth Israel Deaconess Medical Center.
What this is: a genuinely interesting research result, peer-reviewed and published in a serious journal. What it is not: a product you or anyone in your life will encounter soon. Watch for independent replication of the benchmark results — that will tell you far more than this announcement can.
Sources

