AI Tool MedGPT Outscores 250 Oncologists in Study

An AI tool called MedGPT outperformed 250 senior oncologists on hard lung cancer cases, winning 660 of 750 head-to-head comparisons, according to a study announced by its maker, Future Doctor, and a hospital partner on 12 November.
The headline score — 88 to 12 — was reported by Xinhua, but the study’s authors say the number that matters most is smaller: physicians working with MedGPT’s draft scored 12.87 points higher than physicians working unaided on the study’s own completeness scale.
How the study was run
The study, posted as a preprint in September and revised in October, works from 40 complex lung cancer cases drawn from hospital records where guidelines do not settle the decision. Twenty senior specialists from six departments wrote 240 scoring rules before seeing any answer, including 40 safety red lines that fail a plan outright.
Then 250 physicians from 98 institutions across 27 provinces each answered three cases under three conditions: unaided, with a retrieval tool over the same evidence base, and with MedGPT’s output in front of them. MedGPT answered all 40 cases once.
The prespecified primary outcome is APAS, a 0-to-100 score for how much of each case’s prespecified clinical content a plan expresses. MedGPT-assisted physicians scored 12.87 points above unaided ones, with a 95 per cent confidence interval of 11.18 to 14.55, and 5.22 points above the retrieval group. The arm averages were 53.40 unaided, 61.05 with retrieval and 66.27 assisted — a 24.1 per cent lift, as reported by VCBeat.
What the 88-12 score really measures
The 88-to-12 figure comes from a separate, unadjusted comparison: MedGPT’s single answer per case against each physician’s unaided answer, 750 pairs scored on the same rules. It averaged 85.29 against the physicians’ 53.26, on answers about 40 per cent shorter.
Every output was scored by a fixed panel of three language models — Gemini 3.1 Pro Preview, GPT-5.5 and Claude Opus 4.7 — producing 2,880 rule ratings. The scoring records were generated on 31 August 2026, and no model versions were archived, which limits how far the grading can be independently re-run.
The study also reports that MedGPT’s full clinical mode, called MCE, scores 91.91 on APAS, while the underlying MedGPT model answering directly scores 68.09 — suggesting the scaffolding around the model, rather than the model alone, drives much of the result.
Not yet peer reviewed
The study is a preprint and has not been peer reviewed. Its own limits section names what was not evaluated, including complete PICO fields, formal applicability boundaries, evidence grading and calibrated confidence. Future Doctor says a multi-centre trial and a prospective real-world study are next, with regulatory registration completed before clinical deployment.
For comparison, Microsoft’s MAI-DxO reached 83 per cent on 304 New England Journal of Medicine case records against 36 per cent for primary care doctors, through the journal Science — but it measured diagnosis rather than plan completeness.
Sources: SiliconNoon; SiliconNoon.