The best-documented Epic AI failure is the Epic Sepsis Model (ESM), a tool designed to flag patients at risk of sepsis. A large evaluation summarized by the NCBI Bookshelf found weak discrimination, missed many patients, and raised concerns about alert burden. A separate study at five University of Colorado Health hospitals reported better results in its own setting. Together, the findings show why a model’s performance must be tested against local patients and workflows—not inferred from its deployment footprint.
What went wrong with the Epic Sepsis Model?
The NCBI Bookshelf’s summary of a large evaluation reports that the ESM had been implemented across hundreds of U.S. hospitals without adequate evaluation before widespread use. The summarized study included 27,697 patients and 38,455 hospitalizations; sepsis occurred in 7% of hospitalizations. The reported area under the receiver operating characteristic curve (AUC) was 0.63 (95% confidence interval, 0.62–0.64), indicating limited ability to distinguish patients with sepsis from those without it in that evaluation. NCBI Bookshelf
The same summary says the model identified only 183 of 2,552 patients with sepsis who did not receive timely antibiotics, and failed to identify 1,709 sepsis patients (67%). It generated alerts for 6,971 hospitalizations (18%). These are figures reported in the Bookshelf’s account of that evaluation, not newly measured results or a description of every ESM version.
The University of Melbourne’s case summary separately reports that 86% of the alerts it discusses were false alarms. That figure has a different source and framing; it should not be combined with the NCBI summary’s alert count as if the two measures shared a denominator. University of Melbourne
Quick wins for a faster PC:
Fix the driver behind crashes, sound loss and screen glitchesFind Drivers →Repair Windows errors before they cause bigger problemsFix Now →Scan for outdated or missing drivers - takes under a minuteDriver Scan →Why did another study report better performance?
A 2019 retrospective study across five University of Colorado Health hospitals found more favorable results for the ESM than the system’s existing Early Warning Score (EWS) program, at the tested ESM score threshold of 5. The authors described ESM performance as moderately accurate in their setting. Read the study
| Measure at the tested threshold | Epic Sepsis Model | Early Warning Score |
|---|---|---|
| AUC | 0.73 | 0.62 |
| Positive predictive value | 0.44 | 0.33 |
| Recall | 0.66 | 0.61 |
Those results are specific to a regional, retrospective evaluation and its comparator. They do not cancel out the larger evaluation or establish that the ESM works equally well elsewhere. The studies differ in evaluation population and hospital setting, study period and potentially the deployed version, outcome definition and timing, threshold, and comparator. Metrics can only be compared responsibly when those details are understood.
What the ESM case teaches health systems
Deployment is not proof of effectiveness
A model can spread widely without evidence that it performs reliably for every intended patient population. The ESM’s reported deployment across hundreds of hospitals alongside concerns about pre-deployment evaluation illustrates why adoption alone is not a clinical validation result.
Track missed cases and alert burden together
Evaluating only how many alerts a model generates—or only its ability to identify cases—gives an incomplete safety picture. The ESM findings make both questions essential: how many patients at risk are missed, and how many alerts clinicians must review. Alert volume also affects whether clinicians can respond consistently to useful warnings.
Free tools Windows power users keep installed
One-click scans. No signup required.
Test locally before alerts affect care
A prospective silent trial runs a model on local data while keeping its predictions from influencing patient care. That lets a health system assess performance before clinicians must act on alerts. The University of Melbourne case summary says such a trial could have surfaced problems before broad deployment.
Recheck performance when the tool or workflow changes
Clinical software, patient populations, and care workflows change. A model that performed acceptably in one evaluation may not retain that performance after changes in version or setting. Health systems need ongoing review of outcomes, missed cases, alert burden, and clinician experience rather than treating validation as a one-time exercise.
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.What is known about other Epic AI tools?
The ESM findings should not be generalized to every Epic AI feature. Separate reporting from Becker’s Hospital Review in October 2026 described health systems holding back or piloting other Epic AI capabilities while assessing accuracy and clinician experience. Children’s Healthcare of Atlanta CIO Jeremy Meller said one inpatient insights capability had “too many inaccuracies across diagnosis and patient locations, and produced excessively long narratives.” The system planned to reevaluate that capability. This is a separate tool and decision, not evidence about the ESM. Becker’s Hospital Review
Epic’s position should also be distinguished from independent evaluation. In STAT’s 2022 account, Epic said: “Tens of thousands of clinicians have access to the sepsis model and transparency into how it works.” That is Epic’s corporate statement as reported by STAT, not an independent finding about model accuracy. STAT
The sources cited here do not establish independent external validation results for a later version of Epic’s sepsis model. The historical evaluation therefore cannot establish how a subsequent version performs today; that requires version-specific evidence.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
