Advance Alert Monitor, a sophisticated system that analyzes electronic data about hospital patients, identifies those at risk of deteriorating, and issues an alert to a centralized team of specially trained nurses, has resulted in a lower mortality rate, Kaiser Permanente researchers found.

The evaluation of the Advance Alert Monitor, or AAM, used in all 21 Kaiser Permanente Northern California hospitals, was published in the New England Journal of Medicine.

The study describes the results of a staggered deployment to Kaiser Permanente hospitals in Northern California between August 2016 and February 2019. The authors compared the outcomes for 15,487 patients who reached the alert threshold and 28,462 comparison patients who would have triggered an alert if the system had been active. The analysis found a 16% lower mortality rate among patients in the intervention cohort.

Gabriel Escobar, MD

Gabriel Escobar, MD

“Along with saving lives, the Advance Alert Monitor has demonstrated that it is possible to integrate predictive models into day-to-day operations in our medical centers,” said lead author Gabriel Escobar, MD, a research scientist with the Kaiser Permanente Division of Research and regional director for Kaiser Permanente Northern California hospital operations research.

“Predictive analytics and machine learning are unlocking new frontiers in the use of complex patient data to improve our care in real time,” said coauthor Vincent Liu, MD, MS, who is a practicing intensivist for TPMG and research scientist with the Division of Research, and regional director for Kaiser Permanente Northern California hospital advanced analytics. “They augment our clinicians’ practice by finding signals hidden within the electronic health record.” AAM predicts the probability that hospitalized patients are likely to decline, require transfer to the intensive care unit or emergency resuscitation, and benefit from interventions. Early warnings could be helpful for patients at risk of deterioration where early intervention may improve outcomes.

photo of Vincent Liu, MD, MS

Vincent Liu, MD, MS

The predictive model uses algorithms created from machine learning and data from more than 1.5 million patients. It employs severity-of-illness and longitudinal comorbidity scores, vital signs and vital sign trends, neurological status checks, and laboratory tests.

The system was tested in 2013 and rolled out to all 21 Kaiser Permanente Northern California hospitals between 2016 and 2019. This study, which compared patients with and without AAM in place, found the system was associated with better outcomes within 30 days of an alert. The alert system scans hospitals’ electronic health records hourly. If a patient’s score is above threshold, indicating significant risk of decline over the next 12 hours, an alert is issued. This alert is initially reviewed by a regional team of specially trained registered nurses that evaluates the alert using information from the patient’s medical record to determine if on-site intervention is needed. The nurses contact a rapid response team on that hospital unit, which performs a structured assessment and then works with the patient’s physician to determine further action.

Patients in the intervention cohort had lower ICU admission rates (17.7% versus 20.9%), shorter hospital length of stay (6.7 days versus 7.5 days), and lower mortality within 30 days of an alert (15.8% versus 20.4%). Patients who had an AAM alert were also less likely to die without a palliative care referral. The improved outcomes resulted not just from electronic tools, a recent report on AAM’s implementation concluded, but also from system integration, workflow development, and close collaboration among physicians, nurses, and other caregivers.

Alerts are nothing new to hospitals and in fact have proliferated to the point of causing “alert fatigue” for nurses, but the AAM system is different from other alert systems in several ways, Dr. Escobar said. It has a powerful analytical engine taking into account many patient status factors. It is automated, so it does not require manual risk calculation by hospital staff. And, importantly, the alerts are curated by trained nurses off-site, so bedside caregivers do not get unnecessary interruptions.

“The Advance Alert Monitor program is a wonderful example of how we combine high-tech and high-touch in caring for hospitalized patients,” said Stephen Parodi, MD, national infectious disease lead for Kaiser Permanente. “This study’s findings support an intervention employing both cutting-edge data analysis and the judgment of our top-notch professional nursing staff to identify patients who need immediate attention.”

The study was funded by the Sidney Garfield Memorial Fund, the Gordon and Betty Moore Foundation, the Agency for Healthcare Research and Quality, The Permanente Medical Group, and Kaiser Foundation Hospitals.

Co-authors were Patricia Kipnis, PhD; Alejandro Schuler, PhD; Brian Lawson, PhD; and John D. Greene, MA, all of the Division of Research.

This story originally appeared in Division of Research Spotlight