July 26, 2021
Health systems could use data from routine clinical care to identify the onset of upcoming COVID-19 surges as many as 6 weeks before they occur, Kaiser Permanente researchers found in a study published in the journal BMJ Open.
Vincent Liu, MD
Even though the impact on hospitals of COVID-19 cases has subsided in Northern California with successful vaccination campaigns, there are concerns that the virus and hospitalizations may surge again locally as they are elsewhere in the world, said lead author Vincent Liu, MD, MS, a research scientist with the Kaiser Permanente Division of Research.
“Over the course of 2020, COVID-19 surprised us at nearly every turn, making longer-term predictions of its impact on our patients, health system, and communities extremely challenging,” said Dr. Liu, who is also a practicing intensivist with The Permanente Medical Group and regional director of hospital advanced analytics for Kaiser Permanente in Northern California.
“At the same time, shorter-term predictions — looking only 1 to 3 weeks out — left little time to respond adequately,” he said. “By examining hundreds of millions of data points across our entire health system, we have now identified key leading indicators that forecast impending COVID-19 surges as much as 6 weeks before they hit our medical centers.”
The team tested dozens of different data elements from the electronic health record system and data used by Kaiser Permanente Northern California to find any elements that might be indicative of an impending surge in COVID-19 admissions. To determine what data points might be leading indicators, they looked at seasonal surges of influenza as far back as 2015.
Ultimately, the investigators chose 10 indicators and determined how many weeks of lead time they could provide health system and community officials. While individual indicators could forecast an upcoming surge in the next 1 to 3 weeks, the combined CHOTS — the “COVID-19 HotSpotting Score” — significantly increased the lead time to as far as 6 weeks prior to a surge.
CHOTS includes 4 major variables — cough and cold calls, relevant subject headers from patient emails, positive COVID-19 test rates, and current COVID-19 hospital census — along with 6 minor variables, including COVID-19-related calls to the call center, both routine and urgent clinic visits for respiratory infections, clinic and urgent clinic COVID-19 visits, and respiratory virus tests ordered.
Public health officials and individual health systems could use the information to boost local testing, expand contact tracing, adjust policy measures, and prepare expanded hospital space and supplies for anticipated patients. The authors found early indicators of waning COVID-19 activity were also valuable because they could help identify when relief from the surge of COVID-19 was on the horizon.
Stephen Parodi, MD
“The development of CHOTS is an inspiring example of how Kaiser Permanente’s integrated care delivery system and robust research capabilities can come together to provide the greater community with vital public health tools,” said co-author Stephen Parodi, MD, national infectious disease leader for Kaiser Permanente. “What is even more impressive is that this work was carried out while our medical centers were caring for large numbers of COVID-19 patients. Our researchers worked alongside operational leaders to better prepare our management of future surges.”
Developers chose simplicity
Because of the urgency to develop an early warning system for the COVID-19 hospital surge in May 2020, the authors chose to keep CHOTS relatively simple. They wanted to ensure the tool was available in real time by June 2020 and could be duplicated by other health care organizations. The authors are making the CHOTS methodology freely available for others to use.
“We use machine learning and artificial intelligence every day in our research group to develop predictive models to improve patient care.” said co-author Patricia Kipnis, PhD, principal statistician at the Division of Research. “We applied these tools when we were developing CHOTS but didn’t find that they improved the tool’s value. Our research group focuses on pairing the right algorithm with the right use case and, in this case, a simpler tool showed excellent performance and could be readily implemented and shared.”
CHOTS went live in June 2020 and the team evaluated it against actual COVID-19 hospital activity through December 31, 2020, throughout Northern California, and for each of 20 Kaiser Permanente hospitals in the region. The correlation of the regional CHOTS with hospital census was very strong, peaking with a 28- to 35-day lead time, but with continued correlation when tested out to 6 weeks. Because the impact of COVID-19 varied tremendously within individual medical centers over time, the team also evaluated CHOTS at each hospital and found strong, leading correlation at most sites.
The authors noted a limitation to CHOTS: It relies on comprehensive data from a robust electronic health record system, which may not be available to health systems in some locations and those without regional call centers. A simplified tool, without these indicators, also showed strong, but modestly lower, correlation.
The study was funded by The Permanente Medical Group, Kaiser Foundation Hospitals, and the National Institute of General Medical Sciences.
Co-authors also included Gabriel Escobar, MD, Laura Myers, MD, Lawrence Gerstley, PhD, and Khanh K. Thai, MS, of the Division of Research; and Yi-Fen “Irene” Chen, MD, Nancy Goler, MD, and Jessica Galin, MPH, of The Permanente Medical Group.
This article originally appeared in Division of Research Spotlight
