Appl Clin Inform 2021; 12(02): 372-382
DOI: 10.1055/s-0041-1726422
Research Article

Effect of a Real-Time Risk Score on 30-day Readmission Reduction in Singapore

Christine Xia Wu
1  Quality, Innovation and Improvement, Ng Teng Fong General Hospital, Singapore
,
Ernest Suresh
2  Department of Medicine, Ng Teng Fong General Hospital, Singapore
,
Francis Wei Loong Phng
1  Quality, Innovation and Improvement, Ng Teng Fong General Hospital, Singapore
,
Kai Pik Tai
1  Quality, Innovation and Improvement, Ng Teng Fong General Hospital, Singapore
,
Janthorn Pakdeethai
2  Department of Medicine, Ng Teng Fong General Hospital, Singapore
,
Jared Louis Andre D'Souza
2  Department of Medicine, Ng Teng Fong General Hospital, Singapore
,
Woan Shin Tan
3  Health Services and Outcomes Research, National Healthcare Group, Singapore
,
Phillip Phan
4  Department of Medicine, Johns Hopkins University, Baltimore, Maryland, United States
5  Department of Medicine, National University of Singapore, Singapore
,
Kelvin Sin Min Lew
1  Quality, Innovation and Improvement, Ng Teng Fong General Hospital, Singapore
,
Gamaliel Yu-Heng Tan
6  Group Medical Informatics Office, National University Health System, Singapore
,
Gerald Seng Wee Chua
2  Department of Medicine, Ng Teng Fong General Hospital, Singapore
,
Chi Hong Hwang
1  Quality, Innovation and Improvement, Ng Teng Fong General Hospital, Singapore
› Author Affiliations

Abstract

Objective To develop a risk score for the real-time prediction of readmissions for patients using patient specific information captured in electronic medical records (EMR) in Singapore to enable the prospective identification of high-risk patients for enrolment in timely interventions.

Methods Machine-learning models were built to estimate the probability of a patient being readmitted within 30 days of discharge. EMR of 25,472 patients discharged from the medicine department at Ng Teng Fong General Hospital between January 2016 and December 2016 were extracted retrospectively for training and internal validation of the models. We developed and implemented a real-time 30-day readmission risk score generation in the EMR system, which enabled the flagging of high-risk patients to care providers in the hospital. Based on the daily high-risk patient list, the various interfaces and flow sheets in the EMR were configured according to the information needs of the various stakeholders such as the inpatient medical, nursing, case management, emergency department, and postdischarge care teams.

Results Overall, the machine-learning models achieved good performance with area under the receiver operating characteristic ranging from 0.77 to 0.81. The models were used to proactively identify and attend to patients who are at risk of readmission before an actual readmission occurs. This approach successfully reduced the 30-day readmission rate for patients admitted to the medicine department from 11.7% in 2017 to 10.1% in 2019 (p < 0.01) after risk adjustment.

Conclusion Machine-learning models can be deployed in the EMR system to provide real-time forecasts for a more comprehensive outlook in the aspects of decision-making and care provision.

Author Contributions

C.X.W. provided the study concept and design and contributed to the critical review of the statistical analysis and manuscript revision. E.S. participated in the conception and design of the interventions and contributed to the manuscript revision. F.W.L.P. constructed the write-up of the manuscript and participated in the statistical analysis. K.P.T. built the risk score model and performed the statistical analysis. J.P. participated in constructing the intervention write-up and reviewed the manuscript for important intellectual content. J.L.A.D.S. contributed in building the risk score into the NTFGH Epic EMR system. W.S.T. provided advice to the team on the initial model development and reviewed the manuscript for important intellectual content. P.P. reviewed the manuscript for important intellectual content. K.S.M.L. provided inputs on the intervention write-up. G.Y.H.T. oversaw the implementation of the risk score into the hospital's EMR system. G.S.W.C. oversaw the conception and design of the interventions and provided advice on parameters with clinical significance. C.H.H. contributed to the critical review of the study design, statistical analysis, and reviewed the manuscript for important intellectual content. All the authors reviewed and approved the final manuscript.


Note

This study received ethical approval from the National Healthcare Group Domain Specific Review Board (NHG DSRB 2021/00284).


Supplementary Material



Publication History

Received: 29 October 2020

Accepted: 12 May 2021

Publication Date:
19 May 2021 (online)

© 2021. Thieme. All rights reserved.

Georg Thieme Verlag KG
Rüdigerstraße 14, 70469 Stuttgart, Germany