Subscribe to RSS
Application of a Machine Learning–Based Decision Support Tool to Improve an Injury Surveillance System WorkflowFunding This work was supported by an Australian Research Council Discovery Grant (grant no.: DP170103136).
Background Emergency department (ED)-based injury surveillance systems across many countries face resourcing challenges related to manual validation and coding of data.
Objective This study describes the evaluation of a machine learning (ML)-based decision support tool (DST) to assist injury surveillance departments in the validation, coding, and use of their data, comparing outcomes in coding time, and accuracy pre- and postimplementations.
Methods Manually coded injury surveillance data have been used to develop, train, and iteratively refine a ML-based classifier to enable semiautomated coding of injury narrative data. This paper describes a trial implementation of the ML-based DST in the Queensland Injury Surveillance Unit (QISU) workflow using a major pediatric hospital's ED data comparing outcomes in coding time and pre- and postimplementation accuracies.
Results The study found a 10% reduction in manual coding time after the DST was introduced. The Kappa statistics analysis in both DST-assisted and -unassisted data shows increase in accuracy across three data fields, that is, injury intent (85.4% unassisted vs. 94.5% assisted), external cause (88.8% unassisted vs. 91.8% assisted), and injury factor (89.3% unassisted vs. 92.9% assisted). The classifier was also used to produce a timely report monitoring injury patterns during the novel coronavirus disease 2019 (COVID-19) pandemic. Hence, it has the potential for near real-time surveillance of emerging hazards to inform public health responses.
Conclusion The integration of the DST into the injury surveillance workflow shows benefits as it facilitates timely reporting and acts as a DST in the manual coding process.
Protection of Human and Animal Subjects
Human and/or animal subjects were not included in the project. In addition, the study analyzed nonidentifiable data; therefore, consent from patients was not required.
Received: 31 January 2022
Accepted: 26 May 2022
Accepted Manuscript online:
29 May 2022
Article published online:
13 July 2022
© 2022. Thieme. All rights reserved.
Georg Thieme Verlag KG
Rüdigerstraße 14, 70469 Stuttgart, Germany
- 1 Chen L, Vallmuur K, Nayak R. Injury narrative text classification using factorization model. BMC Med Inform Decis Mak 2015; 15 (Suppl. 01) S5-S5
- 2 Nanda G, Vallmuur K, Lehto M. Improving autocoding performance of rare categories in injury classification: is more training data or filtering the solution?. Accid Anal Prev 2018; 110: 115-127
- 3 Nanda G, Vallmuur K, Lehto M. Semi-automated text mining strategies for identifying rare causes of injuries from emergency room triage data. IISE Trans Healthc Syst Eng 2019; 9 (02) 157-171
- 4 Nanda G, Vallmuur K, Lehto M. Intelligent human-machine approaches for assigning groups of injury codes to accident narratives. Saf Sci 2020; 125: 104585
- 5 Vallmuur K. Machine learning approaches to analysing textual injury surveillance data: A systematic review. Accid Anal Prev 2015; 79: 41-49
- 6 Vallmuur K, Marucci-Wellman HR, Taylor JA, Lehto M, Corns HL, Smith GS. Harnessing information from injury narratives in the ‘big data’ era: understanding and applying machine learning for injury surveillance. Inj Prev 2016; 22 (Suppl. 01) i34-i42
- 7 National Injury Surveillance Unit. National Data Standards for Injury Surveillance Ver 2.1. Available at: https://nla.gov.au/nla.cat-vn730536
- 8 Mitchell TM. Machine Learning. 1st ed.. Ithaca, NY: McGraw-Hill, Inc.; 1997
- 9 Chae YM, Kim HS, Tark KC, Park HJ, Ho SH. Analysis of healthcare quality indicator using data mining and decision support system. Expert Syst Appl 2003; 24 (02) 167-172
- 10 Kaushal R, Shojania KG, Bates DW. Effects of computerized physician order entry and clinical decision support systems on medication safety: A systematic review. Arch Intern Med 2003; 163 (12) 1409-1416
- 11 Kong G, Xu D, Yang J, Wang T, Jiang B. Evidential reasoning rule-based decision support system for predicting ICU admission and in-hospital death of trauma. IEEE Trans Syst Man Cybern Syst 2020; 51 (11) 7131-7142
- 12 Romero-Brufau S, Wyatt KD, Boyum P, Mickelson M, Moore M, Cognetta-Rieke C. Implementation of artificial intelligence-based clinical decision support to reduce hospital readmissions at a regional hospital. Appl Clin Inform 2020; 11 (04) 570-577
- 13 Shah M, Shu D, Prasath VBS, Ni Y, Schapiro AH, Dufendach KR. Machine learning for detection of correct peripherally inserted central catheter tip position from radiology reports in infants. Appl Clin Inform 2021; 12 (04) 856-863
- 14 Soldaini L, Cohan A, Yates A, Goharian N, Frieder O. Retrieving Medical Literature for Clinical Decision Support. Paper presented at: European Conference on Information Retrieval, Advances in Information Retrieval; 2015, 2015;538–549
- 15 Teng AK, Wilcox AB. A review of predictive analytics solutions for sepsis patients. Appl Clin Inform 2020; 11 (03) 387-398
- 16 Goh YM, Ubeynarayana CU. Construction accident narrative classification: An evaluation of text mining techniques. Accid Anal Prev 2017; 108: 122-130
- 17 Goldberg DM. Characterizing accident narratives with word embeddings: Improving accuracy, richness, and generalizability. J Safety Res 2022; 80: 441-455
- 18 Nanda G, Grattan KM, Chu MT, Davis LK, Lehto MR. Bayesian decision support for coding occupational injury data. J Safety Res 2016; 57: 71-82
- 19 Zhong B, Pan X, Love PED, Ding L, Fang W. Deep learning and network analysis: Classifying and visualizing accident narratives in construction. Autom Construct 2020; 113: 103089
- 20 Bussone A, Stumpf S, O'Sullivan D. The Role of Explanations on Trust and Reliance in Clinical Decision Support Systems. 2015; 160-169 DOI: 10.1109/ICHI.2015.26.
- 21 Lehto MR, Sorock GS. Machine learning of motor vehicle accident categories from narrative data. Methods Inf Med 1996; 35 (4-5): 309-316
- 22 Zhu W, Lehto MR. Decision support for indexing and retrieval of information in hypertext systems. Int J Hum Comput Interact 1999; 11 (04) 349-371
- 23 Leman S, Lehto MR. Interactive decision support system to predict print quality. Ergonomics 2003; 46 (1–3): 52-67
- 24 Choe P, Lehto MR, Shin GC, Choi KY. Semiautomated identification and classification of customer complaints. Hum Factors Ergon Manuf Serv Ind 2013; 23 (02) 149-162
- 25 McKenzie K, Scott DA, Waller GS, Campbell M. Reliability of routinely collected hospital data for child maltreatment surveillance. BMC Public Health 2011; 11 (01) 8-8
- 26 Liberati EG, Ruggiero F, Galuppo L. et al. What hinders the uptake of computerized decision support systems in hospitals? A qualitative study and framework for implementation. Implement Sci 2017; 12 (01) 113-113
- 27 May CR, Mair F, Finch T. et al. Development of a theory of implementation and integration: normalization process theory. Implement Sci 2009; 4 (01) 29-29
- 28 Mishuris RG, Palmisano J, McCullagh L. et al. Using normalisation process theory to understand workflow implications of decision support implementation across diverse primary care settings. BMJ Health Care Inform 2019; 26 (01) e100088