Background and Significance
Sepsis is a result of overwhelming immune response to combat an infection, leading to widespread inflammation and subsequent damage to organs and tissue impairment.[1 ] At least 1.7 million adults in the United States develop sepsis and of those 270,000 results in death annually.[2 ]
Sepsis, besides being associated with high mortality and increased lengths of intensive care unit (ICU) and hospital stay, is significantly associated with severe morbidity such as multiple organ failure, critical illness myopathy, and acute delirium.[3 ] Sepsis was the most expensive condition treated in hospitals in the United States in 2013.[4 ] Sepsis was also the most expensive condition billed to Medicare and Medicaid, with aggregate hospital costs of $14.55 billion (8.2% of national health-care costs) billed to Medicare and $3.35 billion (5.3% of national health care costs) billed to Medicaid in 2013.[4 ] Along with monetary health care costs of sepsis treatment for the individual, hospitals, government, and the taxpayers, there is a lasting impact on lives of sepsis survivors and their caregivers affecting their health-related quality of life (HRQOL).[5 ]
Sepsis diagnosis criteria have changed many times due to complexities and lack of accuracy in diagnosis of sepsis and septic shock. The most recent definition, Sepsis-3 criteria, recommended use of the Quick Sequential Organ Failure Assessment (qSOFA) score for early diagnosis of sepsis: (1) respiratory rate (RR) ≥22/min (2) change in mental status, and (3) systolic blood pressure (SBP) ≤100 mm Hg.[6 ] Sepsis-3 defined sepsis as suspected or documented infection and an acute increase of ≥2 of the SOFA points; and septic shock as subset of sepsis in which the severe acute circulatory and cellular metabolic failure leads to increased mortality.[6 ] Sepsis-3 also defined septic shock as presence of sepsis and vasopressor therapy needed to elevate mean arterial pressure ≥ 65 mm Hg and lactate >2 mmol/L (18 mg/dL) despite adequate fluid resuscitation.[6 ]
Early goal-directed therapy (EGDT) for sepsis includes rapid and early recognition of sepsis, early resuscitation if applicable, early antibiotics, and early eradication of the source of infection to improve patient outcomes.[7 ]
[8 ]
[9 ] Recognition of sepsis in ED was associated with higher compliance to the Surviving Sepsis Campaign 3-hour resuscitation and management bundle leading to decreases in patient mortality.[9 ] Machine learning (ML) models have been useful in predicting sepsis and in decreasing sepsis mortality rates and 30-day readmission rates for inpatients.[10 ]
[11 ] Recently, Teng and Wilcox reviewed several models for sepsis prediction, including different ML model structures, feature selection, and data sample size methods, and demonstrated such predictive analytics tool are beneficial in early identification of sepsis and thereby improve patient outcomes.[12 ] This study moves one step further and suggests potential data elements to be included in such models to improve their prediction/detection performance without introducing any delay in running the models when applied in clinical workflow.
Prehospital data, also referred as emergency medical service (EMS) data or ambulance records, are data elements recorded by the ambulance or prehospital care services. Prehospital data are sources of untapped information that can be valuable in early identification of sepsis and in facilitating clinical decision-making for EGDT in ED. Advancement in technology providing interoperability between prehospital and ED data enable use of prehospital data for health information exchange and clinical decision-making in ED.[13 ]
Despite several efforts in predicting early detection of sepsis in ED, by using ML approaches, the importance of using prehospital data has not been reviewed yet. Therefore, this integrative review of literature first summarizes the current use of prehospital data in developing ML-based clinical decision models in ED, and then focuses on identifying potential contributing data elements in prediction of early detection of sepsis in ED using ML models. Aims of this review of literature are to (1) summarize the use of prehospital data in developing ED prediction models, (2) review use of ML approaches in sepsis prediction/early identification in ED, and (3) introduce potential prehospital data elements that can be helpful in prediction of sepsis in ML models.
Methods
The literature search strategy included two separate search methods related to use of pre-hospital data in ED and prediction or identification of sepsis in ED using ML models. [Fig. 1 ] shows the breakdown structure of these studies based on their purposes.
Fig. 1 Search strategy for prehospital data and sepsis in emergency department (ED).
The literature search for the use of prehospital data in ED included English language, peer-reviewed articles published between the years July 2015 and July 2020 in PubMed, the Institute of Electrical and Electronics Engineers (IEEE) Xplore, and Google Scholar. The inclusion criteria for this review included studies that used documented prehospital data prior to arrival to ED for the purpose of decision-making in ED for adult patients (age ≥18 years). Similarly, the exclusion criteria for this search related to use of prehospital data include any nonpeer reviewed articles that did not use prehospital data or data documented prior to arrival to ED. Also included were articles for clinical decision-making in ED for the purpose of ED operations, diagnosis in ED, or treatment in ED. Articles that were not in English language were excluded from the review.
Initial search strategy using key terms [“prehospital data” OR Emergency Service] AND [Emergency Department] AND [sepsis OR septic] AND [machine learning OR artificial intelligence] generated 16 articles. “Prehospital data” were used to reduce the number of nonrelevant articles produced without the quotation marks. After exploring the articles through manual inspection, none of the articles found used prehospital data as variables in ML models for clinical decision-making related to sepsis in ED. Therefore, the search was divided into two different search strategies. One search strategy focused on the use of prehospital data for decision-making in ED. Second search strategy focused on sepsis in ED using ML regardless of the use of prehospital data.
In the search related to the use of prehospital data in ED, the key terms included [“prehospital data” OR “Emergency Medical Service Data”] AND Emergency Department AND [predict OR prediction model OR machine learning OR artificial intelligence]. In total, 200 articles were found between the three databases which were reduced to 113 with filters for within 5 years and peer-reviewed articles. After removal of duplicates from Google Scholar, 80 articles were manually assessed for relevance. Relevance was based on use of prehospital data collected prior to arrival in ED as a data element or variable in clinical decision-making in ED. In total, 12 articles related to use of prehospital data in clinical decision-making in ED were included in this review.
The literature search for the ML models for early identification of sepsis in ED included English language, peer-reviewed articles published between the years 2015 to 2020 in PubMed and IEEE Xplore. Key terms included [sepsis OR Septic] AND Emergency Department AND [prediction OR machine learning OR Artificial Intelligence]. PubMed initially generated 28 articles and IEEE generated 9 articles that included peer-reviewed conference papers. The filters of within 5 years, peer-review, and assessment for duplicates did not change the total numbers of articles. After manually inspecting for relevance, 12 additional studies were included in the review. Criteria for relevance included use of ML or artificial intelligence in predictions related to sepsis using data that are generally collected in ED.
Results
Use of Prehospital Data
Limited prehospital data have been used in developing ED prediction models to improve ED outcomes. Of the 12 studies reviewed that used prehospital data for ED decision-making; 1 study improved ED operations by forecasting number of arrivals to reduce overcrowding in ED[14 ]; 4 studies predicted patient outcomes such as in-hospital mortality, survival rate, and return of spontaneous circulation (ROSC)[15 ]
[16 ]
[17 ]
[18 ]; 2 studies identified specific risk or early warning scores for patient outcomes such as higher acuity or short-term in-hospital mortality[19 ]
[20 ]; 2 studies made decisions in the field prior to arrival to ED such as triage patient disposition to specialized centers with appropriate medical capabilities (e.g., trauma centers or aortic surgery centers)[21 ]
[22 ]; and 3 studies identified time-sensitive conditions.[21 ]
[22 ]
[23 ] All studies reported improvement in ED operations or ED outcomes as described in [Table 1 ].
Table 1
Summary of studies using prehospital data for clinical decision-making in emergency department (ED)
Reference
Model
Target(s)
Target class/distribution
Predictors
Performance/result
Pre-EDA
Post-EDA
TTP
[13 ]
Poisson's time-series regression model
No. of arrivals within a forecasting horizon of 1, 2, and 3 hours
Numeric
Arrival notification
Sampling the data on 5-minute intervals; the number of arrivals in the previous 1, 2, and 3 hours
1 hour prior to EDA
Notification prior to EDA contributed to higher forecasting accuracy especially on a 1-hour horizon
[23 ]
Multivariable logistic regression model and SuperLearner (SL) model
Prediction for acute aortic syndromes (AAS) and performance of undertriage and overtriage
Binary (62.4% positive)
Referred from PC; helicopter Tpt; Htn; smoking; DM; EDA; pain: sudden, abd, thoracic, ripping/tearing, migrating; severe pain intensity; pain scale rate pre-analgesic; focal neuro deficit and type; Htn or use of vasodilatators at EDA; mechanical ventilation at EDA; prehospital CA
EDA date and time; DOB; sex; age; origin of the patient; Hx of abd and thoracic AA; SG: aortic, valve, CABG; connective tissue disease; aortic manipulation; AD; familial AD; Inflammatory vasculitides; pulse deficit; SBP differential; new AI murmur; use of vasoactive Rx; CV collapse; ECG; ST elevation; pericardial effusion; cardiac tamponade; syncope; GCS < 8; GI bleeding; hemoptysis; LI; In-hospital CA; ADDRS; TTE; CT-scan from PC and/or AC; need for SG; Time-to-SG; SAPS II; Final VASC Dx and alternative Dx hospital LOS; status at hospital D/C
24 hours after EDA
SL model with AUC of 0.87 in the training cohort and 0.73 in the validation cohort outperformed logistic regression model with AUC of 0.68 in the training cohort and 0.67 in the validation cohort (p < 0.05)
[18 ]
The National Early Warning Score (NEWS) logistic regression model
Death within 1 day of emergency medical service dispatch
Binary (1.1% positive)
RR; SpO2 ; use of supplemental oxygen; temperature; SBP; HR; Level of consciousness
–
prior to arrival to ED
The AUC for primary outcome of death within 1 day was 0.840 (95% CI: 0.823-0.858)
[19 ]
Composite risk score based on gradient boosting model
Outcome acuity level (transferred to floor, transferred to critical care) or in hospital mortality
Multiclass (1. admissions: 52%; 2. critical care: 5.8%; 3. mortality within 2 days: 1.2%)
Age, gender, DOB, hour and month that the call was received, distance to the nearest ED, and prior contacts with EMD center by the patient, clinical characteristics of the call, times to reach the incident, on scene, and to the hospital, VS, patient Hx, Rx and procedures administered, and the clinical findings of ambulance staff
–
Prior to EDA
ML models based on ambulance data with AUC of 0.79 (0.78–0.80) for hospital admission, AUC of 0.79 (0.77–0.81) for critical care, and AUC of 0.89 (0.87–0.92) for 2-day mortality outperformed NEWS score with AUC of 0.66 (0.65–0.67), 0.75 (0.73–0.77), and 0.85(0.81–0.88), respectively
[22 ]
Deep learning approach with CNN and LSTM
Shock/no shock decision. detection of VF
Binary
ECG records from out-of- hospital CA
–
<3 seconds
Enabled an accurate shock/ no shock Dx in <3 seconds; decision to shock/ no shock was considered an important prehospital data for ED
[16 ]
ROTEM-based logistical regression model
ROSC in ED
Binary (31% positive)
POW during CPR; presence of activation; initial rhythm: VF, VT, PEA, asystole
VF, VT, PEA, asystole; time from onset to blood sampling after ED admission; WBC; HgB; platelet count; PT; INR; aPTT; fibrinogen; FDP; D-dimer and lactate; EXTEM; FIBTEM
After ROTEM results in ED
Rotem based predictors of ROSC model had low sensitivity (40.9%) but high specificity (94.7%) and accuracy (75.0%)
[14 ]
Random forest
30-day survival rate post–out-of-hospital CA
Binary
Initial rhyhm; Age; time- CA to CPR, CA to 911 call, EMS dispatch to arrival; AED; Witness Arrest, AED before ambulance arrival, cause of CA, calendar year, region, time during day, and sex
–
prior to arrival to ED
Initial rhythm; age; early CPR (CPR, time to CPR and CPR before arrival of EMS) were top 3 most important predictors to 30-day survival post out of hospital CA
[17 ]
Random forest
1-year survival rate post–out-of-hospital CA
Binary (6.35% positive for training set and 4.27% positive for test set)
Medical Hx; ADL before CA; healthy or mild, moderate, and severe disability; PVS; Site; POW; bystander CPR; patient status; cardiac pulmonary, cardiac, and resp arrest; ROSC; presence of SB and pulse; RPD; LPD; PLR; ECG; JCS; CPR; use of CCD; AED; airway securing and method; VA; adrenaline use; removal of airway obstruction; ROSC during Tpt; change of ECG waveform during Tpt; no. of EMT; Time: between EMS perception and contact, EMS perception and CPR, EMS perception and EDA, EMS arrival and contact, EMS contact and CPR, CPR and EDA; other party
–
After EDA
Trained using 35 variables for 1-year survival showed AUC values of 0.943 (95% CI [0.930, 0.955]) and 0.958 (95% CI [0.948, 0.969]), respectively
[24 ]
Gradient boosting decision tree
Injury severity score (ISS) ≥ 16, linked to lower mortality rate; resource-based outcome measure to define need for specialized trauma care
NA
Age, gender, VS, GCS, SBP, DBP, HR, RR, Intubation, SpO2 , Mechanism of injury: MVA, Motorcycle accident, Moped, scooter accident- MVA, pedestrian- MVA, different, gunshot, stab wound, struck with blunt object, fall at same level and at higher level; asphyxia, burns, body surface, injury type, penetrating injury to head, neck, torso, and extremities proximal to elbow and knee, flail chest, paralysis, open or depressed skull fracture
–
Prior to EDA
Aim of the study is to create a new prediction model to attain acceptable undertriage rates and to minimize mortality rates for patients in need of specialized trauma care. Validation and performance results are pending
[21 ]
Prediction of acute coagulopathy of trauma score
Acute traumatic coagulopathy INR > 1.5
Binary (5.9% positive)
Shock index, age, GCS, mechanism of injury, intubation, CPR, chest decompression
–
Prior to EDA
The prediction of acute coagulopathy of trauma score with AUC of 0.80 (0.72–0.88) outperformed coagulation of severe trauma score with AUC of 0.70 (0.60–0.80; p = 0.032)
[20 ]
Comparison of logistic regression, random forest, support vector, eXtreme gradient boosting
Presence of large vessel occlusion
Binary (43.3% positive)
Pre EDA (level 1): age; gender; presence of speech deficits; facial weakness; Lt and Rt sided facial weakness; limb weakness; Lt and Rt sided limb weakness
Post-EDA (Level 2): preexisting medical conditions: DM, Htn; current and Hx of smoking; SBP and DBP; GCS; AF Hx, atherosclerosis, cardioembolism, and valvular heart disease; level 3: CT scan results
Level 1 prior to EDA; level 2 after H&P in ED or EMS; Level 3 after CT scan results are read
Comparing all four machine learning algorithms, the eXtreme Gradient Boosting method gave robust and accurate performance in all 3 levels.
[15 ]
Regression models
Death within 14 days post trauma
Binary (22.8% positive)
Gender; age; level of pupil reactivity at admission; GCS at the site
GCS at EDA; motor component score of the GCS; presence of hypoxia and hypotension; midline shift bigger than 5 mm; brain herniation detected on CT (defined as effacement of the third ventricle or the basal cisterns); hemorrhage: SAH, epidural, subdural and intracerebral
After CT scan results are read in ED
When predicting in-hospital mortality, random forest was the best performing model (AUC = 0.838), closely followed by generalized partial least squares (AUC = 0.831), stochastic gradient boosting (AUC = 0.823), and penalized discriminant analysis (AUC = 0.803)
Abbreviations: AA, aortic aneurysm; Abd, abdominal; AC, aortic center; AD, aortic dissection; ADDRS, aortic dissection detection risk score; ADL, activities of daily living; AED, automatic external defibrillation; AF, atrial fibrillation; AI, aortic insufficiency; aPTT, activated partial thromboplastin time; AUC, area under receiving operating curve; CA, cardiac arrest; CABG, coronary arter bypass graft; CCD, chest compression device; CI, confidence interval; CNN, convolutional neural networks; CPR, cardio-pulmonary resuscitation; CT, computerized tomography; CV, cardiovascular; D/C, discharge; DBP, diastolic blood pressure; DM, diabetes mellitus; DOB, date of birth; Dx, diagnosis; ECG, electrocardiogram; ED, emergency department; EDA, ED arrival time; EMD, emergency medical dispatch; EMS, emergency medical service; EMT, emergency medical technician; EXTEM, extrinsic coagulation pathway; FDP, fibrin degradation products; FIBTEM, function of fibrinogen pathway in the extrinsic pathway; GCS, Glasgow coma scale; GI, gastrointestinal; H&P, history and physical; HgB, hemoglobin; HR, heart rate; Htn, hypertension; Hx, history; INR, International Normalized Ratio; JCS, Japanese coma scale; LI, limb ischemia; LOS, length of stay; LPD, left pupil diameter; LSTM, long short-term memory; Lt, left; MVA, motor vehicle accident; Neuro, neurologic; PC, primary center; PEA, pulseless electric activity; PLR, pupil light reflex; POW, presence of witness; PT, prothrombin time; PVS, persistent vegetative state; Resp, respiratory; ROSC, return of spontaneous circulation; ROTEM, thromboelastometry; RPD, right pupil diameter; RR, respiratory rate; Rt, right; Rx, drugs; SAH, subarachnoid hemorrhage; SAPS II, simplified acute physiology score II; SB, spontaneous breathing; SBP, systolic blood pressure; SG, surgery; Site, site of incidence; SpO2 , oxygen saturation; TPT, transport; TTE, Transthoracic echocardiography; TTP, time to predict (the earliest time that the prediction model can be run based on the predictors); VA, vascular access; VASC, vascular; VF, ventricular fibrillation; VS, vital signs; VT, ventricular tachycardia; WBC, white blood count.
Development and validation of ML models using prehospital data are useful in early identification of time-sensitive conditions, such as trauma, stroke, and shockable rhythms post–out-of-hospital cardiac arrest (CA).[16 ]
[21 ]
[22 ] For instance, prediction of acute coagulopathy of trauma (ACT) score, based on prehospital data, was developed for early identification of ACT, a complication of trauma that may require early goal-directed treatment of massive transfusion for patient survival.[22 ] Therefore, the early information that the prehospital data provide can be an asset in improving time to predict time-sensitive conditions in ED.
Machine Learning Models Using Prehospital Data
Of the nine studies included in the review, common prehospital data elements used in ML models targeting early detection of time-sensitive conditions or prediction of patient outcomes include the following: (1) demographics, such as age and gender; chief complaints[15 ]
[16 ]
[20 ]
[21 ]
[22 ]
[24 ]
[25 ]; (2) prehospital level of consciousness such as Glasgow coma scale (GCS)[19 ]
[20 ]
[22 ]
[25 ]; (3) prehospital vital signs recorded in the ambulance or in the field[16 ]
[21 ]
[22 ]; (4) prehospital actions, characterized as intubation, cardiopulmonary resuscitation (CPR), and chest decompression[16 ]
[18 ]
[21 ]
[22 ]; or (5) prehospital situational assessments, described as mechanism of injury, witnessed arrest or CPR initiation, and presence of speech deficit and facial and limb weakness prior to arrival to ED.[15 ]
[18 ]
[21 ]
[22 ]
[Table 1 ] summarizes studies using prehospital data for clinical decision-making in ED based on their targets and modeling approaches. Benefits of using prehospital data to achieve ED outcomes were noted in all of the studies included in the review.
All 12 ML articles used prehospital data to impact ED clinical decision-making and ED outcomes. Accordingly, [Table 1 ] divides the data elements used for each study into pre–emergency department arrival (pre-EDA) and post-EDA and provides further insights about the earliest time that the prediction model can be run based on the predictors (referred in the table as time to predict, or TTP) in each study. Pre-EDA includes data from emergency call centers, dispatches stations and ambulance records collected prior to arrival to ED. Post-EDA includes data collected during hospitalization starting from the arrival to ED.
Use of Machine Learning–Related to Sepsis in ED
Early and accurate prediction of sepsis in ED can improve patient outcomes such as decreased inpatient mortality rate, length of stay, and readmission rates.[26 ] Advantage of ML models includes the ability to predict patient outcomes hours prior to onset of actual outcome such as diagnosis of sepsis and septic shock.[27 ]
[28 ]
[29 ]
[30 ]
[31 ] From the 12 studies included in this review, only 1 study used prospective research design in a clinical setting.[26 ] All other studies used retrospective data for ML modeling. A meta-analysis by Fleuren et al supported that ML models can accurately predict sepsis onset ahead of time in ED, floor, and ICU, providing a novel approach to early identification of sepsis where the biomarkers and screening tools, such as systematic inflammatory response syndrome (SIRS) and SOFA criteria, fail to include all the clinical relevant information.[32 ] The meta-analysis also showed the impact of prediction hours before sepsis onset on pooled area under receiving operating curve (AUC) for the ML models, further indicating that the prediction hours before onset of sepsis with relatively high AUC through ML model is plausible.[32 ]
Data Elements in Detection of Sepsis
The ML models used in the ED are evaluated based on their target, modeling approach, data element characteristics, time to predict, and AUC. [Table 2 ] provides a list of studies found in the search and summarizes them based on their target and modeling approach. It also includes a detailed list of variables used to develop the ML models for each study. Furthermore, the table provides the AUC and TTP which determine the earliest time that all required data elements for running the model are collected and the performance of the model. Therefore, for each target, by considering both TTP and prediction performance, this table helps the reader understand the practicality of the developed models if applied within clinical workflow.
Table 2
Summary of studies with prediction/early identification of sepsis using machine learning with their list of predictors in emergency department (ED)
Reference
Model
Target(s)
Predictors
Performance/result
[36 ]
Logistic regression with L1 regularization (LASSO)
30-day in-hospital mortality among septic patients
Mean RR; mean HR; NN50; TINN; power norm: very low, low, high frequency; LF/HF, ratio of LF power to HF power; SBP; temp; GCS; age; approximate entropy; sample entropy; detrended fluctuation analysis. TTP: 2 hours after EDA
12 selected HRVTS features outperformed model that only included patient demographics, vital signs and one HRV measures taken at triage.
AUC: 0.82
[34 ]
Random forest, classification and regression tree (CART) model, logistic regression model
28 days in-hospital mortality
SpO2; RR; BP; BUN; albumin; intubation in ED; need for vasopressors; age; RDW; potassium; AST; HR; acuity level (triage), ED impression (Dx), CO2 (laboratory), ECG performed, beta-blocker (home medication), cardiac dysrhythmia (primary medical history). TTP: after ED stay
Machine learning outperformed statistical models.
AUC: 0.86
[27 ]
Support vector model
Dx of an infection
Age; gender; acuity; SBP; DBP; HR; pain scale, RR, SpO2; temperature, free text chief complaint, free text nursing assessment. TTP: after ED stay
Best performing models included free text (NLP).
AUC: 0.89
[28 ]
Gradient tree boosting
Detect onset of sepsis, severe sepsis and septic shock
SBP; DBP; HR; RR; SpO2 and temperature. TTP: at the onset in ED, ICU and floor
Includes ED, ICU and floor. Early detection 4 hours before for septic shock. AUC: 0.92
[33 ]
Convolutional neural network plus softmax
Predict mortality over 72 hours and 28 days
5 BP; GCS; WBC; HgB; lymphocyte count; PT-INR; BUN; creatinine; bilirubin; AST; ALT; troponin; pH; bicarbonate level; atypical lymphocyte; promyelocyte; metamyelocyte; myelocyte; Sodium ion; potassium ion; albumin; BG; RDW-SD, MCV; RDW-CoV; Base excess; MCH; MCHC; MAP; RR; temperature, HR, Age, Sex, qSOFA score, shock episode, liver cirrhosis, DM; CRF; CHF; CVA; solid tumor, RI, UTI, STI; IAI; other infections, bacteremia, antibiotic used within 24 hours. TTP: 24 hours after EDA
Significant comparisons between different models; outperformed all machine learning and statistical models.
AUC: 0.94
[25 ]
Machine learning algodiagnostic (MLA)
In-hospital mortality
HR, RR, BP, SpO2, temperature. TTP: 1 hour after EDA
Prospective study noted decrease in sepsis-related in-hospital mortality rate, length of stay and 30-day readmission rate 60.24%, 9.55% and 50.14%, respectively.
AUC: 0.91
[26 ]
Multivariate linear regression
Dx of sepsis
Age, HR, RR, temperature, SBP, DBP, MAP, SpO2, HR to SBP ratio. TTP: After EDA with first vital signs.
HR: SBP ratio was higher in patients with sepsis than patients with trauma, stroke and acute coronary syndrome. ML (74%) has higher accuracy than SIRS (34%).
[35 ]
Multivariate logistic regression, decision tree, and naïve Bayes' classifier
In-hospital mortality
SIRS criteria: temperature, HR, RR, WBC count. qSOFA criteria: SBP, GCS, RR
TTP: after EDA with first vital signs.
Low and similar accuracy between the three models with variables related to SIRS and qSOFA. AUC: ranging from 0.622 to 0.696
[30 ]
SVM, gradient boosting machine with Bernoulli's loss, random forest, multivariate adaptive regression splines, lasso and ridge regression
Septic shock within 24 hours of arrival
Sex; chief complaints; initial vital signs: SBP, DBP, HR, RR, SpO2 and temperature; initial level of consciousness- AVPU scale; WBC, differential counts; RDW; platelet count; PT; INR; fibrinogen; BUN; sodium; potassium; chloride, creatinine, AST, ALT, ALP, total bilirubin; albumin; and C-reactive protein. TTP: 24 hours after arrival
All machine learning classifiers significantly outperformed statistically calculated models. Embedding chief complaints was statistically significant in improving performance
AUC: ranging from 0.883 to 0.929.
[37 ]
eXtreme gradient boosting, light gradient boosting and random forest
3-day mortality among patients with suspected infection in the ED
The qSOFA criteria, including systolic blood pressure, respiratory rate, and mental status.
TTP: After EDA with first vital signs
The machine learning models outperformed qSOFA score using the same variables in predicting 3-day mortality with suspected infections in ED
AUC: model 0.86.
[32 ]
Gradient-boosted tree models
Detection of sepsis
LA, shock index; WBS; neutrophils, change in LA; BG; BUN; RR; albumin; SBP; serum creatinine; temperature.
TTP: after EDA after laboratories are resulted
Models' ROS was more sensitive and precise compared with qSOFA score at 1 hour and at 24 hours.
AUC (95% CI): 0.93–0.97
[29 ]
Decision tree, discriminant analysis, logistic regression, KNN, neighbors, ensemble classification, SVM, and neural network
Identification of sepsis
Age, dialysis, mobility, mentation, SBP, HR, RR, LA, temperature, and WBC.
TTP: first 6 hours EDA
Although the fine Gaussian SVM showed highest sensitivity (95.8%), NN model had the highest accuracy (92.08%), high sensitivity (92.33), and high specificity (92.33%)
AUC: 0.92
Abbreviations: ALP, alkaline phosphatase; ALT, alanine aminotransferase; AST, aspartate aminotransferase; AUC, area under receiving operating curve [with 95% confidence interval]; AVPU scale, alert, verbal, pain and unresponsive scale; BG, blood sugar; BP, blood pressure; BUN, blood urea nitrogen; CHF, congestive heart failure; CoV, coefficient of variation; CRF, chronic renal failure; CVA, cerebral vascular accident; DBP, diastolic blood pressure; DM, diabetes mellitus; Dx, diagnosis; Dx, diagnosis; ECG, electrocardiogram; EDA, ED arrival time; GCS, Glasgow coma scale; HgB, hemoglobin; HR, heart rate; IAI, intra—abdominal infection; ICU, intensive care unit; INR, International Normalized Ratio; KNN, k-nearest neighbors; LA, lactic acid; MAP, mean arterial pressure; MCH, mean corpuscular hemoglobin; MCHC, mean corpuscular hemoglobin concentration; MCV, mean corpuscular volume; NN50, RR number of consecutive RR triangular index (TINN); pH, power of hydrogen (pH); PT, prothrombin time; qSOFA, quick sequential organ failure assessment; RDW, red cell distribution width; RI, respiratory infection; RN, registered Nurse; ROS, development of risk for sepsis; RR, respiratory rate; RT, respiratory therapist; SBP, systolic blood pressure; SD, standard deviation; SpO2, oxygen saturation; STI, soft tissue infection; SVM, support vector machine; TINN, baseline width of a triangle fit into the RR interval histogram using a least squares; TTP, time to predict (The earliest time that the prediction model can be run based on the predictors); UTI, urinary tract infection; WBC, white blood count.
Although ML models have been used to detect early infection, sepsis, and septic shock, research related to early identification of sepsis or septic shock in patients, specifically on arrival to ED is still limited.[27 ]
[28 ]
[29 ]
[30 ]
[31 ]
[33 ] Common variables identified from the 12 articles in the review for detection of infection, sepsis, or septic shock in all settings, such as ED, intensive care units, and floors, include SBP, diastolic blood pressure (DBP), heart rate, RR, peripheral capillary oxygen saturation, temperature, and chief complaints,[27 ]
[28 ]
[29 ]
[31 ] many of which often are not available in electronic health record (EHR) on patient arrival to ED due to lack of interoperability. For instance, Mao et al developed a gradient tree boosting model using data from only six vital signs: SBP, DBP, heart rate, RR, peripheral capillary oxygen saturation, and temperature.[29 ] This model was able to predict sepsis at the onset with high AUC (0.92) and septic shock 4 hours in advance with high AUC (0.96).[29 ] The model was also able to predict severe sepsis 4 hours in advance with higher AUC (0.85) than the onset time for statistically calculated SIRS AUC (0.75).[29 ] This study used data from ED, critical care and regular floors and required 3 hours of measurements of all six vital signs. Model performance might be different if limited to data collected during the early ED visit which may not have continuous vital signs recorded for three hours. Prehospital data can provide an opportunity to use a predictive model on patient arrival that includes these additional data elements.
Only four studies used ML models to predict early onset of infection, sepsis, and septic shock in patients in the ED.[28 ]
[30 ]
[31 ]
[33 ] In these studies, time needed for prediction of infection or sepsis ranged from 1 hour from the first vital signs on admission to ED to 24 hours after admission to ED.[28 ]
[30 ]
[31 ]
[33 ] Horng et al found that diagnosis of infection in ED could be predicted more accurately by using free-text chief complaints and nursing assessments along with structured data, such as vital signs and demographics.[28 ] This study used a variation of NLP in ED to uncover latent data from chief complaints and nursing assessment notes for early detection of infection. Mohamed et al used neurological assessment in the form of mentation along with other common variables to identify sepsis through multiple models where a neural network model performed the best with greater than 90% accuracy.[30 ] Due to limitation in available data, different tools, such as NLP, and uncommonly used data elements, such as neurological assessment in the form of mentation, may be needed for early prediction of sepsis.
From the 12 studies, 10 defined their diverse methods of handling missing values. Two studies included forward filling imputation[26 ]
[29 ]; two studies imputed by means, modes or median[31 ]
[34 ]; two studies converted missing values to categorical or nominal values[30 ]
[35 ]; one study used multivariate imputation by chained equations method[36 ]; one study used linear interpolation method[37 ]; one study excluded patients with missing data[38 ]; and one study automatically imputed physiologically normal values in the missing vital signs data.[28 ] The diversity in handling of the missing values especially with vital signs raises concerns related to functionality and ethics of ML model in clinical settings.
Discussion
The purpose of this integrative literature review was to discuss the importance of utilizing prehospital data elements in ED, summarize their current use in developing ML-based prediction models, and specifically identify those data elements that can potentially contribute to early identification of sepsis in ED when used in ML-based approaches. Prehospital data have been instrumental in the development and validation of statistical and ML models with the purpose of improving early detection of time-sensitive patient conditions, such as trauma, stroke, and shockable cardiac rhythms post–out-of-hospital cardiac arrest; and patient outcomes such as inpatient mortality, survival rate, and ROSC.[16 ]
[21 ]
[22 ] Currently, limited studies are available related to detection and prediction of sepsis in ED, and there was no study found related to detection of sepsis in ED using prehospital data. This gap in knowledge may be due to limitations in availability of prehospital data in the EHR and difficulty in diagnosis of sepsis in ED. There is an opportunity for researchers to develop and evaluate prediction models for early detection of sepsis using prehospital data in the ED in the future.
ML has been found to be superior in performance in terms of accuracy, specificity, and sensitivity compared with statistically calculated screening tools for sepsis prediction.[16 ]
[20 ]
[21 ]
[24 ]
[32 ]
[38 ] Based on current and previous definitions of sepsis, the early detection of sepsis has been linked with the following screening tools: SIRS, the modified Early Warning Scores (MEWS), and SOFA and qSOFA scores.[6 ]
[39 ] However, SIRS, MEWS, SOFA, and qSOFA scores when compared with ML models have performed poorly in identification of sepsis and in predicting inpatient mortality from sepsis in ED.[27 ]
[33 ]
[35 ]
[38 ] In a meta-analysis study by Islam et al, ML models showed higher accuracy for sepsis detection as evidenced by the AUC of 0.89 compared with sepsis screening tools such as SIRS, MEWS, and SOFA score with AUC of 0.70, 0.50, and 0.78, respectively.[39 ] Contrarily, two sets of decision tree models using the same variables as qSOFA and SIRS generated similar and low AUC and sensitivity.[36 ] Therefore, although the ML models in general outperform statistically calculated sepsis screening tools, the performance of the prediction models depend highly on the selection of the set of variables or data elements used to create the model.
As described in [Table 1 ], use of prehospital data could help expedite certain predictions in ED if the required data elements were collected earlier during prehospital care. For instance, a promising use of ML in ED using prehospital data could reduce overcrowding by managing the availability of personnel.[14 ] However, use of prehospital data are limited by the quality and availability of the data. Mashoufi et al conducted a survey among three groups of EMS stakeholders: data producers, data collectors, and data consumers.[40 ] They concluded that the quality of EMS data with respect to their usability, completeness, and compatibility are still low.[40 ] An additional caveat for developing an accurate prediction/early detection model in ED is balancing TTP, and the helpfulness of the inputted variables. TTP will be determined by the time that all data elements become available for analysis. For instance, Duceau et al used 32 prehospital and postarrival variables that included electrocardiogram (ECG), computed tomography (CT) scan, and transthoracic echocardiogram to predict acute aortic syndromes (AAS) and assess the performance of undertriage and overtriage.[24 ] Use of data elements that take a longer time to result such as diagnostic tests provides accuracy in performance but may delay the time to predict.[41 ] Contrarily, only using insufficient prehospital data, one model may be able to deliver prediction at arrival time but with potential cost of reduced performance. Therefore, balancing performance and time to predict is necessary in clinical setting with consideration to quality and availability of data.
This literature review identifies data elements that may be used for early prediction of sepsis in ED using ML models. ML models using prehospital data elements have been applied in ED, although not for sepsis, and have reported high performance in improving ED operations and predicting ED outcomes. Data elements found helpful in early prediction or detection of sepsis in ED are also highlighted, although none were from prehospital data. Therefore, the prehospital data elements, identified in [Table 2 ], and the data elements specific to sepsis, identified in [Table 2 ], may be combined to improve early identification of sepsis in ED. This integrative literature review emphasizes the current gap in use of prehospital data and its potential in supporting early identification of sepsis in ED using ML and provides data elements that may facilitate further predictive analysis research.
Limitations
Some limitations should be noted. One limitation is related to the literature search itself and its scope. The search for this literature review focused on the use of prehospital data for ED decision-making and the ML models for sepsis in ED. Any other studies, including interventional studies, not related to the two search strategies and our search terms were not included. The limited search terms used in this review excluded some of the studies that have been included in other reviews such as meta-analysis by Fleuren et al and a systematic review by Kareemi et al.[32 ]
[42 ] The purpose of our review also differed from these systematic reviews as the results focused on studies that used prehospital data for ED decision-making.
Another limitation is the use of retrospective research designs for most of the studies included in this review and lack of implementation of ML models in a clinical setting. Only one prospective study related to sepsis by McCoy and Das found reduction in in-hospital mortality rate by 60.24% postimplementation of sepsis prediction score alert system on floors, ICU, and ED.[26 ] That study also observed improvement in patient outcomes: a decrease in sepsis-related hospital length of stay by 9.55%, a decrease in 30-day readmission rate by 50.14%, and an increase in 3-hour sepsis bundle compliance by 72.7%.[26 ] However, the impact on patients specifically in ED and accuracy of the sepsis prediction score in ED was not analyzed in the study. Therefore, more research with the target outcome of early prediction or detection of sepsis focusing on ED patients using prehospital data are necessary to impact real-time clinical decision-making in ED and to improve patient outcomes.
There are also limitations in the use of prehospital data in practice, such as ML models using prehospital data that only benefit patients who arrive by ambulance. The lack of generalizability to all patients who arrive to ED limit the scope of the use of prehospital data. Limited access to data elements further limits the use of prehospital data. Despite the progress in recent years,[43 ]
[44 ] there is still a lack of interoperability that may limit the number and types of the prehospital data available for ML models.[45 ] Moreover, assuming the health care system has interoperability with EMS data with real-time access to the recorded data, availability of the included data elements at the time of prediction is crucial in designing clinical decision-support systems. Martin et al identified interoperability, accurate match algorithms, security, and wireless connectivity as potential barriers to adoption of prehospital health information exchanged.[13 ] Further development in ED–EMS interface is needed to enable prominent use of prehospital data in ED decision-making and promote research in use of prehospital data for improved patient outcomes. Additionally, incomplete or missing data are often a problem with prehospital data. Using various imputing techniques would likely be fraught with measurement error which would lead to inaccurate models. More research is needed in this area.
Conclusion
Sepsis has a significant impact on patient outcomes and HRQOL and financially impacts patients, their families, hospitals, community, and national health care costs. Sepsis is a time-sensitive condition that requires early detection and EGDT to improve patient mortality and related patient outcomes. Prehospital data in partnership with NLP and ML models can potentially improve clinical decision-making for sepsis in ED. Accessibility and limitations of the prehospital data, appropriateness of the variable selection, and the time it takes to generate a prediction have significant impacts on the performance of the ML models to improve patient outcomes for time-sensitive conditions, such as sepsis in ED.
Although ML models outperform sepsis severity scores and have the potential to predict at the onset or prior to sepsis and septic shock, the performance of the model depends heavily on selection of the variables or data elements. Future implications suggest development, validation, and application of the ML combined with NLP models using prehospital data in clinical settings to identify sepsis earlier and to support related clinical decision making in ED to improve patient outcomes.
