Optimizing Clinical Decision Support in the Electronic Health Record

Summary

Objective

Adoption of clinical decision support (CDS) tools by clinicians is often limited by workflow barriers. We sought to assess characteristics associated with clinician use of an electronic health record-embedded clinical decision support system (CDSS).

Methods

In a prospective study on emergency department (ED) activation of a CDSS tool across 14 hospitals between 9/1/14 to 4/30/15, the CDSS was deployed at 10 active sites with an on-site champion, education sessions, iterative feedback, and up to 3 gift cards/clinician as an incentive. The tool was also deployed at 4 passive sites that received only an introductory educational session. Activation of the CDSS – which calculated the Pulmonary Embolism Severity Index (PESI) score and provided guidance – and associated clinical data were collected prospectively. We used multivariable logistic regression with random effects at provider/facility levels to assess the association between activation of the CDSS tool and characteristics at: 1) patient level (PESI score), 2) provider level (demographics and clinical load at time of activation opportunity), and 3) facility level (active vs. passive site, facility ED volume, and ED acuity at time of activation opportunity).

Results

Out of 662 eligible patient encounters, the CDSS was activated in 55%: active sites: 68% (346/512); passive sites 13% (20/150). In bivariate analysis, active sites had an increase in activation rates based on the number of prior gift cards the physician had received (96% if 3 prior cards versus 60% if 0, p<0.0001). At passive sites, physicians < age 40 had higher rates of activation (p=0.03). In multivariable analysis, active site status, low ED volume at the time of diagnosis and PESI scores I or II (compared to III or higher) were associated with higher likelihood of CDSS activation.

Conclusions

Performing on-site tool promotion significantly increased odds of CDSS activation. Optimizing CDSS adoption requires active education.

Citation: Ballard DW, Vemula R, Chettipally UK, Kene MV, Mark DG, Elms AK, Lin JS, Reed ME, Huang J, Rauchwerger AS, Vinson DR, for the KP CREST Network Investigators. Optimizing clinical decision support in the electronic health record – clinical characteristics associated with the use of a decision tool for disposition of ED patients with pulmonary embolism.

Keywords

Decision-support - computerized - pulmonary embolism - electronic health record - research network

 

Conflict of Interest

The authors declare that they have no conflicts of interest in the research.

• References

• 1 McGlynn EA, Asch SM, Adams J, Keesey J, Hicks J, DeCristofaro A, Kerr EA. The quality of health care delivered to adults in the United States. The New England journal of medicine 2003; 348 (26) 2635-2645 doi: 10.1056/NEJMsa022615. PubMed PMID: 12826639.
• 2 Grimshaw JM, Eccles MP, Lavis JN, Hill SJ, Squires JE. Knowledge translation of research findings. Implementation science: IS 2012; 07: 50 doi: 10.1186/1748–5908–7–50. PubMed PMID: 22651257; PubMed Central PMCID: PMC3462671.
• 3 Balas EA, Boren SA. Managing clinical knowledge for health care improvement. Yearbook of Medical Informatics. 2000 65. 70.
• 4 Davidoff F. On the undiffusion of established practices. JAMA internal medicine 2015; 175 (05) 809-811 doi: 10.1001/jamainternmed.2015.0167. PubMed PMID: 25774743.
• 5 Michie S, van Stralen MM, West R. The behaviour change wheel: a new method for characterising and designing behaviour change interventions. Implementation science: IS 2011; 06: 42 doi: 10.1186/1748–5908–6–42. PubMed PMID: 21513547; PubMed Central PMCID: PMC3096582.
• 6 Diner BM, Carpenter CR, O’Connell T, Pang P, Brown MD, Seupaul RA, Celentano JJ, Mayer D, Members K-CTI. Graduate medical education and knowledge translation: role models, information pipelines, and practice change thresholds. Academic emergency medicine : official journal of the Society for Academic Emergency Medicine 2007; 14 (11) 1008-1014 doi: 10.1197/j.aem.2007.07.003. PubMed PMID: 17967963.
• 7 Carpenter CR, Milne WK. Knowledge translation and implementation science. Pseudoscience or the missing link between research and the bedside Emergency Medicine & Acute Care Essays 2015; 39 (06) 1-4.
• 8 Lang ES, Wyer PC, Eskin B. Executive summary: Knowledge translation in emergency medicine: establishing a research agenda and guide map for evidence uptake. Academic emergency medicine : official journal of the Society for Academic Emergency Medicine 2007; 14 (11) 915-918 doi: 10.1197/j.aem.2007.07.005. PubMed PMID: 17923716.
• 9 Kawamoto K, Houlihan CA, Balas EA, Lobach DF. Improving clinical practice using clinical decision support systems: a systematic review of trials to identify features critical to success. Bmj 2005; 330 7494 765 doi: 10.1136/bmj.38398.500764.8F. PubMed PMID: 15767266; PubMed Central PMCID: PMC555881.
• 10 Melnick ER, Nielson JA, Finnell JT, Bullard MJ, Cantrill SV, Cochrane DG, Halamka JD, Handler JA, Holroyd BR, Kamens D, Kho A, McClay J, Shapiro JS, Teich J, Wears RL, Patel SJ, Ward MF, Richardson LD. Delphi consensus on the feasibility of translating the ACEP clinical policies into computerized clinical decision support. Annals of emergency medicine 2010; 56 (04) 317-320 doi: 10.1016/j.annemergmed.2010.03.006. PubMed PMID: 20363531.
• 11 Bright TJ, Wong A, Dhurjati R, Bristow E, Bastian L, Coeytaux RR, Samsa G, Hasselblad V, Williams JW, Musty MD, Wing L, Kendrick AS, Sanders GD, Lobach D. Effect of Clinical Decision-Support Systems: A Systematic Review. Ann Intern Med 2012; 157 (01) 29-43 Epub 2012/04/25.doi: 0003–4819–157–1–201207030–00450 [pii] 10.1059/0003–4819–157–1–201207030–00450. PubMed PMID: 22529043.
• 12 Roy PM, Durieux P, Gillaizeau F, Legall C, Armand-Perroux A, Martino L, Hachelaf M, Dubart AE, Schmidt J, Cristiano M, Chretien JM, Perrier A, Meyer G. A computerized handheld decision-support system to improve pulmonary embolism diagnosis: a randomized trial. Ann Intern Med 2009; 151 (10) 677-686 Epub 2009/11/19. doi: 151/10/677 [pii] 10.1059/0003–4819–151–10–200911170–00003. PubMed PMID: 19920268.
• 13 Drescher FS, Chandrika S, Weir ID, Weintraub JT, Berman L, Lee R, Van Buskirk PD, Wang Y, Adewunmi A, Fine JM. Effectiveness and acceptability of a computerized decision support system using modified Wells criteria for evaluation of suspected pulmonary embolism. Ann Emerg Med 2011; 57 (06) 613-621 Epub 2010/11/06.doi: S0196–0644(10)01557-X [pii] 10.1016/j.annemergmed.2010.09.018. PubMed PMID: 21050624.
• 14 Patel PB, Vinson DR. Team assignment system: expediting emergency department care. Annals of emergency medicine 2005; 46 (06) 499-506 Epub 2005/11/26.doi: 10.1016/j.annemergmed.2005.06.012. PubMed PMID: 16308063.
• 15 Escobar GJ, Greene JD, Scheirer P, Gardner MN, Draper D, Kipnis P. Risk-adjusting hospital inpatient mortality using automated inpatient, outpatient, and laboratory databases. Medical care 2008; 46 (03) 232-239 doi: 10.1097/MLR.0b013e3181589bb6. PubMed PMID: 18388836.
• 16 Escobar GJ, Ragins A, Scheirer P, Liu V, Robles J, Kipnis P. Nonelective Rehospitalizations and Postdischarge Mortality: Predictive Models Suitable for Use in Real Time. Medical care 2015; 53 (11) 916-923 doi: 10.1097/MLR.0000000000000435. PubMed PMID: 26465120.
• 17 van Walraven C, Escobar GJ, Greene JD, Forster AJ. The Kaiser Permanente inpatient risk adjustment methodology was valid in an external patient population. Journal of clinical epidemiology 2010; 63 (07) 798-803 doi: 10.1016/j.jclinepi.2009.08.020. PubMed PMID: 20004550.
• 18 Ballard DW, Kim AS, Huang J, Park DK, Kene MV, Chettipally UK, Iskin HR, Hsu J, Vinson DR, Mark DG, Reed ME, Investigators KCN. Implementation of Computerized Physician Order Entry Is Associated With Increased Thrombolytic Administration for Emergency Department Patients With Acute Ischemic Stroke. Annals of emergency medicine 2015; 66 (06) 601-10 doi: 10.1016/j.annemergmed.2015.07.018. PubMed PMID: 26362574.
• 19 Aujesky D, Obrosky DS, Stone RA, Auble TE, Perrier A, Cornuz J, Roy PM, Fine MJ. Derivation and validation of a prognostic model for pulmonary embolism. American journal of respiratory and critical care medicine 2005; 172 (08) 1041-1046 doi: 10.1164/rccm.200506–862OC. PubMed PMID: 16020800; PubMed Central PMCID: PMC2718410.
• 20 Aujesky D, Roy PM, Le Manach CP, Verschuren F, Meyer G, Obrosky DS, Stone RA, Cornuz J, Fine MJ. Validation of a model to predict adverse outcomes in patients with pulmonary embolism. European heart journal 2006; 27 (04) 476-481 doi: 10.1093/eurheartj/ehi588. PubMed PMID: 16207738.
• 21 Brehaut JC, Hamm R, Majumdar S, Papa F, Lott A, Lang E. Cognitive and social issues in emergency medicine knowledge translation: a research agenda. Academic emergency medicine: official journal of the Society for Academic Emergency Medicine 2007; 14 (11) 984-990 doi: 10.1197/j.aem.2007.06.025. PubMed PMID: 17893396.
• 22 Reed M, Huang J, Brand R, Graetz I, Neugebauer R, Fireman B, Jaffe M, Ballard DW, Hsu J. Implementation of an outpatient electronic health record and emergency department visits, hospitalizations, and office visits among patients with diabetes. Jama 2013; 310 (10) 1060-1065 doi: 10.1001/jama.2013.276733. PubMed PMID: 24026601.
• 23 Ballard DW, Reed ME, Singh N, Rauchwerger AS, Hamity CA, Warton EM, Chettipally UK, Mark DG, Vinson DR, Investigators KPCN. Emergency Department Management of Atrial Fibrillation and Flutter and Patient Quality of Life at One Month Postvisit. Ann Emerg Med 2015; 66 (06) 646-654 e2. doi: 10.1016/j.annemergmed.2015.04.011. PubMed PMID: 26033532.
• 24 Kaji AH, Schriger D, Green S. Looking through the retrospectoscope: reducing bias in emergency medicine chart review studies. Ann Emerg Med 2014; 64 (03) 292-298 doi: 10.1016/j.annemergmed.2014.03.025. PubMed PMID: 24746846.
• 25 Song H, Tucker AL, Murrell KL. The Diseconomies of Queue Pooling: An Empirical Investigation of Emergency Department Length of Stay. Management Science 2015; 61 (12) 3032-3053 doi: http://dx.doi.org/10.1287/mnsc.2014.2118.
• 26 Rogers GRle, Narvaez Y, Venkatesh AK, Fleischman W, Hall MK, Taylor RA, Hersey D, Sette L, Melnick ER. Improving emergency physician performance using audit and feedback: a systematic review. The American journal of emergency medicine 2015; 33 (10) 1505-1514 doi: 10.1016/j.ajem.2015.07.039. PubMed PMID: 26296903.
• 27 Ivers N, Jamtvedt G, Flottorp S, Young JM, Odgaard-Jensen J, French SD, O’Brien MA, Johansen M, Grimshaw J, Oxman AD. Audit and feedback: effects on professional practice and healthcare outcomes. The Cochrane database of systematic reviews 2012; 06: CD000259 doi: 10.1002/14651858.CD000259.pub3. PubMed PMID: 22696318.
• 28 Meeker D, Linder JA, Fox CR, Friedberg MW, Persell SD, Goldstein NJ, Knight TK, Hay JW, Doctor JN. Effect of Behavioral Interventions on Inappropriate Antibiotic Prescribing Among Primary Care Practices: A Randomized Clinical Trial. Jama 2016; 315 (06) 562-570 doi: 10.1001/jama.2016.0275. PubMed PMID: 26864410.
• 29 Genco EK, Forster JE, Flaten H, Goss F, Heard KJ, Hoppe J, Monte AA. Clinically Inconsequential Alerts: The Characteristics of Opioid Drug Alerts and Their Utility in Preventing Adverse Drug Events in the Emergency Department. Annals of emergency medicine. 2015 doi: 10.1016/j.annemergmed.2015.09.020. PubMed PMID: 26553282.
• 30 Hill Jr RG, Sears LM, Melanson SW. 4000 clicks: a productivity analysis of electronic medical records in a community hospital ED. The American journal of emergency medicine 2013; 31 (11) 1591-1594 doi: 10.1016/j.ajem.2013.06.028. PubMed PMID: 24060331.
• 31 Yadav K, Chamberlain JM, Lewis VR, Abts N, Chawla S, Hernandez A, Johnson J, Tuveson G, Burd RS. Designing Real-time Decision Support for Trauma Resuscitations. Academic emergency medicine : official journal of the Society for Academic Emergency Medicine 2015; 22 (09) 1076-1084 doi: 10.1111/acem.12747. PubMed PMID: 26300010.
• 32 Ballesca MA, Laguardia JC, Lee PC, Hwang AM, Park DK, Gardner MN, Turk BJ, Kipnis P, Escobar GJ. An electronic order set for acute myocardial infarction is associated with improved patient outcomes through better adherence to clinical practice guidelines. J Hosp Med 2014; 09 (03) 155-161 doi: 10.1002/jhm.2149. PubMed PMID: 24493376.
• 33 Sheehan B, Nigrovic LE, Dayan PS, Kuppermann N, Ballard DW, Alessandrini E, Bajaj L, Goldberg H, Hoffman J, Offerman SR, Mark DG, Swietlik M, Tham E, Tzimenatos L, Vinson DR, Jones GS, Bakken S. Pediatric Emergency Care Applied Research N. Informing the design of clinical decision support services for evaluation of children with minor blunt head trauma in the emergency department: a sociotechnical analysis. Journal of biomedical informatics 2013; 46 (05) 905-913 doi: 10.1016/j.jbi.2013.07.005. PubMed PMID: 23892207.
• 34 Hwang U, McCarthy ML, Aronsky D, Asplin B, Crane PW, Craven CK, Epstein SK, Fee C, Handel DA, Pines JM, Rathlev NK, Schafermeyer RW, Zwemer Jr FL, Bernstein SL. Measures of crowding in the emergency department: a systematic review. Academic emergency medicine: official journal of the Society for Academic Emergency Medicine 2011; 18 (05) 527-538 doi: 10.1111/j.1553–2712.2011.01054.x. PubMed PMID: 21569171.
• 35 Ben-Yakov M, Kapral MK, Fang J, Li S, Vermeulen MJ, Schull MJ. The Association Between Emergency Department Crowding and the Disposition of Patients With Transient Ischemic Attack or Minor Stroke. Academic emergency medicine : official journal of the Society for Academic Emergency Medicine. 2015 doi: 10.1111/acem.12766. PubMed PMID: 26398233.
• 36 Hollander MJ, Kadlec H. Incentive-Based Primary Care: Cost and Utilization Analysis. The Permanente journal 2015; 19 (04) 46-56 doi: 10.7812/TPP/15–045. PubMed PMID: 26263389.
• 37 Nishijima DK, Offerman SR, Ballard DW, Vinson DR, Chettipally UK, Rauchwerger AS, Reed ME, Holmes JF. Clinical Research in Emergency S, Treatment N. Immediate and delayed traumatic intracranial hemorrhage in patients with head trauma and preinjury warfarin or clopidogrel use. Annals of emergency medicine 2012; 59 (06) 460-468 e1-7. doi: 10.1016/j.annemergmed.2012.04.007. PubMed PMID: 22626015; PubMed Central PMCID: PMC3361700.
• 38 Eagles D, Stiell IG, Clement CM, Brehaut J, Taljaard M, Kelly AM, Mason S, Kellermann A, Perry JJ. International survey of emergency physicians’ awareness and use of the Canadian Cervical-Spine Rule and the Canadian Computed Tomography Head Rule. Academic emergency medicine : official journal of the Society for Academic Emergency Medicine 2008; 15 (12) 1256-1261 doi: 10.1111/j.1553–2712.2008.00265.x. PubMed PMID: 18945241.
• 39 Graham ID, Stiell IG, Laupacis A, O’Connor AM, Wells GA. Emergency physicians’ attitudes toward and use of clinical decision rules for radiography. Academic emergency medicine : official journal of the Society for Academic Emergency Medicine 1998; 05 (02) 134-140 PubMed PMID: 9492134.
• 40 Kastner M, Estey E, Perrier L, Graham ID, Grimshaw J, Straus SE, Zwarenstein M, Bhattacharyya O. Understanding the relationship between the perceived characteristics of clinical practice guidelines and their uptake: protocol for a realist review. Implementation science: IS 2011; 06: 69 doi: 10.1186/1748–5908–6–69. PubMed PMID: 21733160; PubMed Central PMCID: PMC3224565.