PDF herunterladen
Appl Clin Inform 2017; 08(03): 910-923
DOI: 10.4338/ACI-2017-01-RA-0006
Research Article
Schattauer GmbH

Clinical decisions support malfunctions in a commercial electronic health record

Steven Z. Kassakian
1   Department of Medical Informatics and Clinical Epidemiology, Oregon Health and Science University, Portland, Oregon/USA
,
Thomas R. Yackel
1   Department of Medical Informatics and Clinical Epidemiology, Oregon Health and Science University, Portland, Oregon/USA
,
Paul N. Gorman
1   Department of Medical Informatics and Clinical Epidemiology, Oregon Health and Science University, Portland, Oregon/USA
,
David A. Dorr
1   Department of Medical Informatics and Clinical Epidemiology, Oregon Health and Science University, Portland, Oregon/USA
› Institutsangaben Funding Research reported in this publication was supported by the National Library of Medicine of the National Institutes of Health under Award Number T15LM007088. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.
Weitere Informationen

Publikationsverlauf

received: 10. Januar 2017

accepted in revised form: 21. Mai 2017

Publikationsdatum:
20. Dezember 2017 (online)

   Lizenzen und Reprints

Summary

Objectives: Determine if clinical decision support (CDS) malfunctions occur in a commercial electronic health record (EHR) system, characterize their pathways and describe methods of detection.

Methods: We retrospectively examined the firing rate for 226 alert type CDS rules for detection of anomalies using both expert visualization and statistical process control (SPC) methods over a five year period. Candidate anomalies were investigated and validated.

Results: Twenty-one candidate CDS anomalies were identified from 8,300 alert-months. Of these candidate anomalies, four were confirmed as CDS malfunctions, eight as false-positives, and nine could not be classified. The four CDS malfunctions were a result of errors in knowledge management: 1) inadvertent addition and removal of a medication code to the electronic formulary list; 2) a seasonal alert which was not activated; 3) a change in the base data structures; and 4) direct editing of an alert related to its medications. 154 CDS rules (68%) were amenable to SPC methods and the test characteristics were calculated as a sensitivity of 95%, positive predictive value of 29% and F-measure 0.44.

Discussion: CDS malfunctions were found to occur in our EHR. All of the pathways for these malfunctions can be described as knowledge management errors. Expert visualization is a robust method of detection, but is resource intensive. SPC-based methods, when applicable, perform reasonably well retrospectively.

Conclusion: CDS anomalies were found to occur in a commercial EHR and visual detection along with SPC analysis represents promising methods of malfunction detection.

Citation: Kassakian SZ, Yackel TR, Gorman PN, Dorr DA. Clinical decisions support malfunctions in a commercial electronic health record. Appl Clin Inform 2017; 8: 910–923 https://doi.org/10.4338/ACI-2017-01-RA-0006

Keywords

Clinical decision support - errors - malfunction - alerts - electronic health record - electronic medical record

Human Subjects Protection

The study was performed in compliance with World Medical Association Declaration of Helsinki on Ethical Principles for Medical Research Involving Human Subjects.


 

  • References

  • 1 Bright TJ, Wong A, Dhurjati R, Bristow E, Bastian L, Coeytaux RR, Samsa G, Hasselblad V, Williams JW, Musty MD, Wing L, Kendrick AS, Sanders GD, Lobach D. Effect of clinical decision-support systems: a systematic review. Annals of internal medicine 2012; 157 (01) 29-43.
    CrossrefPubMedGoogle Scholar
  • 2 Lobach D. Enabling Health Care Decision-making Through Clinical Decision Support and Knowledge Management. 2012
    PubMedGoogle Scholar
  • 3 McDonald CJ. Protocol-based computer reminders, the quality of care and the non-perfectability of man. The New England journal of medicine 1976; 295 (24) 1351-5.
    CrossrefPubMedGoogle Scholar
  • 4 Moja L, Kwag KH, Lytras T, Bertizzolo L, Brandt L, Pecoraro V, Rigon G, Vaona A, Ruggiero F, Mangia M, Iorio A, Kunnamo I, Bonovas S. Effectiveness of computerized decision support systems linked to electronic health records: a systematic review and meta-analysis. Am J Public Health 2014; 104 (12) e12-22.
    CrossrefPubMedGoogle Scholar
  • 5 Roshanov PS, Fernandes N, Wilczynski JM, Hemens BJ, You JJ, Handler SM, Nieuwlaat R, Souza NM, Beyene J, Van Spall HG, Garg AX, Haynes RB. Features of effective computerised clinical decision support systems: meta-regression of 162 randomised trials. Bmj. 2013;346:f657. doi: 10.1136/bmj.f657. PubMed PMID: 23412440.
    PubMedGoogle Scholar
  • 6 Ash JS, Sittig DF, Campbell EM, Guappone KP, Dykstra RH. Some unintended consequences of clinical decision support systems. AMIA Annu Symp Proc 2007: 26-30.
    PubMedGoogle Scholar
  • 7 Shah NR, Seger AC, Seger DL, Fiskio JM, Kuperman GJ, Blumenfeld B, Recklet EG, Bates DW, Gandhi TK. Improving acceptance of computerized prescribing alerts in ambulatory care. Journal of the American Medical Informatics Association : JAMIA 2006; 13 (01) 5-11.
    CrossrefPubMedGoogle Scholar
  • 8 Embi PJ, Leonard AC. Evaluating alert fatigue over time to EHR-based clinical trial alerts: findings from a randomized controlled study. Journal of the American Medical Informatics Association JAMIA 2012; 19 e1 e145-8.
    CrossrefPubMedGoogle Scholar
  • 9 Cash JJ. Alert fatigue. Am J Health Syst Pharm 2009; 66 (23) 2098-101.
    CrossrefPubMedGoogle Scholar
  • 10 Osheroff JA, Teich JM, Levick D, Saldana, Velasco F, Sittig D, Rogers K, Jenders RA. Improving Outcomes with Clinical Decision Support An Implementers Guide. Second ed: Healthcare Information Management Society 2012
    PubMedGoogle Scholar
  • 11 Ash JS, Sittig DF, Dykstra R, Campbell E, Guappone K. The unintended consequences of computerized provider order entry: findings from a mixed methods exploration. International journal of medical informatics 2009; 78 (Suppl. 01) S69-76.
    CrossrefPubMedGoogle Scholar
  • 12 Wright A, Hickman TT, McEvoy D, Aaron S, Ai A, Andersen JM, Hussain S, Ramoni R, Fiskio J, Sittig DF, Bates DW. Analysis of clinical decision support system malfunctions: a case series and survey. Journal of the American Medical Informatics Association JAMIA. 2016 doi: 10.1093/jamia/ocw005. PubMed PMID: 27026616.
    PubMedGoogle Scholar
  • 13 Ash JS, Sittig DF, Campbell EM, Guappone KP, Dykstra RH, Ash JS, Sittig DF, Campbell EM, Guappone KP, Dykstra RH. Some unintended consequences of clinical decision support systems. AMIA Annu Symp Proc. 2007;Annual Symposium Proceedings/AMIA Symposium 26–30. PubMed PMID: 18693791.
    PubMedGoogle Scholar
  • 14 Ash JS, Sittig DF, Dykstra R, Wright A, McMullen C, Richardson J, Middleton B. Identifying best practices for clinical decision support and knowledge management in the field. Studies in Health Technology & Informatics 2010; 160 Pt 2 806-10.
    PubMedGoogle Scholar
  • 15 Wright A, Sittig DF, Ash JS, Feblowitz J, Meltzer S, McMullen C, Guappone K, Carpenter J, Richardson J, Simonaitis L, Evans RS, Nichol WP, Middleton B. Development and evaluation of a comprehensive clinical decision support taxonomy: comparison of front-end tools in commercial and internally developed electronic health record systems. Journal of the American Medical Informatics Association : JAMIA 2011; 18 (03) 232-42.
    CrossrefPubMedGoogle Scholar
  • 16 Wright A, Sittig DF, Ash JS, Sharma S, Pang JE, Middleton B. Clinical decision support capabilities of commercially-available clinical information systems. Journal of the American Medical Informatics Association JAMIA 2009; 16 (05) 637-44.
    CrossrefPubMedGoogle Scholar
  • 17 Carey RG. Improving healthcare with control charts : basic and advanced SPC methods and case studies. Milwaukee, WI: ASQ Quality Press; 2003. xxiv, 194 p. p.
    Google Scholar
  • 18 Ben-Gal I. Outlier Detection: Kluwer Academic Publishers. 2005.
    PubMedGoogle Scholar
  • 19 Smith Jr., SC, Benjamin EJ, Bonow RO, Braun LT, Creager MA, Franklin BA, Gibbons RJ, Grundy SM, Hiratzka LF, Jones DW, Lloyd-Jones DM, Minissian M, Mosca L, Peterson ED, Sacco RL, Spertus J, Stein JH, Taubert KA, World Heart F, the Preventive Cardiovascular Nurses A. AHA/ACCF Secondary Prevention and Risk Reduction Therapy for Patients with Coronary and other Atherosclerotic Vascular Disease: 2011 update: a guideline from the American Heart Association and American College of Cardiology Foundation. Circulation 2011; 124 (22) 2458-73.
    CrossrefPubMedGoogle Scholar
  • 20 Keim DA. Information Visualization and Visual Data Mining. IEEE Trans Vis Comput Graph 2002; 8 (01) 1-8.
    CrossrefPubMedGoogle Scholar
  • 21 Zahabi M, Kaber DB, Swangnetr M. Usability and Safety in Electronic Medical Records Interface Design: A Review of Recent Literature and Guideline Formulation. Hum Factors 2015; 57 (05) 805-34.
    CrossrefPubMedGoogle Scholar
  • 22 Schiff G, Wright A, Bates DW, Salazar A, Amato MG, Slight SP, Sequist TD, Loudin B, Smith D, Adelman J, Lambert B, Galanter W, Koppel R, McGreevey J, Taylor K, Chan I, Brennan C. Computerized Prescriber Order Entry Medication Safety (CPOEMS): Uncovering and Learning From Issues and Errors. Sliver Spring, MD: US Food and Druf Administration, 2015 December, 15, 2015. Report No.
    PubMedGoogle Scholar
  • 23 van der Sijs H, Aarts J, Vulto A, Berg M. Overriding of drug safety alerts in computerized physician order entry. Journal of the American Medical Informatics Association JAMIA 2006; 13 (02) 138-47.
    CrossrefPubMedGoogle Scholar