Measures of User experience in a Streptococcal pharyngitis and Pneumonia Clinical Decision Support Tools

Summary

Objective: To understand clinician adoption of CDS tools as this may provide important insights for the implementation and dissemination of future CDS tools.

Materials and Methods: Clinicians (n=168) at a large academic center were randomized into intervention and control arms to assess the impact of strep and pneumonia CDS tools. Intervention arm data were analyzed to examine provider adoption and clinical workflow. Electronic health record data were collected on trigger location, the use of each component and whether an antibiotic, other medication or test was ordered. Frequencies were tabulated and regression analyses were used to determine the association of tool component use and physician orders.

Results: The CDS tool was triggered 586 times over the study period. Diagnosis was the most frequent workflow trigger of the CDS tool (57%) as compared to chief complaint (30%) and diagnosis/antibiotic combinations (13%). Conversely, chief complaint was associated with the highest rate (83%) of triggers leading to an initiation of the CDS tool (opening the risk prediction calculator). Similar patterns were noted for initiation of the CDS bundled ordered set and completion of the entire CDS tool pathway. Completion of risk prediction and bundled order set components were associated with lower rates of antibiotic prescribing (OR 0.5; CI 0.2-1.2 and OR 0.5; CI 0.3-0.9, respectively).

Discussion: Different CDS trigger points in the clinician user workflow lead to substantial variation in downstream use of the CDS tool components. These variations were important as they were associated with significant differences in antibiotic ordering.

Conclusions: These results highlight the importance of workflow integration and flexibility for CDS success.

Citation: Mann D, Knaus M, McCullagh L, Sofianou A, Rosen L, McGinn T, Kannry J. Measures of user experience in a streptococcal pharyngitis and pneumonia clinical decision support tools. Appl Clin Inf 2014; 5: 824–835

http://dx.doi.org/10.4338/ACI-2014-04-RA-0043

Keywords

Usability - workflow - clinical decision support - electronic health records - clinical prediction rules - evidence-based medicine

 

Conflicts of interest

None of the listed authors have any financial or personal relationships with other people or organizations that may inappropriately influence or bias the objectivity of submitted content and/or its acceptance for publication in this journal.

• References

• 1 Bright TJ, Wong A, Dhurjati R. et al. Effect of clinical decision-support systems: A systematic review. Ann Intern Med. 2012 doi: 10.1059/0003–4819–157–1–201207030–00450. PubMed PMID: 22529043.
• 2 Jha AK. Meaningful use of electronic health records: the road ahead. JAMA 2010; 304 (15) 1709-1710. doi: 10.1001/jama.2010.1497. PubMed PMID: 20959581.
• 3 Blumenthal D, Tavenner M. The „meaningful use“ regulation for electronic health records. N Engl J Med 2010; 363 (06) 501-504. doi: 10.1056/NEJMp1006114. PubMed PMID: 20647183.
• 4 Jones SS, Rudin RS, Perry T. et al. Health information technology: An updated systematic review with a focus on meaningful use. Annals of Internal Medicine 2014; 160 (01) 48-54. doi: 10.7326/M13–1531.
• 5 Raja AS, Ip IK, Prevedello LM. et al. Effect of computerized clinical decision support on the use and yield of CT pulmonary angiography in the emergency department. Radiology 2012; 262 (02) 468-474. doi: 10.1148/radiol.11110951. PubMed PMID: 22187633; PubMed Central PMCID: PMC3267081.
• 6 Agency for Healthcare Research and Quality.. Enabling health care decision making through clinical decision support and knowledge management. In: AHRQ; editor. Rockville, MD: 2012
• 7 HealthIT.gov. Meaningful use stage 2 2013.. Available from: http://www.healthit.gov/policy-researchers-implementers/meaningful-use-stage-2.
• 8 Services CfMaM.. EHR incentive program: Data and reports 2012 [cited 2012 July 27]. Available from: http://www.cms.gov/Regulations-and-Guidance/Legislation/EHRIncentivePrograms/DataAndReports.html.
• 9 Fung CH, Tsai JS, Lulejian A. et al. An evaluation of the Veterans Health Administration’s clinical reminders system: a national survey of generalists. J Gen Intern Med 2008; 23 (04) 392-398. doi: 10.1007/s11606–007–0417–8. PubMed PMID: 18373135; PubMed Central PMCID: PMC2359516.
• 10 Mann D. Making clinical decision support more supportive. Medical Care 2011; 39 (02) 115-116.
• 11 Persell SD, Kaiser D, Dolan NC. et al. Changes in performance after implementation of a multifaceted electronic-health-record-based quality improvement system. Med Care 2011; 49 (02) 117-125. doi: 10.1097/MLR.0b013e318202913d. PubMed PMID: 21178789.
• 12 Fiks AG. Designing computerized decision support that works for clinicians and families. Curr Probl Pediatr Adolesc Health Care 2011; 41 (03) 60-88. doi: 10.1016/j.cppeds.2010.10.006. PubMed PMID: 21315295.
• 13 Gonzales R, Anderer T, McCulloch CE. et al. A cluster randomized trial of decision support strategies for reducing antibiotic use in acute bronchitis. JAMA Internal Medicine 2013; 173 (04) 267-273. doi: 10.1001/jamainternmed.2013.1589.
• 14 Smith DH, Perrin N, Feldstein A. et al. The impact of prescribing safety alerts for elderly persons in an electronic medical record: an interrupted time series evaluation. Arch Intern Med 2006; 166 (10) 1098-1104. PubMed PMID: 16717172.
• 15 Seidling HM, Schmitt SP, Bruckner T. et al. Patient-specific electronic decision support reduces prescription of excessive doses. Qual Saf Health Care 2010; 19: e15.
• 16 Linder JA, Schnipper JL, Tsurikova R. et al. Electronic health record feedback to improve antibiotic prescribing for acute respiratory infections. Am J Manag Care 2010; 16 12 Suppl. HIT e311-e319. PubMed PMID: 21322301.
• 17 Sequist TD, Morong SM, Marston A. et al. Electronic risk alerts to improve primary care management of chest pain: a randomized, controlled trial. J Gen Intern Med 2012; 27 (04) 438-444. doi: 10.1007/s11606–011–1911–6. PubMed PMID: 21993999; PubMed Central PMCID: PMC3304044.
• 18 Garg AX, Adhikari NK, McDonald H. et al. Effects of computerized clinical decision support systems on practitioner performance and patient outcomes: a systematic review. JAMA 2005; 293 (10) 1223-1238. doi: 10.1001/jama.293.10.1223. PubMed PMID: 15755945.
• 19 Krall MA, Sittig DF. Clinician’s assessments of outpatient electronic medical record alert and reminder usability and usefulness requirements. Proc AMIA Symp. 2002 400–404. PubMed PMID: 12463855.
• 20 Li AC, Kannry JL, Kushniruk A. et al. Integrating usability testing and think-aloud protocol analysis with „near-live“ clinical simulations in evaluating clinical decision support. Int J Med Inform. 2012 Epub 2012/03/30. doi: 10.1016/j.ijmedinf.2012.02.009. PubMed PMID: 22456088.
• 21 Horsky J, McColgan K, Pang JE. et al. Complementary methods of system usability evaluation: surveys and observations during software design and development cycles. J Biomed Inform 2010; 43 (05) 782-790. doi: 10.1016/j.jbi.2010.05.010. PubMed PMID: 20546936.
• 22 Bates DW, Kuperman GJ, Wang S. et al. Ten commandments for effective clinical decision support: making the practice of evidence-based medicine a reality. J Am Med Inform Assoc 2003; 10 (06) 523-530. doi: 10.1197/jamia. M1370. PubMed PMID: 12925543; PubMed Central PMCID: PMC264429.
• 23 Jamoom E, Beatty P, Bercovitz A. et al. Physician adoption of electronic health record systems: United States, 2011. In: Statistics NCfH. editor. Hyattsville, MD: 2012
• 24 Persell SD, Dolan NC, Friesema EM. et al. Frequency of inappropriate medical exceptions to quality measures. Ann Intern Med 2010; 152 (04) 225-231. Epub 2010/02/17. doi: 10.7326/0003–4819–152–4–201002160–00007. PubMed PMID: 20157137.
• 25 Kawamoto K, Houlihan C, Balas E. et al. Improving clinical practice using clinical decision support systems: a systematic review of trials to identify features critical to success. BMJ. 2005; 9 (03) 208-10.
• 26 Prevention CfDCa.. Antibiotic resistance threats in the United States, 2013. Atlanta, GA: Centers for Disease Control and Prevention, Division of Healthcare Quality Promotion NCfEaZID; 2013
• 27 Kashfi H. Applying a user centered design methodology in a clinical context. Studies in health technology and informatics 2010; 160 Pt 2 927-931. PubMed PMID: 20841820.
• 28 McGinn TG, McCullagh L, Kannry J. et al. Efficacy of an evidence-based clinical decision support in primary care practices: A randomized clinical trial. JAMA Internal Medicine; 2013. doi: 10.1001/jamainternmed.2013.8980.
• 29 Mann DM, Kannry JL, Edonyabo D. et al. Rationale, design, and implementation protocol of an electronic health record integrated clinical prediction rule (iCPR) randomized trial in primary care. Implement Sci 2011; 6 (01) 109. doi: 10.1186/1748–5908–6–109. PubMed PMID: 21929769; PubMed Central PMCID: PMC3184082.
• 30 Litvin CB, Ornstein SM, Wessell AM. et al. Adoption of a clinical decision support system to promote judicious use of antibiotics for acute respiratory infections in primary care. Int J Med Inform 2012; 81 (08) 521-526. doi: 10.1016/j.ijmedinf.2012.03.002. PubMed PMID: 22483528.
• 31 Litvin CB, Ornstein SM, Wessell AM. et al. Use of an electronic health record clinical decision support tool to improve antibiotic prescribing for acute respiratory infections: The ABX-TRIP Study. J Gen Intern Med. 2012 doi: 10.1007/s11606–012–2267–2. PubMed PMID: 23117955.
• 32 Mainous AG, 3rd Lambourne CA, Nietert PJ. Impact of a clinical decision support system on antibiotic prescribing for acute respiratory infections in primary care: quasi-experimental trial. J Am Med Inform Assoc. 2012 doi: 10.1136/amiajnl-2011–000701. PubMed PMID: 22759620.
• 33 Linder JA, Schnipper JL, Tsurikova R. et al. Documentation-based clinical decision support to improve antibiotic prescribing for acute respiratory infections in primary care: a cluster randomised controlled trial. Inform Prim Care 2009; 17 (04) 231-240. PubMed PMID: 20359401.
• 34 Vishwanath A, Singh SR, Winkelstein P. The impact of electronic medical record systems on outpatient workflows: a longitudinal evaluation of its workflow effects. Int J Med Inform 2010; 79 (11) 778-791. doi: 10.1016/j.ijmedinf.2010.09.006. PubMed PMID: 20947415.
• 35 Shortliffe E, Cimino J. editors. Biomedical informatics: Computer applications in health care and biomedicine. Third ed Springer; 2006
• 36 Warner HR, Toronto AF, Veasey LG. et al. A mathematical approach to medical diagnosis: application to congenital heart disease. 1961. MD computing: computers in medical practice 1992; 9 (01) 43-50. PubMed PMID: 1556914.
• 37 Shortliffe EH, Davis R, Axline SG. et al. Computer-based consultations in clinical therapeutics: explanation and rule acquisition capabilities of the MYCIN system. Computers and biomedical research, an international journal 1975; 8 (04) 303-320. PubMed PMID: 1157471.
• 38 Miller RA. A history of the INTERNIST-1 and Quick Medical Reference (QMR) computer-assisted diagnosis projects, with lessons learned. Yearbook of medical informatics 2010: 121–136. PubMed PMID: 20938584.
• 39 Sirajuddin AM, Osheroff JA, Sittig DF. et al. Implementation pearls from a new guidebook on improving medication use and outcomes with clinical decision support. Effective CDS is essential for addressing healthcare performance improvement imperatives. Journal of healthcare information management: JHIM 2009; 23 (04) 38-45. PubMed PMID: 19894486; PubMed Central PMCID: PMCPMC3316472.
• 40 Ash JS, Sittig DF, Campbell EM. et al. Some unintended consequences of clinical decision support systems. AMIA Annu Symp Proc. 2007: 26-30. PubMed PMID: 18693791; PubMed Central PMCID: PMCPMC2813668.
• 41 Sittig DF, Wright A, Osheroff JA. et al. Grand challenges in clinical decision support. J Biomed Inform 2008; 41 (02) 387-392. doi: 10.1016/j.jbi.2007.09.003. PubMed PMID: 18029232; PubMed Central PMCID: PMC2660274.