Appl Clin Inform 2016; 07(01): 128-142
DOI: 10.4338/ACI-2015-08-RA-0108
Research Article
Schattauer GmbH

Usability Evaluation of a Clinical Decision Support System for Geriatric ED Pain Treatment

Nicholas Genes
1  Department of Emergency Medicine, Icahn School of Medicine at Mount Sinai, New York, NY
,
Min Soon Kim
2  Department of Health Management & Informatics, University of Missouri School of Medicine, Columbia, MO
3  Informatics Institute, University of Missouri, Columbia, MO
,
Frederick L. Thum
1  Department of Emergency Medicine, Icahn School of Medicine at Mount Sinai, New York, NY
,
Laura Rivera
1  Department of Emergency Medicine, Icahn School of Medicine at Mount Sinai, New York, NY
,
Rosemary Beato
1  Department of Emergency Medicine, Icahn School of Medicine at Mount Sinai, New York, NY
,
Carolyn Song
1  Department of Emergency Medicine, Icahn School of Medicine at Mount Sinai, New York, NY
,
Jared Soriano
4  Information Technology, Mount Sinai Health System, New York, NY
,
Joseph Kannry
4  Information Technology, Mount Sinai Health System, New York, NY
5  Division of General Internal Medicine, Icahn School of Medicine at Mount Sinai, New York, NY
,
Kevin Baumlin
1  Department of Emergency Medicine, Icahn School of Medicine at Mount Sinai, New York, NY
,
Ula Hwang
6  Brookdale Department of Geriatrics and Palliative Medicine, Icahn School of Medicine at Mount Sinai, New York, NY
7  Geriatric Research, Education and Clinical Center, James J Peters VAMC, Bronx, NY
› Author Affiliations
Further Information

Publication History

received: 21 September 2015

accepted: 05 January 2016

Publication Date:
16 December 2017 (online)

Summary

Background

Older adults are at risk for inadequate emergency department (ED) pain care. Unrelieved acute pain is associated with poor outcomes. Clinical decision support systems (CDSS) hold promise to improve patient care, but CDSS quality varies widely, particularly when usability evaluation is not employed.

Objective

To conduct an iterative usability and redesign process of a novel geriatric abdominal pain care CDSS. We hypothesized this process would result in the creation of more usable and favorable pain care interventions.

Methods

Thirteen emergency physicians familiar with the Electronic Health Record (EHR) in use at the study site were recruited. Over a 10-week period, 17 1-hour usability test sessions were conducted across 3 rounds of testing. Participants were given 3 patient scenarios and provided simulated clinical care using the EHR, while interacting with the CDSS interventions. Quantitative System Usability Scores (SUS), favorability scores and qualitative narrative feedback were collected for each session. Using a multi-step review process by an interdisciplinary team, positive and negative usability issues in effectiveness, efficiency, and satisfaction were considered, prioritized and incorporated in the iterative redesign process of the CDSS. Video analysis was used to determine the appropriateness of the CDS appearances during simulated clinical care.

Results

Over the 3 rounds of usability evaluations and subsequent redesign processes, mean SUS progressively improved from 74.8 to 81.2 to 88.9; mean favorability scores improved from 3.23 to 4.29 (1 worst, 5 best). Video analysis revealed that, in the course of the iterative redesign processes, rates of physicians’ acknowledgment of CDS interventions increased, however most rates of desired actions by physicians (such as more frequent pain score updates) decreased.

Conclusion

The iterative usability redesign process was instrumental in improving the usability of the CDSS; if implemented in practice, it could improve geriatric pain care. The usability evaluation process led to improved acknowledgement and favorability. Incorporating usability testing when designing CDSS interventions for studies may be effective to enhance clinician use.