PDF herunterladen
Appl Clin Inform 2023; 14(02): 212-226
DOI: 10.1055/a-2006-4936
Research Article

Usability Testing of an Interoperable Computerized Clinical Decision Support Tool for Fall Risk Management in Primary Care

Kristen Shear
1   Department of Family, Community, and Health Systems Science, College of Nursing, University of Florida, Gainesville, Florida, United States
,
Hannah Rice
2   Department of General Internal Medicine and Primary Care, Brigham and Women's Hospital, Harvard Medical School, Boston, Massachusetts, United States
,
Pamela M. Garabedian
3   Department of Information Systems, Mass General Brigham, Somerville, Massachusetts, United States
,
Ragnhildur Bjarnadottir
1   Department of Family, Community, and Health Systems Science, College of Nursing, University of Florida, Gainesville, Florida, United States
,
Nancy Lathum
2   Department of General Internal Medicine and Primary Care, Brigham and Women's Hospital, Harvard Medical School, Boston, Massachusetts, United States
,
Ann L. Horgas
4   Department of Biobehavioral Nursing Science, College of Nursing, University of Florida, Gainesville, Florida, United States
,
Christopher A. Harle
5   Department of Health Outcomes and Biomedical Informatics, College of Medicine, University of Florida, Gainesville, Florida, United States
,
Patricia C. Dykes
2   Department of General Internal Medicine and Primary Care, Brigham and Women's Hospital, Harvard Medical School, Boston, Massachusetts, United States
,
Robert Lucero
6   UCLA School of Nursing, University of California Los Angeles, Los Angeles, California, United States
› Institutsangaben Funding The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: this work was supported by the Agency for Healthcare Research and Quality [grant number: U18HS027557].
› Weitere Informationen
Auch verfügbar auf
   Lizenzen und Reprints

Abstract

Background Falls are a widespread and persistent problem for community-dwelling older adults. Use of fall prevention guidelines in the primary care setting has been suboptimal. Interoperable computerized clinical decision support systems have the potential to increase engagement with fall risk management at scale. To support fall risk management across organizations, our team developed the ASPIRE tool for use in differing primary care clinics using interoperable standards.

Objectives Usability testing of ASPIRE was conducted to measure ease of access, overall usability, learnability, and acceptability prior to pilot .

Methods Participants were recruited using purposive sampling from two sites with different electronic health records and different clinical organizations. Formative testing rooted in user-centered design was followed by summative testing using a simulation approach. During summative testing participants used ASPIRE across two clinical scenarios and were randomized to determine which scenario they saw first. Single Ease Question and System Usability Scale were used in addition to analysis of recorded sessions in NVivo.

Results All 14 participants rated the usability of ASPIRE as above average based on usability benchmarks for the System Usability Scale metric. Time on task decreased significantly between the first and second scenarios indicating good learnability. However, acceptability data were more mixed with some recommendations being consistently accepted while others were adopted less frequently.

Conclusion This study described the usability testing of the ASPIRE system within two different organizations using different electronic health records. Overall, the system was rated well, and further pilot testing should be done to validate that these positive results translate into clinical practice. Due to its interoperable design, ASPIRE could be integrated into diverse organizations allowing a tailored implementation without the need to build a new system for each organization. This distinction makes ASPIRE well positioned to impact the challenge of falls at scale.

Keywords

usability testing - clinical decision support - falls - primary care

Protection of Human and Animal Subjects

IRB approval was received for protocol number: Site 1: 2020P002075; Site 2: CED000000426_.


Note

P.C.D. and R.L. contributed equally to the development of the manuscript as co–senior investigators.

The views expressed herein are those of the author(s) and do not reflect the official policy or position of Brooke Army Medical Center, the Department of Defense, or any agencies under the U.S. Government.




Publikationsverlauf

Eingereicht: 07. Juli 2022

Angenommen: 02. Januar 2023

Accepted Manuscript online:
04. Januar 2023

Artikel online veröffentlicht:
15. März 2023

© 2023. Thieme. All rights reserved.

Georg Thieme Verlag KG
Rüdigerstraße 14, 70469 Stuttgart, Germany

 

  • References

  • 1 Florence CS, Bergen G, Atherly A, Burns E, Stevens J, Drake C. Medical costs of fatal and nonfatal falls in older adults. J Am Geriatr Soc 2018; 66 (04) 693-698
    CrossrefPubMedSuche in Google Scholar
  • 2 Moreland B, Kakara R, Henry A. Trends in nonfatal falls and fall-related injuries among adults aged ≥65 years - United States, 2012-2018. MMWR Morb Mortal Wkly Rep 2020; 69 (27) 875-881
    CrossrefPubMedSuche in Google Scholar
  • 3 Clinical Decision Support. The Office of the National Coordinator for Health Information Technology. Accessed March 7, 2021 at: https://www.healthit.gov/topic/safety/clinical-decision-support
    PubMed
  • 4 Bryan C, Boren SA. The use and effectiveness of electronic clinical decision support tools in the ambulatory/primary care setting: a systematic review of the literature. Inform Prim Care 2008; 16 (02) 79-91
    PubMedSuche in Google Scholar
  • 5 Trinkley KE, Kroehl ME, Kahn MG. et al. Applying clinical decision support design best practices with the practical robust implementation and sustainability model versus reliance on commercially available clinical decision support tools: randomized controlled trial. JMIR Med Inform 2021; 9 (03) e24359
    CrossrefPubMedSuche in Google Scholar
  • 6 Taylor SF, Coogle CL, Cotter JJ, Welleford EA, Copolillo A. Community-dwelling older adults' adherence to environmental fall prevention recommendations. J Appl Gerontol 2019; 38 (06) 755-774
    CrossrefPubMedSuche in Google Scholar
  • 7 Assurance NCfQ. Fall risk management. National Committee for Quality Assurance. Accessed April 25, 2022 at: https://www.ncqa.org/hedis/measures/fall-risk-management/
    PubMed
  • 8 Howland J, Hackman H, Taylor A, O'Hara K, Liu J, Brusch J. Older adult fall prevention practices among primary care providers at accountable care organizations: a pilot study. PLoS One 2018; 13 (10) e0205279
    CrossrefPubMedSuche in Google Scholar
  • 9 Spears GV, Roth CP, Miake-Lye IM, Saliba D, Shekelle PG, Ganz DA. Redesign of an electronic clinical reminder to prevent falls in older adults. Med Care 2013; 51 (03) (3, suppl 1): S37-S43
    CrossrefPubMedSuche in Google Scholar
  • 10 Zheng MY, Suneja A, Chou AL, Arya M. Physician barriers to successful implementation of US Preventive Services Task Force routine HIV testing recommendations. J Int Assoc Provid AIDS Care 2014; 13 (03) 200-205
    CrossrefPubMedSuche in Google Scholar
  • 11 Hoskins KF, Tejeda S, Vijayasiri G. et al. A feasibility study of breast cancer genetic risk assessment in a federally qualified health center. Cancer 2018; 124 (18) 3733-3741
    CrossrefPubMedSuche in Google Scholar
  • 12 Kurth AE, Krist AH, Borsky AE. et al. U.S. Preventive Services Task Force methods to communicate and disseminate clinical preventive services recommendations. Am J Prev Med 2018; 54 (1S1): S81-S87
    CrossrefPubMedSuche in Google Scholar
  • 13 Braunstein ML. Health Informatics on FHIR: How HL7's New API is Transforming Healthcare. Cham: Springer; 2018: 292
    CrossrefSuche in Google Scholar
  • 14 Barnett ML, Bitton A, Souza J, Landon BE. Trends in outpatient care for medicare beneficiaries and implications for primary care, 2000 to 2019. Ann Intern Med 2021; 174 (12) 1658-1665
    CrossrefPubMedSuche in Google Scholar
  • 15 Sutton RT, Pincock D, Baumgart DC, Sadowski DC, Fedorak RN, Kroeker KI. An overview of clinical decision support systems: benefits, risks, and strategies for success. NPJ Digit Med 2020; 3: 17
    CrossrefPubMedSuche in Google Scholar
  • 16 Bright TJ, Wong A, Dhurjati R. et al. Effect of clinical decision-support systems: a systematic review. Ann Intern Med 2012; 157 (01) 29-43
    Suche in Google Scholar
  • 17 Nielsen J. Chapter 2 - What is usability? In: Nielsen J, ed. Usability Engineering. Burlington, MA: Morgan Kaufmann; 1993: 23-48
  • 18 Peute LW, Spithoven R, Bakker PJ, Jaspers MW. Usability studies on interactive health information systems; where do we stand?. Stud Health Technol Inform 2008; 136: 327-332
    PubMedSuche in Google Scholar
  • 19 Nielsen J, Mack RL. Usability Inspection Methods. Morristown, NJ: Wiley; 1994
    CrossrefSuche in Google Scholar
  • 20 Li AC, Kannry JL, Kushniruk A. et al. Integrating usability testing and think-aloud protocol analysis with “near-live” clinical simulations in evaluating clinical decision support. Int J Med Inform 2012; 81 (11) 761-772
    CrossrefPubMedSuche in Google Scholar
  • 21 Rice H, Garabedian PM, Shear K. et al. Clinical decision support for fall prevention: defining end-user needs. Appl Clin Inform 2022; 13 (03) 647-655
    Thieme ConnectPubMedSuche in Google Scholar
  • 22 Rice HG, Garabedian P, Shear K. et al. Computerized clinical decision support for fall prevention: defining end-user requirements for primary care staff and patients. Paper presented at: 2022 AMIA Informatics Summit; March 21–24, 2022; Chicago, IL, United States. Accessed January 18, 2023 at: https://knowledge.amia.org/75287-amia-1.4633888/t005-1.4635399/t005-1.4635400/2082-1.4635455/2082-1.4635456
    PubMed
  • 23 Grossman DC, Curry SJ, Owens DK. et al; US Preventive Services Task Force. Interventions to prevent falls in community-dwelling older adults: US Preventive Services Task Force recommendation statement. JAMA 2018; 319 (16) 1696-1704
    CrossrefPubMedSuche in Google Scholar
  • 24 Shubert TE, Smith ML, Jiang L, Ory MG. Disseminating the Otago Exercise Program in the United States: perceived and actual physical performance improvements from participants. J Appl Gerontol 2018; 37 (01) 79-98
    CrossrefPubMedSuche in Google Scholar
  • 25 De Vito Dabbs A, Myers BA, Mc Curry KR. et al. User-centered design and interactive health technologies for patients. Comput Inform Nurs 2009; 27 (03) 175-183
    CrossrefPubMedSuche in Google Scholar
  • 26 Sauro J, Lewis JR. Quantifying the User Experience: Practical Statistics for User Research. 2nd ed. Amsterdam: Elsevier; 2016: 350
    Suche in Google Scholar
  • 27 Faulkner L. Beyond the five-user assumption: benefits of increased sample sizes in usability testing. Behav Res Methods Instrum Comput 2003; 35 (03) 379-383
    CrossrefPubMedSuche in Google Scholar
  • 28 Albert W, Tullis TS. Chapter 5 - Self-reported metrics. In: Albert W, Tullis TS, eds. Measuring the User Experience. 3rd ed. Burlington, MA: Morgan Kaufmann; 2023: 109-151
    Suche in Google Scholar
  • 29 Lewis JR, Sauro J. The Factor Structure of the System Usability Scale. Berlin, Heidelberg: Springer; 2009: 94-103
    Suche in Google Scholar
  • 30 Borsci S, Federici S, Bacci S, Gnaldi M, Bartolucci F. Assessing user satisfaction in the era of user experience: comparison of the SUS, UMUX, and UMUX-LITE as a function of product experience. Int J Hum Comput Interact 2015; 31 (08) 484-495
    CrossrefPubMedSuche in Google Scholar
  • 31 Sekhon M, Cartwright M, Francis JJ. Acceptability of healthcare interventions: an overview of reviews and development of a theoretical framework. BMC Health Serv Res 2017; 17 (01) 88
    CrossrefPubMedSuche in Google Scholar
  • 32 Kawamoto K, Houlihan CA, Balas EA, Lobach DF. Improving clinical practice using clinical decision support systems: a systematic review of trials to identify features critical to success. BMJ 2005; 330 (7494): 765
    CrossrefPubMedSuche in Google Scholar
  • 33 Connect Repository CDS. Agency for Healthcare Research and Quality. Accessed April 29, 2022 at: https://cds.ahrq.gov/cdsconnect/repository
    PubMed
  • 34 Camacho PM, Petak SM, Binkley N. et al. American Association of Clinical Endocrinologists/American College of Endocrinology Clinical Practice Guidelines for the Diagnosis and Treatment of Postmenopausal Osteoporosis-2020 Update. Endocr Pract 2020; 26 (suppl 1): 1-46
    Suche in Google Scholar
  • 35 Weinfeld JM, Gorman PN. Primary care physician designation and response to clinical decision support reminders: a cross-sectional study. Appl Clin Inform 2016; 7 (02) 248-259
    Thieme ConnectPubMedSuche in Google Scholar