Development and Evaluation of Offsite Repository for Clinical Assets, a Resilient Solution for Order Set Access during EHR Downtimes

Abstract

Background

Clinical decision support systems (CDSS) are central to modern health care, but their effectiveness is compromised during system downtimes, which affect 96% of health care organizations. During these failures, clinicians lose access to critical decision-making tools like order sets, increasing the risk of medical errors. Traditional downtime solutions, such as paper-based protocols, are often impractical and difficult to maintain.

Objectives

This study introduces and evaluates Offsite Repository for Clinical Assets (ORCA), a resilient web-based solution designed to maintain access to electronic health record (EHR) order sets during system failures. We assessed its usability and effectiveness as a downtime decision support tool across various clinical settings.

Methods

ORCA was developed based on an analysis of previous downtime incidents, replicating essential order set functionality while ensuring offsite accessibility. We conducted usability testing with 16 clinicians from diverse specialties, using structured tasks and think-aloud protocols. User feedback was collected through the Usability Metric for User Experience (UMUX) questionnaire and thematic analysis of interview transcripts.

Results

ORCA demonstrated strong usability (mean UMUX score: 86.2). Thematic analysis revealed key implementation challenges: system limitations, workflow integration, and interface navigation. Users valued ORCA's familiar interface and offsite accessibility but identified critical gaps in dynamic decision support capabilities.

Conclusion

ORCA represents a viable approach to maintaining basic clinical decision support (CDS) during downtimes. However, significant challenges remain in replicating dynamic CDS features and ensuring effective integration with existing downtime procedures. These findings inform future development of resilient CDSS and highlight the importance of planned fallback pathways in clinical systems.

Keywords

EHRs and systems - clinical decision support - human–computer interaction

Protection of Human and Animal Subjects

This study was exempt from review.

Finding

None.

Publication History

Received: 02 January 2025

Accepted: 23 May 2025

Accepted Manuscript online:
26 May 2025

Article published online:
17 October 2025

• References

• 1 Larsen EP, Rao AH, Sasangohar F. Understanding the scope of downtime threats: A scoping review of downtime-focused literature and news media. Health Informatics J 2020; 26 (04) 2660-2672
• 2 Henry J, Pylypchuk Y, Searcy T, Patel V. Adoption of electronic health record systems among US non-federal acute care hospitals: 2008–2015. ONC Data Brief. 2016; 35 (35) 2008-2015
• 3 Bischoff P. Ransomware attacks on US healthcare organizations cost $20.8bn. Accessed April 10, 2025 at: https://www.comparitech.com/studies/ransomware-studies/ransomware-attacks-hospitals-data/
• 4 Sittig DF, Gonzalez D, Singh H. Contingency planning for electronic health record-based care continuity: a survey of recommended practices. Int J Med Inform 2014; 83 (11) 797-804
• 5 Chen J, Wang Y, Magrabi F. Downtime in digital hospitals: an analysis of patterns and causes over 33 months. Integrating and Connecting Care. IOS Press; 2017: 14-20
• 6 Ponemon Institute. Sixth Annual Benchmark Study on Privacy & Security of Healthcare Data. 2016 . Accessed December 1, 2025 at: https://www.ponemon.org/local/upload/file/Sixth%20Annual%20Patient%20Privacy%20%26%20Data%20Security%20Report%20FINAL%206.pdf
• 7 Larsen E, Hoffman D, Rivera C, Kleiner BM, Wernz C, Ratwani RM. Continuing patient care during electronic health record downtime. Appl Clin Inform 2019; 10 (03) 495-504
• 8 Wang Y, Coiera E, Gallego B. et al. Measuring the effects of computer downtime on hospital pathology processes. J Biomed Inform 2016; 59: 308-315
• 9 Larsen E, Fong A, Wernz C, Ratwani RM. Implications of electronic health record downtime: an analysis of patient safety event reports. J Am Med Inform Assoc 2018; 25 (02) 187-191
• 10 Thayer JG, Franklin A, Miller JM, Grundmeier RW, Rogith D, Wright A. A scoping review of rule-based clinical decision support malfunctions. J Am Med Inform Assoc 2024; 31 (10) 2405-2413
• 11 Wright A, Hickman T-TT, McEvoy D. et al. Analysis of clinical decision support system malfunctions: a case series and survey. J Am Med Inform Assoc 2016; 23 (06) 1068-1076
• 12 Wright A, Sittig DF, Carpenter JD, Krall MA, Pang JE, Middleton B. Order sets in computerized physician order entry systems: an analysis of seven sites. AMIA Annu Symp Proc 2010; 2010: 892-896
• 13 Shepard A. Disaster recovery and the electronic health record. Nurs Adm Q 2017; 41 (02) 187-189
• 14 Office of the National Coordinator for Health Information Technology. The Safety Assurance Factors for EHR Resilience (SAFER). Self-Assessment Contingency Planning. General Instructions for the SAFER Assessment Guides. Accessed November 30, 2024 at: https://www.healthit.gov/sites/default/files/safer/guides/safer_contingency_planning.pdf
• 15 AMC PSO. Patient Safety Guidance for Electronic Health Record Downtime - Recommendations of the Electronic Health Record Downtime Task Force. Accessed November 30, 2024 at: https://flbog.sip.ufl.edu/wp-content/uploads/2019/11/AMC-PSO-EHR-Downtime.pdf
• 16 Sittig DF, Wright A, Simonaitis L. et al. The state of the art in clinical knowledge management: an inventory of tools and techniques. Int J Med Inform 2010; 79 (01) 44-57
• 17 Meta Platforms Inc. React: A JavaScript library for building user interfaces. Accessed September 27, 2024 at: https://reactjs.org/
• 18 Finstad K. The usability metric for user experience. Interact Comput 2010; 22 (05) 323-327
• 19 Berkman MI, Karahoca D. Re-assessing the usability metric for user experience (UMUX) scale. J Usability Stud 2016 ;11(3)
• 20 Dai S-C, Xiong A, Ku L-W. LLM-in-the-loop: Leveraging large language model for thematic analysis. arXiv preprint arXiv:2310.15100. 2023
• 21 Rao VN, Agarwal E, Dalal S, Calacci D, Monroy-Hernández A. QuaLLM: An LLM-based framework to extract quantitative insights from online forums. arXiv preprint arXiv:2405.05345. 2024
• 22 De Paoli S. Performing an inductive thematic analysis of semi-structured interviews with a large language model: An exploration and provocation on the limits of the approach. Soc Sci Comput Rev 2024; 42 (04) 997-1019
• 23 Lee VV, van der Lubbe SCC, Goh LH, Valderas JM. Harnessing ChatGPT for thematic analysis: Are we ready?. J Med Internet Res 2024; 26: e54974
• 24 Google DeepMind. Gemini 1.5 Flash [Large Language Model]. 2024 . Accessed May 29, 2025 at: https://deepmind.google/technologies/gemini/flash
• 25 OpenAI. GPT o1-preview [Large Language Model]. 2024 . Accessed May 29, 2025 at: https://openai.com/
• 26 Braun V, Clarke V. Using thematic analysis in psychology. Qual Res Psychol 2006; 3 (02) 77-101
• 27 Sauro J. Measuring Usability with the System Usability Scale (SUS). 2011 . Accessed November 30, 2024 at: https://measuringu.com/sus/
• 28 Sauro J, Lewis JR. Quantifying the user experience: Practical statistics for user research. Morgan Kaufmann; 2016
• 29 Lyon R, Jones A, Burke R, Baysari MT. What goes up, must come down: a state-of-the-art Electronic Health Record Downtime and Uptime Procedure in a Metropolitan Health setting. Appl Clin Inform 2023; 14 (03) 513-520