Design and Implementation of Tabletop Cybersecurity Simulation for Health Informatics Graduate Students

Abstract

Background Experiential learning through simulation allows students to apply didactic knowledge to real-world situations. Tabletop simulation allows for the exploration of a variety of topics, including cybersecurity in health care. Due to its low frequency, yet high-risk nature, simulation is a perfect educational modality to practice responding to a cybersecurity attack. As such, the authors designed and executed a tabletop cybersecurity simulation consisting of a prebriefing, four rounds of injects detailing potential cybersecurity breaches that students must address, and structured debriefings that included input from cybersecurity content experts. This simulation was performed in 2018, 2019, 2022, and 2023, during graduate Health Informatics (HI) students' residential visits.

Objective The simulation allowed opportunities for HI students to apply knowledge of cybersecurity principles to an unfolding tabletop simulation containing injects of scenarios they may encounter in the real world.

Methods Survey data were used to assess the students' perceptions of the simulation. Topics assessed included overall satisfaction, teamwork and communication, and length of the event. Additionally, in 2022 and 2023, data were collected on psychological safety and whether to include them in future HI residential visits.

Results Eighty-eight graduate HI students took part in the cybersecurity simulation over four annual residential visits. Most students were satisfied with the event, found it valuable, and could see it impacting their future practice as informaticists. Additionally, students indicated high levels of psychological safety. Multiple students requested that additional simulations be incorporated into the curriculum.

Conclusion A tabletop cybersecurity simulation was utilized to allow HI students the ability to apply knowledge related to cybersecurity breaches to real-world examples. The simulation's best practices of prebriefing, psychological safety, and structured debriefing with expert feedback were emphasized in the simulation's design and implementation. Students found the simulation valuable and worth including in the curriculum.

Keywords

informatics education - cybersecurity - simulation - health care

Background and Significance

Background

Cybersecurity events in health care can bring clinical and administrative operations to a near, if not absolute, standstill, leading to delays or lapses in patient care and access to many of the essential elements of patient care.[1] [2] Many organizations have cybersecurity response plans in place. However, executing those plans during a cyber event may be met with delays, lapses in communication, and unintended impacts on clinical care. Simulation-based education has been shown to increase technical skills, knowledge levels, and self-confidence.[3] Specifically, simulation has been advocated for as a method of education regarding cybersecurity.[2] The purpose of this cybersecurity simulation was to allow graduate health informatics (HI) students the opportunity to apply knowledge of cybersecurity principles and best practices to a tabletop simulation containing injects of scenarios they may encounter in the real world, including phishing emails, malware, ransomware, systems and clinical operations impact, downtime, emergency operations planning, cybercrime, remediation, and downtime recovery.

Theoretical Framework and Literature Review

This simulation was grounded in Experiential Learning Theory (ELT), which emphasizes the central role that experience plays in the learning process. There are four constructs in ELT: concrete experience, reflection, abstract conceptualization, and active experimentation. The concrete experience provides a basis for the reflections, which are then assimilated into abstract concepts, and actively applied to subsequent experiences.[4]

Health care simulation is an experience that “creates a situation or environment to allow persons to experience a representation of a real event for the purpose of practice, learning, evaluation, testing.”[5] Tabletop simulations often consist of key personnel with assigned roles having discussion-based interactions often using specific scenarios in a prescribed manner.

Simulation in HI is not a new concept. The literature is replete with simulation efforts assessing student learning relative to Electronic Health Record (EHR)[6] [7] [8] [9] and simulation efforts with cybersecurity learning.[10] [11] [12] [13] However, there is little on cybersecurity simulation with graduate HI students.

Ensuring the security of EHR systems and patient information has become a particularly daunting challenge for health care organizations. At the peak of the coronavirus disease (COVID-19) pandemic in October 2020, the Department of Homeland Security issued an ominous warning indicating that threat actors were targeting U.S. Healthcare Systems with ransomware.[14] In 2023, the average cost of a data breach for a U.S. health care organization rose by nearly 30% to $10.93 million.[15] Cybersecurity attacks have emerged as one of the greatest risks to health care operations. Today's HI students must have a basic understanding of the risks associated with a cyberattack against their organization. This tabletop exercise immerses the student in a real-world scenario and highlights how leadership inside and outside of informatics will eventually be faced with tough decisions regarding cybersecurity preparation and response.

Simulation Design and Implementation

An in-person 2-hour tabletop cybersecurity simulation was designed using the Healthcare Simulation Standards of Best Practice.[16] [17] Content experts in HI, adult learning, cybersecurity, and simulation collaborated to develop objectives and discuss details of the unfolding scenarios. A simulation goal was for graduate students to apply didactic coursework to a cybersecurity breach. Preparatory coursework included cybersecurity fundamentals, information risk and decision-making, third-party risk management, protecting digital health information, and incident response. This background, coupled with applied industry examples, paved the way for the students to apply knowledge in simulation.

In the first 2 years of the simulation (2018 and 2019), it was facilitated by a forensics cybercrime expert, a practicing Chief Information Security Officer (CISO), and an expert in simulation. In 2022 and 2023, there was a focus on health care systems, therefore the simulation was facilitated by two practicing CISOs from different health systems and one expert in simulation. The graduate HI program did not have an in-person residential visit in 2020 or 2021 due to the COVID-19 pandemic.

The simulation occurred over a 2-hour session which began with a 15-minute prebrief and orientation, followed by four, 20-minute rounds of injects associated with a cybersecurity event, and concluded with a 25-minute overall cybersecurity simulation debriefing.

Prebriefing

During the prebriefing, facilitators outlined the purpose of the simulation and tenets associated with psychological safety including confidentiality, orientation to their health system operations, and normalizing uncertainty.

Simulation

Prior to the simulation, facilitators created a cyber response plan (CRP) and an event action plan (EAP) form ([Supplementary Appendix A]) for their simulated health system to guide students as they responded to injects. Students received the CRP and EAP at the beginning of the simulation. The simulation progressed through four rounds of injects, escalating with each round ([Table 1]).

Table 1 Overview of simulation

Prebrief and Orientation (15 minutes)
Prebrief	Facilitators reviewed the purpose and objectives of the simulation, basic assumptions, confidentiality, and flow of the simulation
Orientation	Facilitators described the fundamental operations of the simulated health system, the cyber response plan, and event action plans
Small group tasks	Learners oriented themselves to the simulated hospital operations, cyber response plan, and event action plans
Round 1 (20 minutes)
Case stem	IT was notified by automated alerts of several file shares containing viruses at a new clinical site. One user has reported a phishing email and admitted to clicking the link
Small group tasks	• Consider the information provided • Explore resources (policy, internet, content experts, etc.) • Complete Cybersecurity Event Action Plan • Choose a spokesperson for this round and be prepared to share your decisions
In-sim debrief	Facilitator and content experts led a brief in-sim debriefing after a 2-minute small group report out
Round 2 (20 minutes)
Case stem	A week later, IT received multiple calls stating the users could not log into Citrix desktop to access the EHR. “System Unavailable” is the only message conveyed. Several callers stated their computers are displaying a message stating “Your Files Have Been Encrypted”
Small group tasks	Same as Round 1
In-sim debrief	Facilitator and content experts led a brief in-sim debriefing after a 2-minute small group report out
Round 3 (20 minutes)
Case stem	Within hours, you realize 30 desktop computers have been encrypted with ransomware and the Citrix servers are no longer responding. Parts of the Citrix environment appear to be affected by the ransomware. The EHR and other clinical systems are unavailable
Small group tasks	Same as Round 1
In-sim debrief	Facilitator and content experts led a brief in-sim debriefing after a 2-minute small group report out
Round 4 (20 minutes)
Case stem	Next day, the affected systems were quarantined. Backups are being restored with an ETA of 72 hours. All units/clinics are in downtime procedures. The media has called asking for a statement and the FBI has called and wants to come on site
Small group tasks	Same as Round 1
Simulation Debrief, Evaluation, and Wrap-up (25 minutes)
Activities	Facilitator leads structured debriefing which promotes reflection on decisions and actions and explores the frames of the learners

Abbreviation: EHR, electronic health record; IT, information technology.

Structured Debriefing

Essential learning occurs during debriefing.[16] Therefore, facilitators were intentional with their structure and strategy. Each round consisted of a 20-minute simulation with a 10-minute structured debrief. Debriefing included reactions, a description of the current injects, a report from each team, and facilitator-guided reflection. Facilitators debriefed within each round to allow learners to reflect on events and decisions to that point and permit content experts to provide commentary on their plans thus far. For the final debriefing, the same structure was utilized, followed by a culminating reflection on how the students' perspectives changed throughout the simulation.

Evolution of the Simulation

Over the 4 years the simulation has been implemented, the structure has remained the same, though there have been changes to the content. Cybersecurity injects were modified based on learner feedback and content expert input. Originally, the simulation used an actual hospital CRP with extensive details that could not be taken off campus. This was overwhelming for the students. In response, the CISOs worked together to create a generic, streamlined CRP that was easier to interpret and apply throughout the simulation.

Objectives

The purpose of the cybersecurity simulation was to allow an opportunity for graduate HI students to apply knowledge of cybersecurity principles and best practices to a tabletop simulation containing injects they may encounter in the real world.

Methods

Students enrolled in the graduate HI program were included in the online survey and the in-person cybersecurity simulation. Because this simulation is conducted after the students have had the security course, all students are included and there are no exclusion criteria. As the cybersecurity simulation was being developed, the team was consistently checking with the American Medical Informatics Association foundational domains to ensure alignment and agreement (F5, F7).[18]

Our team utilized survey data assessing student perceptions of the cybersecurity simulation, as well as ascertaining beliefs of how they may apply lessons learned to future practice. All surveys were completed anonymously. The standard simulation survey is classified by the University of Alabama at Birmingham Institutional Review Board as exempt (identifier: IRB-120822005). The HI on-site visit survey is conducted as part of ongoing program quality improvement efforts and therefore does not require Institutional Review Board approval.

Standard Simulation Survey

A standardized survey is utilized for all simulation activities on campus. Students were asked questions related to their satisfaction with the event, with items addressing objectives, learning experience, debriefing, impact on future performance, and if they would recommend the simulation to others. Teamwork and communication were also assessed. Additionally, 2022 and 2023 participants were asked questions related to their perceived psychological safety, including if they felt they were able to make mistakes in the simulation without judgment, they felt comfortable checking with each other for questions, the value of fellow participants' skills and knowledge, and ability to bring up problems and tough issues. All these items were assessed on a 5-point Likert scale with 1 being “Strongly agree” and 5 being “Strongly disagree.” Finally, students had the opportunity to answer open-ended questions which included what they liked/learned about the experience, opportunities for improvement, and additional comments. Surveys were administered to the 2018 and 2019 participants via paper and electronically in 2022 and 2023 through Qualtrics. Survey completion was encouraged but not required, and participants could abstain from answering any question.

Health Informatics On-Site Visit Survey

As a routine part of the residential visit for all 4 years, students completed a survey in which they shared feedback on the visit. In 2022 and 2023, additional questions specific to the cybersecurity simulation were added.

Results

Standard Simulation Survey

A total of 88 HI graduate students took part in the cybersecurity simulation in 2018, 2019, 2022, and 2023. Sixty-six (75%) of the participants completed at least one of the surveys.

Results from the survey indicated that students were largely satisfied with the event, including objectives, value of the experience, debriefing, and impact on future practice. Further, 64 (96.9%) respondents indicated that they would recommend the event to others (see [Table 2]). Likewise, participants rated the teamwork and communication components of the simulation favorably (see [Table 3]). Students indicated they felt psychologically safe during the simulation, with respondents agreeing or strongly agreeing that they were able to bring up problems and make mistakes without repercussions (see [Table 4]). Finally, most students agreed that the length of the event was appropriate (see [Table 5]).

Table 2 Overall satisfaction with the event

Statement	M	SD	n
The objectives for this event were met	1.34	0.57	64
The learning experience was valuable	1.23	0.52	66
The debriefing and/or feedback was valuable	1.20	0.49	64
The experience will improve my performance in an actual clinic setting[a]	1.43	0.65	66
I would recommend this event to others	1.32	0.44	64

Abbreviation: SD, standard deviation.

Scoring rubric: 1 = Strongly agree, 2 = Agree, 3 = Neutral, 4 = Disagree, 5 = Strongly disagree.

^a Interpreted as a “professional setting” for the cybersecurity simulation.

Table 3 Teamwork and communication

Statement	M	SD	n
The teamwork/communication objective was met	1.36	0.57	65
My teamwork/communication skills improved because of this experience	1.65	0.77	66

Abbreviation: SD, standard deviation.

Scoring rubric: 1 = Strongly agree, 2 = Agree, 3 = Neutral, 4 = Disagree, 5 = Strongly disagree.

Table 4 Psychological safety

Statement	M	SD	n
In this simulation, you were able to make mistakes without it being held against you	1.15	0.37	13
Participants in this simulation feel comfortable checking with each other if they have questions about the right way to do something	1.15	0.37	13
Participants in this simulation value others' unique skills and knowledge	1.23	0.43	13
In this simulation, you are able to bring up problems and tough issues	1.07	0.27	13

Abbreviation: SD, standard deviation.

Scoring rubric: 1 = Strongly agree, 2 = Agree, 3 = Neutral, 4 = Disagree, 5 = Strongly disagree.

Table 5 Length of event

Statement	Disagree-too short (n)	Agree (n)	Disagree-too long (n)
The length of time for the event was appropriate	12	50	4
The length of time for debriefing and/or feedback was appropriate	8	57	0

Students' comments included they “enjoyed the teamwork experience of trying to figure out the urgency of an incident.” Another stated “I feel this is pertinent to my future career. It helped settle my fears regarding a career in information security in healthcare.” Participants had several suggestions for improvement including “hear[ing] more examples of real-life security incidents,” a desire to “talk about these topics in greater detail,” and exploring the “clinical approach to IT disasters.” Other comments included “I enjoyed the simulation and it brought awareness to security and procedures that are followed in a breach,” as well as suggestions for incorporating the simulation into other coursework that would allow for deeper exploration into cybersecurity.

Health Informatics On-site Visit Survey

A subset of students (n = 19; 21.5%) completed survey questions covering activities during the on-site visits in 2022 and 2023. Ninety-four percent of respondents to the question (n = 18) found the simulation to be informative and helpful, 100% of respondents (n = 15) thought the simulation took an adequate amount of time, and 76% of respondents (n = 13) felt it should be included in future visits. Positive feedback on this survey included “I think exploring more options for simulations in future residential visits would be great” and “informative and thought provoking.” Recommendations for improvement included a suggestion for “more preparation and review” prior to the simulation and providing “clarification of different expectations for each level based on real life application.”

Discussion

Beginning in 2018, students in the graduate HI program participated in a tabletop cybersecurity simulation that focused on navigating various cybersecurity breaches and implementing procedures according to the CRP. Consistent with simulation best practices, cybersecurity experts were available to help students during the simulation and in debriefing.[19] Students felt the simulation and debriefing were valuable, with the potential to improve their future practice as health informaticists, yet there was a sense that the experience would not improve their performance in a live setting. Literature suggests that students do not always appreciate what they have learned in class until they need to apply the learning in real-world experiences. Anecdotally, we experience this with our 1-year postgraduate survey results when compared with our on-graduation survey.[20] Overwhelmingly, students indicated they would recommend the event to others as well as inclusion of the event in future site visits. Students shared that their teamwork and communication abilities improved, vital skills in today's workforce.

With respect to psychological safety, students felt this simulation provided a safe space to have difficult conversations and even make mistakes, a critical component of learning.

Simulation is an innovative approach for integrating theory and practice, allowing students to apply knowledge gained in the didactic setting to real-world problems they may face in the dynamic world in which they will practice. The authors feel strongly that the use of simulation as a teaching modality provides rich opportunities for students to practice in a safe environment and reflect upon their decision-making, furthering their preparation for their careers. Although much health care simulation is aimed at improving clinicians' communication, clinical skills, and teamwork, there are opportunities for academic programs in informatics to develop simulations that require few resources yet make a high impact. Authors have included the EAP ([Supplementary Appendix A]), along with a detailed overview of the simulation ([Table 1]), with the hopes that institutions may implement a similar cybersecurity simulation with their learners.

While the simulation aimed to mimic several types of cybersecurity threats, it by no means covered the full breadth and impact of a cybersecurity breach. Rather, it exposed students to situations they may not encounter in classroom learning and provided in-depth reflection and dialogue with content experts on how to navigate these challenges. Further limitations of the assessment of this simulation include the small sample size, limiting our ability to interpret quantitative data beyond practical utility, and program improvement. Findings cannot be generalized beyond this simulation and an academic context. Finally, data were self-reported by students which introduces the possibility for social desirability bias.[21]

Conclusion

Experiential learning strengthens students' learning by allowing them to apply knowledge to real-world situations in a safe and low-stakes environment. Simulation aligns with the assumptions of andragogy, integrates with the constructs of ELT, and promotes skills that employers value. A simulation such as the one described here has applicability to other levels of adult learning, such as undergraduate HI. Moreover, while the focus was on using simulation to prepare graduate HI students for practice, there is value in including our graduate health administration students in future simulation exercises. Making this an interprofessional simulation would provide an opportunity to practice communication between professions and clarify responsibilities within their scope of practice.

Clinical Relevance Statement

Cybersecurity breaches in health care environments are low-frequency, high-risk events that require teamwork, decisive action, and effective communication. Simulation allows informaticists to practice addressing these breaches, utilizing resources such as a CRP and EAP, in a low-stakes environment before applying these skills to real-world events. A tabletop simulation aimed at developing knowledge, skills, and attitudes related to cybersecurity attacks can better prepare HI students for problems they will have to address as professionals.

Multiple-Choice Questions

1. What is the purpose of health care simulation?

   • Shame and blame

   • Practice, learning, evaluation, and testing

   • Creating environments where the learners feel tricked

   • Introducing new content for the first time

   Correct Answer: The correct answer is option b. Simulation is a representation of a real event where learners can practice and learn. Simulation can also be used to evaluate and test learner's knowledge. Simulation should aim to be psychologically safe, so learners should not feel tricked or shamed. It is best practice to introduce content to learners prior to the simulation so that they can develop knowledge before applying it.

2. Which organization issued a warning indicating that U.S. Healthcare Systems were being targeted with ransomware?

   • Department of Homeland Security

   • World Health Organization

   • Food and Drug Administration

   • The Joint Commission

   Correct Answer: The correct answer is option a. In October 2020, the Department of Homeland Security issued a warning indicating that threat actors were targeting U.S. Healthcare Systems with ransomware.

Conflict of Interest

None declared.

Acknowledgments

The authors are thankful to the HI students who contributed to the ongoing improvement of the tabletop cybersecurity simulation.

Protection of Human Subjects

The HI on-site visit survey is conducted as part of ongoing program quality improvement efforts and therefore exempt from institutional review board approval.

The standard simulation survey is classified by the University of Alabama at Birmingham Institutional Review Board as exempt (approval no.: IRB-120822005).

• References

• 1 Alhuwail D, Al-Jafar E, Abdulsalam Y, AlDuaij S. Information security awareness and behaviors of health care professionals at public health care facilities. Appl Clin Inform 2021; 12 (04) 924-932
• 2 Niki O, Saira G, Arvind S, Mike D. Cyber-attacks are a permanent and substantial threat to health systems: education must reflect that. Digit Health 2022; 8: 20 552076221104665
• 3 Alanazi A, Nicholson N, Thomas S. The use of simulation training to improve knowledge, Skills, and confidence among healthcare students: a systematic review. Internet J Allied Health Sci Pract 2017; 15: Article 2
• 4 Kolb D, Boyatzis R, Mainemelis C. Perspectives on Thinking, Learning, and Cognitive Styles. (Sternberg RJ, Zhang L-F, eds.). Routledge; 2014
• 5 Downing D, Chang TP, Robertson JM. et al. Healthcare Simulation Dictionary. 2nd ed.. (Lioce L, Lopreiato J, eds.). Agency for Healthcare Research and Quality; 2020
• 6 Choi J, Bove LA, Tarte V, Choi WJ. Impact of simulated electronic health records on informatics competency of students in informatics course. Healthc Inform Res 2021; 27 (01) 67-72
• 7 Chan KG, Pawi S, Ong MF, Kowitlawakul Y, Goy SC. Simulated electronic health documentation: A cross-sectional exploration of factors influencing nursing students' intention to use. Nurse Educ Pract 2020; 48: 102864
• 8 Everett-Thomas R, Joseph L, Trujillo G. Using virtual simulation and electronic health records to assess student nurses' documentation and critical thinking skills. Nurse Educ Today 2021; 99: 104770
• 9 Repsha C, Morse B, Lee SE, Katz J, Burrows E, Teates J. Use of a simulated electronic health record to support nursing student informatics knowledge and skills. CIN: Computers, Informatics. Comput Inform Nurs 2020; 38 (02) 55-59
• 10 Winters DM, McDonald JK, Hansen DL. et al. The Playable Case Study: an Online Simulation for Skill and Attitudinal Learning. Faculty Publications; 2020
• 11 Karagiannis S, Magkos E. Engaging students in basic cybersecurity concepts using digital game-based learning: computer games as virtual learning environments. In Advances in Core Computer Science-based Technologies. 2021: 55-81
• 12 Scherb C, Heitz LB, Grimberg F, Grieder H, Maurer M. A cyber attack simulation for teaching cybersecurity. Paper presented at: Society 5.0 Integrating Digital World and Real World to Solve Challenges in Business and Society; June 2023.
• 13 Maggio LA, Dameff C, Kanter SL, Woods B, Tully J. Cybersecurity challenges and the academic health center: an interactive tabletop simulation for executives. Acad Med 2021; 96 (06) 850-853
• 14 Joint Cybersecurity Advisory. Ransomware activity targeting the healthcare and public health sector. 2020. Accessed April 1, 2024 at: https://www.cisa.gov/news-events/cybersecurity-advisories/aa20-302a
• 15 Alder S. IBM: Average Cost of a Healthcare Data Breach Increases to Almost $11 Million. The HIPAA Journal. Accessed April 1, 2024 at: The updated url for question 13 is: https://www.hipaajournal.com/2023-cost-healthcare-data-breach/
• 16 Decker S, Alinier G, Crawford SB, Gordon RM, Jenkins D, Wilson C. Healthcare Simulation Standards of Best Practice™: the debriefing process. Clin Simul Nurs 2021; 58: 27-32
• 17 McDermott DS, Ludlow J, Horsley E, Meakim C. Healthcare Simulation Standards of Best Practice™: Prebriefing: preparation and briefing. Clin Simul Nurs 2021; 58: 9-13
• 18 Valenta AL, Berner ES, Boren SA. et al. AMIA Board White Paper: AMIA 2017 core competencies for applied health informatics education at the master's degree level. J Am Med Inform Assoc 2018; 25 (12) 1657-1668
• 19 Watts PI, McDermott DS, Alinier G. et al. Healthcare simulation standards of best practice™: simulation design. Clin Simul Nurs 2021; 58: 14-21
• 20 Brophy J. Developing students' appreciation for what is taught in school. Educ Psychol 2008; 43 (03) 132-141
• 21 Grimm P. Social desirability bias. In: Wiley International Encyclopedia of Marketing. Wiley; 2010

Publication History

Received: 30 May 2024

Accepted: 13 August 2024

Article published online:
06 November 2024