A Student-Led Clinical Informatics Enrichment Course for Medical Students

Introduction

Clinical informatics leverages data to integrate analytics and clinical decision-making with aims to improve the quality, safety, and cost effectiveness of care delivery.[1] Numerous examples of clinical decision support systems (CDSS) have been shown to aid clinicians in diagnosis, outcome prediction, and disease management. Examples include but are not limited to infectious disease diagnosis,[2] thrombosis prophylaxis,[3] perioperative pediatric management,[4] and neurosurgical outcome prediction.[5] Such systems have been shown to benefit patient safety by reducing the incidence of adverse events such as medication errors,[6] [7] [8] care quality by ensuring adherence to established treatment guidelines, and cost reduction[9] by automating processes and decreasing resource waste.[10] [11]

Clinical informatics is a formally recognized medical subspecialty[12] [13] with a multitude of applications and career opportunities as information technology becomes more intertwined with modern practices of medicine.[14] [15] Significant strides have been taken to integrate core clinical informatics competencies into medical education,[16] [17] [18] as studies have shown medical students desire both broad training in informatics and opportunities for career exploration.[19] [20] Examples include lecture-based online electives,[21] credit-based introductory medical informatics courses,[22] one-on-one faculty mentorship programs,[23] and isolated problem-based learning exercises.[24] While these examples demonstrate a growing emphasis on informatics education, they are limited with respect to number of participants and cohesive, clinically applicable skill-building that go beyond basic acquaintance of core concepts. To address this need, more integrated and contemporary informatics training seminars have recently emerged at institutions around the United States (i.e., New York University Langone Health,[25] University of California at San Francisco,[26] Oregon Health & Science University,[27] and Duke[28]). In this spirit, the authors—four second and third year medical students—designed and delivered a 12-week informatics enrichment elective for first and second year medical students at our institution. Our goal was to introduce clinically relevant informatics concepts and foster development of technical skills, so that participants could apply informatics into research projects and during their time on the wards. We welcome medical students with similar goals to reach out for additional information.

The Challenges

We identified three challenges of introducing a clinical informatics curriculum in medical school as follows: (1) defining clinically applicable curriculum content for varied student needs, (2) supporting inclusivity for diverse student experiences and abilities, and (3) maintaining student engagement and minimizing attrition. After 2 years of piloting our course, the following reflection relays our approach to and learnings from tailoring content to first and second year medical students, who may not yet have deep medical knowledge or technical experience. We offer a few recommendations for others wishing to form similar educational initiatives.

We faced the challenge of tailoring a curriculum for broad student goals and needs. Clinical informatics encompasses a breadth of topics, each with varying relevance for medical students.[16] For example, every physician will interact with electronic health record (EHR) systems and require skills to efficiently navigate, correctly update, and effectively synthesize information. As reflected by accreditation standards to increase resident involvement in quality improvement, medical trainees are expected to engage with and improve patient care quality and safety.[29] As such, the role that informatics can play in monitoring and delivering quality metrics can be appreciated by all trainees. In terms of depth of knowledge, clinicians need not understand the mechanics of advanced statistical tests or be proficient in programming, but it is imperative that they understand how to apply decision support in care and how to interpret common statistical results from clinical studies and recognize limitations before applying results to patient care.[30] Similarly, while advanced gene sequencing technologies and machine learning (ML) algorithms are beyond the scope of medical practice, clinicians should be equipped for the advent of genomics and personalized medicine by understanding how decision support tools can utilize these technologies to transform individual patient care.[31]

Needs Assessment

Prior to the elective's start, we surveyed our peers to understand their goals and topic interests. Participants were eager to learn skills for data capture (by chart review or database query), dataset organization and cleaning, ML, and use of basic statistical tools (Stata/R/Matlab). Additionally, students voiced interest in learning about the EHR structure and workflow optimization for more streamlined patient care encounters. We chose to offer lectures on introductory concepts, informal sessions to introduce career opportunities, and workshops intended to develop technical skills and showcase applications for clinical practice and research endeavors ([Table 1]). Other notable parts of our curriculum included hands-on walk-through tutorials on implementing an ML workflow to predict office visit no-shows, navigating Epic during clerkships, customizing Epic SmartPhrases, cleaning clinical datasets using R, analyzing bioinformatics sequencing data using R, informatician faculty panels, industry speakers from leading digital health companies, and submitting institutional EHR data requests. After the elective's first iteration, certain didactics were modified in collaboration with leadership from our clinical informatics department to reduce redundancies and selected the most clinically relevant topics. Prior attempts at curriculum modifications through interdepartmental collaboration have shown to foster the development of new innovation.[32] After each weekly session, students were also asked to complete meeting-specific feedback forms which allowed us to iteratively adapt the elective's curriculum throughout its progression. Through collective decision-making with students and informatics experts, and keen identification of clinically relevant topics, we were able to design a curriculum at the forefront of students' interests and objectives.

Table 1 Clinical Informatics in Medicine curriculum with session offerings in 2020 and 2021 (n = 42 students; 2020: 24 students; 2021: 18 students)

Session	Hours	Type	Attendance (%)	Description
Introduction to clinical informatics	1	Didactic discussion	100	Overview of data architecture, ontologies, and career opportunities with institution-specific examples
Radiology informatics	1	Didactic discussion	78	Application-focused discussion of informatics in the field of radiology
Supervised learning	3	Experiential	76	Guided walk-through of ML workflow (from raw data to model validation) to design classifier to predict appointment no-shows using Python
Data architecture and ontologies	1	Didactic discussion	81	Advanced lecture on institution data warehouse and ontologies
Pathology informatics	1	Didactic discussion	67	Application-focused discussion of informatics in the field of pathology
Deep learning for image processing	3	Experiential	69	Guided walk-through of ML workflow to design deep learning model to classify images using Python
Industry discussion	1	Didactic discussion	70	Guest speaker from leading digital health company
Assessing model performance	1	Didactic discussion	59	Lecture on common statistical metrics
Leveraging data and ML for leadership decision making	1	Didactic discussion	63	Application-focused discussion of informatics applied to hospital administration-level decisions
Evidence based medicine and clinical decision support	1	Didactic discussion	89	Summary of the clinical decision support developmental life cycle based on evidence based management guidelines
Health Information Technology	1	Didactic discussion	89	Brief overview of the current state of health information including architectures, data networks, and data flow
Precision medicine	1	Didactic discussion	72	Clinical considerations and general themes of applying unique patient specific genes and environmental considerations to improve personalized care
Clinical data analysis	2	Experiential	78	Guided walkthrough of data extraction, cleaning, and confirming data quality using R and deidentified patient dataset
Introduction to single cell sequencing	2	Experiential	72	Guided walk-through of basic analysis for sample single cell sequencing dataset using R
Introduction to Epic EHR	1	Experiential	72	Tutorial on Epic workflow and guided walk-through on building SmartPhrases

Abbreviations: EHR, electronic health record; ML, machine learning.

Focus on Useful Skills

To reach a broad medical student audience, the cognitive load theory was applied as a framework to teach applicable skills during medical school or residency.[33] Hands-on learning, for example, should not rely solely on skeletonized ML models with hyperparameter fine-tuning or other similar exercises. While growing fields, such as ML, are exciting to trainees, students at our institution reported that this highly specialized exercise was not valuable, unless ML was their domain of research interest. In contrast, students reported that the workshop on data extraction, cleaning, and quality checking using R was applicable to student research and quality improvement projects which often involve retrospective clinical data analysis and chart review.[34] [35] Similarly, students appreciated the walk-through of Epic EHR, particularly on customizing templates and Smartphrases which taught skills they can put to use during clerkships.

Focus on Peer Education

As is innately present in any medical school class, our students had exceptionally diverse prior experiences. Students varied from past engineering majors to those with no prior background in programming or data analytics. This divide required consideration early on, as many interested students expressed hesitation during enrollment due to limited prior experience with informatics. The choice of student instructors was central to our mission of making our curriculum accessible to all medical students. Peer-teaching has been shown to result in student performance at least equivalent to that of faculty-led teaching with the additional benefit of cognitive congruence.[36] [37] Cognitive congruence is the shared understanding of prior knowledge and recent challenges from undergoing the same curriculum. In addition, social congruence between peer-teachers and students fosters an open and supportive learning environment for questions and discussion.[38] [39] As credentialed EHR trainers are expensive and often lack the clinical experience of a physician, institutions have also utilized medical students for physician training during new EHR implementations with comparable results, high student satisfaction, and significant cost savings.[40] Thus, technical workshop sessions were entirely led by students to ensure that concepts were simplified and taught in a language understandable by medical students. As upper classmen are uniquely positioned to share their approaches on facing common challenges experienced by medical students, we hoped peer-teaching would promote dissemination of curated, high-yield information for those just beginning their clinical experiences and forays into medical data.

Focus on Individualized Learning Styles

The rigorous medical school preclerkship curriculum necessitates students to be selective in their participation of extracurricular activities, requiring us to address challenges of attendance and engagement early on. Student interest was highest when content delivery was tailored to experiential learning through peer-to-peer hands-on workshops, live demonstrations, and question and answer (Q&A) sessions with industry and academic leaders ([Table 1]). Practically, sessions were recorded online, scheduled around major examinations, and designed to be all inclusive with respect to work and teaching material, with no time commitment requirement outside of weekly meeting times. The elective was graded “Pass”/”Fail” based on 75% attendance which allowed for self-directed learning.[33] Students with little technical background could focus on real-world applications, while those more technical students could explore more advanced topics. Emphasis on student learning styles and understanding of competing priorities were foundational to the development of our informatics curriculum.

Conclusion

Our elective is the first of its kind at our institution, one which brings the growing field of clinical informatics to the early medical education through student-led experiential learning. The lessons presented here were the collective thoughts and trials from 42 medical students who took the elective during a 2-year pilot study. Our emphasis on curriculum content, audience, and delivery methods led to a positive student experience. Also, 100% of respondents in a final feedback survey reported that they “would recommend the elective to future students.” Undoubtedly, the elective can be improved. As future classes and new student instructors take the reins in leading this elective, its content and delivery will continue to evolve and be refined.

Conflict of Interest

None declared.

Protection of Human and Animal Subjects

The University of Texas Southwestern Medical Center Institutional Review Board determined this project to be exempt from further review as this activity did not meet the definition of research.

• References

• 1 Kim GR, Lehmann CU. In search of dialogue and discourse in applied clinical informatics. Appl Clin Inform 2009; 0 (01) 1-7
• 2 Peiffer-Smadja N, Rawson TM, Ahmad R. et al. Machine learning for clinical decision support in infectious diseases: a narrative review of current applications. Clin Microbiol Infect 2020; 26 (05) 584-595
• 3 Lau BD, Haider AH, Streiff MB. et al. Eliminating health care disparities with mandatory clinical decision support: the venous thromboembolism (VTE) example. Med Care 2015; 53 (01) 18-24
• 4 Wang E, Brenn BR, Matava CT. State of the art in clinical decision support applications in pediatric perioperative medicine. Curr Opin Anaesthesiol 2020; 33 (03) 388-394
• 5 Buchlak QD, Esmaili N, Leveque JC. et al. Machine learning applications to clinical decision support in neurosurgery: an artificial intelligence augmented systematic review. Neurosurg Rev 2020; 43 (05) 1235-1253
• 6 Miller MR, Clark JS, Lehmann CU. Computer based medication error reporting: insights and implications. Qual Saf Health Care 2006; 15 (03) 208-213
• 7 Kim GR, Chen AR, Arceci RJ. et al. Error reduction in pediatric chemotherapy: computerized order entry and failure modes and effects analysis. Arch Pediatr Adolesc Med 2006; 160 (05) 495-498
• 8 Lehmann CU, Conner KG, Cox JM. Preventing provider errors: online total parenteral nutrition calculator. Pediatrics 2004; 113 (04) 748-753
• 9 Stenner SP, Chakravarthy R, Johnson KB. et al. ePrescribing: reducing costs through in-class therapeutic interchange. Appl Clin Inform 2016; 7 (04) 1168-1181
• 10 Sick AC, Lehmann CU, Tamma PD, Lee CK, Agwu AL. Sustained savings from a longitudinal cost analysis of an internet-based preapproval antimicrobial stewardship program. Infect Control Hosp Epidemiol 2013; 34 (06) 573-580
• 11 Sutton RT, Pincock D, Baumgart DC, Sadowski DC, Fedorak RN, Kroeker KI. An overview of clinical decision support systems: benefits, risks, and strategies for success. NPJ Digit Med 2020; 3: 17
• 12 Detmer DE, Shortliffe EH. Clinical informatics: prospects for a new medical subspecialty. JAMA 2014; 311 (20) 2067-2068
• 13 Lehmann CU, Gundlapalli AV, Williamson JJ. et al. Five years of clinical informatics board certification for physicians in the United States of America. Yearb Med Inform 2018; 27 (01) 237-242
• 14 Ohno-Machado L. Careers in informatics: a diversity of options with an abundance of jobs. J Am Med Inform Assoc 2012; 19 (06) 919
• 15 Kannry J, Sengstack P, Thyvalikakath TP. et al. The Chief Clinical Informatics Officer (CCIO): AMIA task force report on CCIO knowledge, education, and skillset requirements. Appl Clin Inform 2016; 7 (01) 143-176
• 16 Hersh WR, Gorman PN, Biagioli FE, Mohan V, Gold JA, Mejicano GC. Beyond information retrieval and electronic health record use: competencies in clinical informatics for medical education. Adv Med Educ Pract 2014; 5: 205-212
• 17 Kaufman DM, Jennett PA. Preparing our future physicians: integrating medical informatics into the undergraduate medical education curriculum. Stud Health Technol Inform 1997; 39: 543-546
• 18 Otto A, Kushniruk A. Incorporation of medical informatics and information technology as core components of undergraduate medical education - time for change!. Stud Health Technol Inform 2009; 143: 62-67
• 19 Briscoe GW, Fore Arcand LG, Lin T, Johnson J, Rai A, Kollins K. Students' and residents' perceptions regarding technology in medical training. Acad Psychiatry 2006; 30 (06) 470-479
• 20 Beaudoin DE, Richardson SJ, Sheng X, Mitchell JA. Medical students' perspectives on biomedical informatics learning objectives. Int J Med Educ 2013; 4: 1-8
• 21 Burnette MH, De Groote SL, Dorsch JL. Medical informatics in the curriculum: development and delivery of an online elective. J Med Libr Assoc 2012; 100 (01) 61-63
• 22 Feldman SS, Hersh W. Evaluating the AMIA-OHSU 10 × 10 program to train healthcare professionals in medical informatics. AMIA Annu Symp Proc 2008; 2008: 182-186
• 23 Green EP, Borkan JM, Pross SH. et al. Encouraging scholarship: medical school programs to promote student inquiry beyond the traditional medical curriculum. Acad Med 2010; 85 (03) 409-418
• 24 Burgun A, Darmoni S, Duff FL, Wéber J. Problem-based learning in medical informatics for undergraduate medical students: an experiment in two medical schools. Int J Med Inform 2006; 75 (05) 396-402
• 25 Health NL. Introduction to clinical informatics elective. Accessed December 26, 2021 at: https://med.nyu.edu/education/md-degree/registration-student-records/elective-catalog/interdepartmental/introduction-to-clinical-informatics
• 26 Clinical informatics and data science pathway. Accessed December 26, 2021 at: https://meded.ucsf.edu/residents-clinical-fellows/gme-educational-professional-development-opportunities/gme-pathways/clinical-informatics-and-data-science-pathway#Curriculum-Overview
• 27 Hersh W. Clinical informatics medical student elective. Accessed December 26, 2021 at: https://dmice.ohsu.edu/hersh/minf705B709A.html
• 28 Informatics DCfH. Informatics elective for medical students. Accessed December 26, 2021 at: https://dukeinformatics.org/education/health-informatics-programs-available-at-duke/informatics-for-medical-students/
• 29 Smith SR, Bakshi R. Promoting resident involvement in quality improvement initiatives through faculty involvement and curriculum. J Grad Med Educ 2015; 7 (01) 119-120
• 30 McCoy LG, Nagaraj S, Morgado F, Harish V, Das S, Celi LA. What do medical students actually need to know about artificial intelligence?. NPJ Digit Med 2020; 3: 86
• 31 Williams MS, Taylor CO, Walton NA. et al. Genomic information for clinicians in the electronic health record: lessons learned from the clinical genome resource project and the electronic medical records and genomics network. Front Genet 2019; 10: 1059
• 32 Valerius J, Mohan V, Doctor D, Hersh W. Collaboration leads to enhanced curriculum. Appl Clin Inform 2015; 6 (01) 200-209
• 33 Zackoff MW, Real FJ, Abramson EL, Li ST, Klein MD, Gusic ME. Enhancing educational scholarship through conceptual frameworks: a challenge and roadmap for medical educators. Acad Pediatr 2019; 19 (02) 135-141
• 34 Bleackley J, Kim SYR. The merit and agony of retrospective chart reviews: a medical student's perspective. BC Med J 2013; 55 (08) 374-375
• 35 Worster A, Haines T. Advanced statistics: understanding medical record review (MRR) studies. Acad Emerg Med 2004; 11 (02) 187-192
• 36 Benè KL, Bergus G. When learners become teachers: a review of peer teaching in medical student education. Fam Med 2014; 46 (10) 783-787
• 37 Yu TC, Wilson NC, Singh PP, Lemanu DP, Hawken SJ, Hill AG. Medical students-as-teachers: a systematic review of peer-assisted teaching during medical school. Adv Med Educ Pract 2011; 2: 157-172
• 38 Ten Cate O, Durning S. Peer teaching in medical education: twelve reasons to move from theory to practice. Med Teach 2007; 29 (06) 591-599
• 39 Omar F, Zaheer M, Ahmed M. Effectiveness of peer teaching in medical education: medical student's perspective. Adv Med Educ Pract 2018; 9: 199-201
• 40 Nelson AJ, Nelson SV, Linn AM, Raw LE, Kildea HB, Tonkin AL. Tomorrow's educators … today? Implementing near-peer teaching for medical students. Med Teach 2013; 35 (02) 156-159

Publication History

Received: 03 January 2022

Accepted: 21 January 2022

Article published online:
02 March 2022