Student Perceptions of Instructional Ultrasound Videos as Preparation for a Practical Assessment

Abstract

Background

Learning ultrasound early in the medical school curriculum helps students to understand anatomy and pathology and to perform defined ultrasound standard views. Instructional videos are a potentially valuable tool for improving the process of learning ultrasound skills. It was the aim of the present study to investigate how students perceived instructional videos as a learning aid, compared to other learning opportunities, in preparation for an Objective Structured Clinical Examination (OSCE).

Materials and Methods

Eleven concise ultrasound videos were created and implemented in the 4^th year at the Medical University of Vienna. The videos illustrate the predefined examination process, image optimization, and nine standardized ultrasound views. The videos were available to be used in preparation for the practical ultrasound examination, which was part of the objective structured clinical examination. The students’ perceptions of the instructional videos and other learning methods were surveyed using an online questionnaire.

Results

In total, 445 of 640 students (69.5% of the cohort) used the instructional videos. Of those students, 134 (30%) answered the questionnaire. Of this group, 88.9% rated the instructional videos as very helpful (49.6% as extremely helpful). An ANOVA revealed a significant difference between various learning materials in terms of helpfulness. Post hoc analysis showed that instructional videos were perceived as the second most helpful learning material after “self-execution and feedback.”

Conclusion

The study revealed that students use instructional videos frequently and appreciate them as an extra tool for effective studying.

Key words

ultrasound methods & techniques - education - internet - technology assessment

Introduction

During the last several decades, ultrasound examinations have become established as an effective diagnostic tool in many medical domains [1] [2] [3]. Today, many different standardized examination procedures are in place, such as guidelines for gastrointestinal ultrasound [4] and emergency ultrasound [5] [6], which define the examination process and facilitate the replicability of results. In radiology, internal medicine, intensive care medicine, and many other disciplines, ultrasound examinations are part of daily clinical practice [7], in conjunction with taking a patient’s history or basic physical examinations. Given the significance of ultrasound examinations in modern clinical practice, it has been recommended that systematic training about how to perform clearly defined ultrasound standard views [8] [9] should be provided early in medical education, as part of the undergraduate curricula. Comprehensive, integrated ultrasound training has also been shown to increase medical students’ general understanding of anatomy, physiology, and pathology [10] [11]. However, at present, the practical skills needed to perform an ultrasound examination are taught in only a few medical undergraduate curricula [7] [12].

To address this, more and more medical universities are striving to teach students how to perform ultrasound examinations. Some of the skills related to performing such examinations, such as identifying and naming anatomical structures, can be taught in lectures and practiced using print or electronic learning materials. Other skills, such as ‘operation of the transducer’ and ‘monitoring and re-adjusting the ultrasound machine settings,’ require hands-on training.

While textbooks and handouts have been used for decades as learning materials, the value of instructional videos in supporting ultrasound training has been under-researched, as Woodworth et al. reported in 2014 [13]. With respect to other clinical skills, such as surgical skills [14], critical and intensive care skills [15], and anatomical skills [16] [17], instructional videos have proven to increase students’ subjective feelings of competence [18] [19]. Instructional videos designed to teach students how to perform an exam were investigated in a Korean nationwide survey where it was reported that a majority of the students felt that instructional videos were very useful in learning clinical skills [20]. There is a lack of data about how instructional videos are used in relation to other learning opportunities, such as written materials, demonstrations, and hands-on training. Furthermore, there is not much knowledge about how students use instructional videos and why they perceive them as useful when learning complex practical and cognitive skills.

Thus, it was the aim of the present study to investigate how students perceived instructional videos as a learning aid, compared to other learning opportunities, in preparation for an Objective Structured Clinical Examination (OSCE).

Materials and Methods

This study expanded the learning environment of an existing university course with established learning objectives. The course previously relied mainly on written materials, lectures delivered by teachers, and hands-on training. For the study, the course was enhanced by providing 11 concise instructional videos. In the videos, tutors demonstrated the steps needed to perform standard US views and to identify anatomical landmarks.

Nine of these videos demonstrated the examination process for obtaining predefined standard views ([Table 1]), as outlined in the written materials. One video demonstrated an entire examination procedure, and another one focused on tips about how to optimize ultrasound images ([Fig. 1]). During the production process, each video was reviewed several times by US experts to ensure state-of-the-art demonstration techniques ([Table 1]).

Fig. 1 Two exemplary screenshots of the instructional videos: a how to optimize the image; b example of a standard view (subcostal 4-chamber view)

Table 1 Standard views covered by the instructional videos.

Standard view	Demonstrated skills
Right kidney - longitudinal section	Find and identify the right kidney (sinus, parenchyma, mark pyramids, Morison’s pouch) liver, and psoas muscle
Lung - search for pleural effusion	Find and identify lung (containing B-lines), A-lines, pleura, ribs and rib shadow
Gallbladder - longitudinal section	Find and identify gallbladder, liver, stomach and intestines/gut
Hilum of the liver - CPC	Find and identify liver, VCI, portal vein, common bile duct, stomach and intestines
Vena cava inferior (VCI) - longitudinal section	Find and identify VCI, liver, pancreas, splenic vein, stomach, hepatic artery proper and intestines
Aorta - longitudinal section	Find and identify aorta, celiac trunk, superior mesenteric artery, splenic vein, liver, pancreas and intestines
Subcostal 4-chamber view	Find and identify right aspects of the heart, left aspects of the heart, pericardial sac, myocardium, lung (containing B-lines) and liver
Spleen - longitudinal section	Find and identify spleen, hilum, lung and intestines
Lung - search for pneumothorax	Find and identify pleura, A-lines and ribs

The students’ perception of the usefulness of the instructional videos, compared to the other learning materials, was investigated using an online questionnaire. Furthermore, there were questions regarding personal experiences with five different ultrasound learning materials ([Table 2]), including live demonstration, supervised hands-on training, self-directed hands-on training, written materials, and instructional videos.

Table 2 Usefulness of different learning materials when learning to perform ultrasound standard views (F=17.167, df=4, p<0.001).

Learning material	Mean (SD) of perceived usefulness (N=133)
Live demonstration	3.26 (0.834)
Supervised hands-on training	3.54 (0.734)
Written materials	2.93 (0.863)
Self-directed hands-on training	3.05 (1.160)
Instructional videos	3.37 (0.783)

Five questions asked students to rate the perceived usefulness of each type of learning material using a verbally and numerically anchored rating scale: not useful (0); slightly useful (1); somewhat useful (2); very useful (3); and extremely useful (4). Three questions asked students to choose up to two of the most useful learning materials with regard to mastering each of the skills (the ability to accomplish a correct examination process, to perform a correct standard view, and to identify anatomical landmarks).

Subjective learning experiences associated with the US instructional videos were collected by two open questions: “think about a situation when one of our ultrasound videos helped you learn; describe the situation and explain why the video was so useful,” and “how do we need to improve the videos to support your learning even better?”

Performing an ultrasound examination is a complex skill. Therefore, analyzing how students prefer to learn each of the three defined sub-skills (performing the correct examination process, performing a correct standard view, and identifying anatomical landmarks) is necessary to obtain further insight into the usefulness of learning materials. Students were asked to select their two preferred learning materials (live demonstration, hands-on training, written materials, instructional videos).

Prior to answering the questions about subjective learning experiences, the students were prompted to self-reflect on their US skills with regard to the three learning objectives based on the following question: “If I had to take the practical US exam right now, I am confident that I could meet the required performance standards? Strongly disagree (1), tend to disagree (2), maybe agree (3), agree (4), strongly agree (5).”

Throughout the US course (in January, in the 4^th year of the curriculum), the learning materials were introduced sequentially, starting with written materials, which were available prior to live demonstration, followed by supervised hands-on training (three weeks, 2 h/week, in January). The videos were introduced prior to the three-week preparatory period for self-directed hands-on-training (three weeks, in May), which took place under the supervision of student tutors. Mastery of the US course’s learning objectives was demonstrated in one station of the Objective Structured Clinical Examination (OSCE) (in June). To demonstrate their ability to perform a correct ultrasound examination as part of the OSCE, students had to perform one of nine randomly assigned standard views. All but one of the students passed the US station, 14.6% with a top score (grade 1), 56.1% with a grade 2, 24.5% with a grade 3, and 4.5% just passed the exam (grade 4).

The instructional ultrasound videos and the survey were advertised to all 640 students during the practical training period via e-mail, and through the 4^th-year students’ Facebook group.

Statistical evaluation

Data analyses were performed using IBM SPSS Statistic Version 22 (IBM Corporation, USA).

Descriptive statistics of means, standard deviations, frequencies, and percentages were calculated for variables of interest. To compare the subjectively perceived usefulness of particular learning materials and opportunities, repeated measures ANOVA and Bonferroni corrected post-hoc tests were used.

To compare the reported self-efficacy of the learning objectives, a Friedman test was performed.

P ≤ 0.05 was considered statistically significant.

Answers to open-ended questions were categorized according to their content to summarize what students perceived as subjectively useful.

Results

In total, 445 of 640 students (69.5%) used the instructional videos according to the tracked usage statistics. Of these 445 students, 134 (30.1%) responded to the questionnaire. In total, 57 male and 72 female students participated in the study. Five students did not report their gender.

Self-efficacy of the students with regard to the three learning objectives

Overall, 127 of 134 students who answered the questionnaire reported having watched the videos and answered questions about their self-efficacy. Overall, 110 (86.7%) of the students felt able to achieve at least two of three required learning objectives by the time they were asked.

122 (96.1%) of the students agreed or strongly agreed that they felt confident in their ability to achieve the learning objective “accomplishing the correct examination process”, while 102 (80.3%) felt confident “performing the correct standard view” and 101 (79.5%) felt confident “identifying the anatomical landmarks,” indicating a high level of perceived self-efficacy. Students felt significantly better prepared for the learning objective “accomplishing the correct examination process” (p<0.001).

Subjective experiences with five different ultrasound learning materials

Students who were preparing for the upcoming practical OSCE exam reported different perceptions of the usefulness of the US learning materials ([Table 2]).

118 students considered “supervised hands-on training” as at least “very useful.” There were 120 students who stated that “instructional videos” were “very useful,” and 92 students perceived written learning materials as “very useful,” indicating that students, in general, appreciated all of the provided learning materials ([Table 2], [Fig. 2]).

Fig. 2 Perceived usefulness of different learning materials: a instructional videos; b self-directed hands-on training; c written materials; d supervised hands-on training; and e live demonstration.

Post hoc comparison revealed that “supervised hands-on training” was, overall, perceived as the most valuable learning material followed by “instructional videos.” Compared to the other learning materials, written materials were perceived as the least useful ([Table 3]).

Table 3 Post hoc comparison (p-values) between different learning materials.

	Live demonstration	Supervised hands-on training	Self-directed hands-on training	Instructional videos	Written materials
Live demonstration		0.007	0.725	1	0.008
Supervised hands-on training	0.007		<0.001	0.68	<0.001
Self-directed hands-on training	0.725	<0.001		0.05	1
Instructional videos	1	0.68	0.05		<0.001
Written materials	0.008	<0.001	1	<0.001

Most useful learning materials for three ultrasound sub-skills

With regard to “performing the correct examination process,” students reported that they relied heavily on the instructional videos. 20.7% reported that the combination of instructional videos and written materials was preferred, and 17.8% reported instructional videos and hands-on training as the favorite materials for learning.

Students preferred hands-on training for learning to perform a correct standard view, accompanied by instructional videos (25.2%) or live demonstration (28.9%). For this learning objective, hands-on training alone was perceived as the optimal learning material for 13.3% of the students.

For identifying anatomical landmarks, written materials were preferred, in combination with instructional videos (25.2%), hands-on training (16.3%), or live demonstration (16.3%). Written materials alone were the optimal learning material for 9.6% of the students ([Table 4]).

Table 4 The two most useful learning materials to master the three ultrasound-related skills in absolute numbers (percentages) as selected by the students. “2^nd missing” denotes those who didn’t select a second choice.

	Accomplishing the correct examination process
	Live demonstration	Hands-on training	Instructional videos	Written materials	2nd missing
Live demonstration		13 (9.6)	9 (6.7)	7 (5.2)	3 (2.2)
Hands-on training	13 (9.6)		24 (17.8)	7 (5.2)	9 (6.7)
Instructional videos	9 (6.7)	24 (17.8)		28 (20.7)	6 (4.4)
Written materials	7 (5.2)	7 (5.2)	28 (20.7)		3 (2.2)
2nd missing	3 (2.2)	9 (6.7)	6 (4.4)	3 (2.2)
	Performing the correct standard view
	Live demonstration	Hands-on training	Instructional videos	Written materials	2nd missing
Live demonstration		39 (28.9)	7 (5.2)	1 (0.7)	-
Hands-on training	39 (28.9)		34 (25.2)	4 (3.0)	18 (13.3)
Instructional videos	7 (5.2)	34 (25.2)		5 (3.7)	3 (2.2)
Written materials	1 (0.7)	4 (3.0)	5 (3.7)		–
2nd missing	–	18 (13.3)	3 (2.2)	–
	Identifying anatomical landmarks
	Live demonstration	Hands-on training	Instructional videos	Written materials	2nd missing
Live demonstration		6 (4.4)	2 (1.5)	22 (16.3)	3 (2.2)
Hands-on training	6 (4.4)		5 (3.7)	22 (16.3)	5 (3.7)
Instructional videos	2 (1.5)	5 (3.7)		31 (23.0)	6 (4.4)
Written materials	22 (16.3)	22 (16.3)	31 (23.0)		13 (9.6)
2nd missing	3 (2.2)	5 (3.7)	6 (4.4)	13 (9.6)

Subjective learning experiences with regard to the use of instructional ultrasound videos

In their responses to the survey, a total of 84 students elaborated on situations where using instructional videos helped them learn. The open-ended responses were classified into three categories of positive feedback (‘handling of the ultrasound machine,’ ‘examination process,’ and ‘transducer handling’).

A typical response that illustrated the usefulness of the videos was when the students reported that they felt more confident ‘handling US machines’ in practice in cases in which the instructional videos demonstrated the use of the same sonographic unit as used in hands-on training (28.5%). The advantages of instructional videos with respect to learning the examination process were typically demonstrated by the standardized procedure shown in the videos about how to obtain each standard view correctly (24%). Many students also considered repeated viewing of the handling of the transducer to be extremely useful (16.5%). 31% of the students reported at least two of the prior categories as extremely useful.

Another 64 students responded to the open-ended question about how to further improve the instructional videos. 28% mentioned that more practice tips (e. g., how to image obese patients) during a standard examination would be useful, 19% referred to technical difficulties with regard to the download of the instructional videos, and 16% stated that a video explaining possible pathological findings should be made available. One student claimed that the examination process was not explained well, while 36% of the students stated that the instructional videos were “perfect” or “very good” and did not need any kind of modification.

Discussion

This study shows that, in preparation for upcoming OSCE exams, students relied heavily on available learning materials, and, particularly, instructional videos. The findings demonstrate that instructional videos are largely perceived as a “very useful” tool with which to learn practical US skills. Instructional videos can be used as an “on-demand” learning tool, providing a similarly valuable experience as in-person teachers who give a live demonstration. Videos allow students to observe subtle movements of the transducer as an important aspect of in-depth knowledge. This is in line with Schwerdtfeger et al. (2014), who demonstrated that instructional videos enabled students to train in skills that represented “difficult to illustrate” situations in the acute treatment of trauma patients, and provided an “optimal observation point” for students. Global rating scales in an OSCE were thereby significantly improved [18].

In our study, instructional videos were the preferred learning material for achieving the specific learning objective of “accomplishing the correct examination process.” Xiao et al. [15] reported similar results when evaluating the effect of “online video-based training” on sterile-practice compliance during central venous catheterization. It was shown that compliance was better in the “online video-based training” group, compared to the “paper-based training” group.

In a Korean nationwide survey, Jang et al. evaluated the usefulness of OSCE videos in learning clinical skills [20]. By means of a questionnaire, students’ responses on three different topics were collected. Jang et al. prompted responses as to the perception of the effectiveness of the instructional videos. The results were comparable to the present study. A large portion of students [42% of 411 students from 31 medical schools) reported that the videos enabled them to review the execution of difficult clinical skills. Furthermore, 37% of 411 of the students used the instructional videos to refresh their memory of previously studied skills [20]. A majority of the students felt that instructional videos for OSCE were very useful in learning clinical skills [20].

Alqahtani et al. investigated the use of instructional videos compared to live demonstration for teaching pre-clinical dental skills with two groups. In both groups, the performance was evaluated and an eight-item questionnaire (Likert scale: 1=strongly disagree, 5=strongly agree) was developed to measure the students’ subjective perception of instructional videos and live demonstrations. Instructional videos received a higher mean score in response to the statement “the steps in the teaching method were presented in a clear fashion and were easy to understand,” compared to live demonstration. This finding is congruent with the present evaluation in which instructional videos were preferred as a means of studying specific learning objectives ([Table 4]).

Alqahtani et al., however, demonstrated that instructional videos were perceived to be less useful when compared to live demonstration. It was reported that 40% of the procedural video group preferred the learning material they received versus 59% of the students in the live demonstration group, who favored their learning method [21]. In the present study, students had the opportunity to use various learning materials. It was shown that there was no statistically significant difference in the preference for instructional videos compared to live demonstration in learning sonographic skills.

Interestingly, Alqahtani et al. showed that 35% of the students were not satisfied with instructional videos as a sole teaching method. A minority of the students (18.8%) were not satisfied with live demonstration as the only teaching method [21]. In the present study, students tended to report that instructional videos were as useful as live demonstrations ([Table 3]).

The results of the present study indicate that different learning objectives require different learning materials. Therefore, the availability of various learning materials is appreciated by students. Instructional videos for ultrasound are not widely used in the learning and teaching of sonographic skills [13]. Given the increasing role of ultrasound in diagnostic procedures, it is recommended that ultrasound teaching should be implemented early in the medical curriculum. Students already benefit from learning ultrasound as undergraduates, as it improves their understanding of anatomy and pathology [16] [17]. Furthermore, students should learn ultrasound with hands-on training in an ultrasound skills lab and also be provided instructional videos [11], although instructional videos should be supplemented with additional learning materials [20]. Offering students instructional videos is desirable so that they can prepare to learn ultrasound skills before participating in cost-intensive skills labs, which would help to reduce personnel costs [11] [18].

As an additional marker of learning success, self-efficacy was evaluated. In a study by Sowan et al. in 2014 [19], 102 undergraduate nursing students answered questions about their self-efficacy with regard to videos as a learning resource to understand the skillset in clinical nursing work. Half the students stated that they understood 75% of the procedures without further explanation by their instructors [19]. In our study, students also reported a high perceived overall self-efficacy after using the instructional videos, with 86.7% of students feeling able to achieve at least two of three required learning objectives. Even though a high self-efficacy for the three learning objectives was reported, students felt significantly better prepared for the objective “accomplishing a correct examination process” (96.1%). This is also in accordance with the finding that students rely on instructional videos more than the other provided learning materials to acquire this specific learning objective. This suggests that videos are especially useful for teaching skills that follow a precise order ([Fig. 3], [4]).

Fig. 3 Reported self-efficacy for achieving three different learning objectives: a accomplishing the correct examination process; b performing the correct standard view; and c identifying anatomical landmarks

Fig. 4 ported self-efficacy for achieving 0, 1, 2, or 3 learning objectives

Despite our success in showing the specific usefulness of videos compared to other means of learning, the present study has several limitations. As usefulness was rated by students, there may have been a bias that resulted from social desirability. Our study took place in the real-life educational setting and targeted the training of all students toward mastery of skills. As such, there was no available quantitative measure that differentiated between students’ US skills. As the study looked for differences in usefulness between learning materials, there was no randomization and no control group to compare the effect of using instructional videos for learning versus a group that did not receive instructional videos for learning.

In conclusion, instructional videos appear to be a valuable learning tool as they support standardized teaching and learning of clinical skills. The use of videos may alleviate the shortage of supervised US training for large cohorts. In addition, instructional videos may support self-regulated learning, as they can be made available independent of time and location, and relevant sequences can be studied as often as needed. Despite their advantages, instructional videos should not be seen as a replacement for other learning materials. Students value practicing together with their clinical supervisors as much as practicing alone using written or video-recorded material. A rich learning environment, thus, should strive to provide students with all types of learning materials, as each type serves a specific function in the learning process.

Conflict of Interest

The authors declare that they have no conflict of interests.

• References

• 1 O'Riordan C, Condon F, Conhyea D, Kaliszer M, O'Brien T. The role of ultrasound screening for hip dysplasia. Ir Med J 2005; 98: 147-149
• 2 Pleister A, Kahwash R, Haas G, Ghio S, Cittadini A, Baliga RR. Echocardiography and heart failure: a glimpse of the right heart. Echocardiography. 2015; 32 S95-S107
• 3 Lichtenstein DA. Lung ultrasound in the critically ill. Ann Intensive Care 2014; 4: 1
• 4 Nylund K, Maconi G, Hollerweger A, Ripolles T, Pallotta N, Higginson A et al. EFSUMB Recommendations and Guidelines for Gastrointestinal Ultrasound - Part 1: Examination Techniques and Normal Findings (Long version). Ultraschall in Med 2016
• 5 Gaudet J, Waechter J, McLaughlin K, Ferland A, Godinez T, Bands C. et al. Focused critical care echocardiography: Development and evaluation of an image acquisition assessment tool. Crit Care Med 2016; 44: e329-e335
• 6 Perera P, Mailhot T, Riley D, Mandavia D. The RUSH exam: Rapid Ultrasound in SHock in the evaluation of the critically lll. Emerg Med Clin North Am 2010; 28: 29-56 vii
• 7 Konge L, Albrecht-Beste E, Bachmann Nielsen M. Ultrasound in Pre-Graduate Medical Education. Ultraschall in Med 2015; 36: 213-215
• 8 Chiem AT, Soucy Z, Dinh VA, Chilstrom M, Gharahbaghian L, Shah V et al. Integration of Ultrasound in Undergraduate Medical Education at the California Medical Schools: A Discussion of Common Challenges and Strategies From the UMeCali Experience. J Ultrasound Med 2016
• 9 Garde S, Heid J, Haag M, Bauch M, Weires T, Leven FJ. Can design principles of traditional learning theories be fulfilled by computer-based training systems in medicine: the example of CAMPUS. Int J Med Inform 2007; 76: 124-129
• 10 Swamy M, Searle RF. Anatomy teaching with portable ultrasound to medical students. BMC Med Educ 2012; 12: 99
• 11 Cantisani V, Dietrich CF, Badea R, Dudea S, Prosch H, Cerezo E. et al. EFSUMB Statement on Medical Student Education in Ultrasound [long version]. Ultrasound Int Open 2016; 2: E2-E7
• 12 Solomon SD, Saldana F. Point-of-care ultrasound in medical education–stop listening and look. N Engl J Med 2014; 370: 1083-1085
• 13 Woodworth GE, Chen EM, Horn JL, Aziz MF. Efficacy of computer-based video and simulation in ultrasound-guided regional anesthesia training. J Clin Anesth 2014; 26: 212-221
• 14 Xeroulis GJ, Park J, Moulton CA, Reznick RK, Leblanc V, Dubrowski A. Teaching suturing and knot-tying skills to medical students: a randomized controlled study comparing computer-based video instruction and (concurrent and summary) expert feedback. Surgery 2007; 141: 442-449
• 15 Xiao Y, Seagull FJ, Bochicchio GV, Guzzo JL, Dutton RP, Sisley A. et al. Video-based training increases sterile-technique compliance during central venous catheter insertion. Crit Care Med 2007; 35: 1302-1306
• 16 Tshibwabwa ET, Groves HM. Integration of ultrasound in the education programme in anatomy. Med Educ 2005; 39: 1148
• 17 Chin KJ, Perlas A, Chan V. The ultrasound-assisted paraspinous approach to lumbar neuraxial blockade: a simplified technique in patients with difficult anatomy. Acta Anaesthesiol Scand 2015; 59: 668-673
• 18 Schwerdtfeger K, Wand S, Schmid O, Roessler M, Quintel M, Leissner KB. et al. A prospective, blinded evaluation of a video-assisted ‘4-stage approach’ during undergraduate student practical skills training. BMC Med Educ 2014; 14: 104
• 19 Sowan AK, Idhail JA. Evaluation of an interactive web-based nursing course with streaming videos for medication administration skills. Int J Med Inform 2014; 83: 592-600
• 20 Jang HW, Kim KJ. Use of online clinical videos for clinical skills training for medical students: benefits and challenges. BMC Med Educ 2014; 14: 56
• 21 Alqahtani ND, Al-Jewair T, Al-Moammar K, Albarakati SF, EA AL. Live demonstration versus procedural video: a comparison of two methods for teaching an orthodontic laboratory procedure. BMC Med Educ 2015; 15: 199