Ophthalmology Residency Virtual Interviews in the Setting of the COVID-19 Pandemic: Perspectives of Applicants, Selection Committee Members, and Current Residents

Abstract

Background In the setting of the COVID-19 pandemic, residency programs implemented videoconferencing “virtual” interviews for the 2020 to 2021 match cycle. There is limited published information on virtual ophthalmology residency interviews.

Objective The study aimed (1) to assess applicant, selection committee member, and resident opinions of technical quality, communication quality, and ability to assess applicant or program “fit” during virtual interviews; (2) to determine which interview format—in-person or virtual—each party would prefer in the future; and (3) to survey which residency resources applicants found helpful.

Design Surveys were sent to applicants, selection committee members, and residents to assess the above objectives for the 2020 to 2021 match cycle virtual interviews at the Moran Eye Center, University of Utah.

Setting This study was conducted in a single residency program interview season from 2020 to 2021.

Participants Forty applicants, eight committee members, and seven residents who participated in the virtual interview process were surveyed.

Intervention or Exposure Prior to interviews, various avenues were implemented to connect with applicants. A videoconferencing software was utilized for interviews. Applicants and selection committee members met in one-on-one or small group interviews. Residents communicated with applicants in a large group setting between interviews.

Main Outcome and Measure The study aims to survey the participants as stated in the objectives. There was no planned outcome for this quality improvement study.

Results Survey response rate was 98.2% (54/55). All parties rated the technical components as good or very good. Applicants and selection committee members rated communication as overall good or very good, although residents thought communication was very poor. A total of 92.3% applicants, 75% selection committee members, and 0% residents were reported that they were able to appropriately assess fit of the program with the applicant. However, 46.3% respondents preferred in-person interviews in the future. Popular applicant resources were resident-produced videos (82.1%), conversations with residents (46.2%), and a gift bag (43.6%).

Conclusion and Relevance Overall, the technical components of the interview were successful. Small, structured group interactions led to better communication and assessment of fit. There were variable opinions regarding future interview format preference between in-person, virtual, or choice. Amid the COVID-19 pandemic, connecting with applicants via various means can optimize the match process.

Keywords

ophthalmology residency - virtual residency interview - resident selection - videoconferencing interview

During the COVID-19 pandemic, the Coalition for Physician Accountability, a group of national medical education organizations, recommended that residency interviews move to an online platform for the 2020 to 2021 match cycle.[1] Some non-ophthalmology residency programs have published their virtual interview experiences. However, there are no published graduate medical education (GME) virtual interview best practices. In this paper, we evaluate the efficacy of virtual interviews at our ophthalmology residency program through applicant, faculty selection committee, and resident perspectives to improve future interviews.

Materials and Methods

To reach applicants prior to interviews, we established a resident-run Instagram account, hosted two “Open House” sessions via Zoom (San Jose, CA) videoconferencing software, and arranged 30 minute to 1 hour phone calls between residents and candidates. Our program's GME website was updated with resident-produced videos about the program and links to curriculum initiatives. If selected for interview, applicants received a hand-written note from a resident and gift bag per Association of University Professors of Ophthalmology guidelines.

Prior to interviews, the selection committee held simulations to practice videoconferencing technical components. Each of the four interview days consisted of (1) academic team introductions, (2) resident question-and-answer session, (3) two selection committee panel interviews, and one-on-one interviews with (4) the chairman and (5) program director. During each interview, the applicant was asked six standardized questions. The one-on-one interviews were unstructured. Between interviews, the applicants returned to an informal virtual “waiting room” with residents and other applicants to play group games and ask residents questions. Applicants were told that they could elect to participate or turn off their video and audio, and spent 1 to 2 hours in the virtual waiting room. Our program coordinators kept track of time and “moved” applicants between interviews.

This project was determined to be exempt as a quality improvement project by our institutional review board (IRB_00136386). After interviews, 40 interviewees, eight selection members, and seven residents were emailed an anonymous survey link regarding their experiences. Participants were informed that the survey would be anonymous, voluntary, and would not influence residency selection. We utilized RedCAP (Nashville, TN) and SurveyMonkey (San Mateo, CA) for survey distribution with a reminder email 1 week after.

Surveys consisted of multiple choice, Likert scale, and short answer questions. The 12-item applicant survey assessed the perception of resources about our program, efficacy of technical components of the virtual platform, quality of interaction with residents, ability to represent oneself, ability to assess if the program was a “good fit,” and preference of interview format. Similarly, the seven-item selection committee member and resident surveys assessed perceptions of the quality of communication, technical components, ability to evaluate applicant fit, and future interview method preference.

Quantitative data were analyzed as means, medians, percentages, and totals. Qualitative data were collected in short answer free-response format. As our sample size was limited, we did not perform statistical analysis.

Results

In total, 39 of 40 (97.5%) applicants completed the survey. Eight of eight (100%) selection committee members and seven of seven (100%) residents responded to their respective surveys. Of the surveys that were completed, 100% of the questions were completed by all.

All resources in order of most to least helpful are listed in [Table 1]. [Table 2] lists the results regarding virtual platform technical quality, communication quality, and the ability to determine if applicants would be a good fit for the program. [Table 3] lists applicant, selection committee member, and resident preference for future interview format.

Table 1 Applicant assessment of resource helpfulness

Applicant question: which resources were helpful in making your decision to apply to and interview at the Moran Eye Center? (Select all that apply)	% (n) Total 24
Resident-produced videos for applicants	82.1 (32)
Reputation or ranking	79.5 (31)
Word of mouth (mentor, faculty, family, and friends)	79.5 (31)
Curriculum on website	74.4 (29)
Moran Eye Center CORE website resources	59 (23)
One-on-one conversations with residents via telephone or videoconferencing	46.2 (18)
Gift bag with handwritten note	43.6 (17)
Virtual open house	38.5 (15)
University of Utah webpages	38.5 (15)
Resident Instagram account	35.9 (14)
GME videos	20.5 (8)
Other	7.7 (3)

Abbreviations: CORE, Moran's Clinical Ophthalmology Resource for Education; GME, Graduate Medical Education.

Table 2 Technical quality, communication quality, and assessment of fit

	Technical quality: audio, video, moving from one interview to the next, and internet connection	Quality of communication	Assessment of fit: was this able to be determined?
	1–5 scale,[a] mean (median)	1–5 scale,[a] mean (median)	Answered yes, % (n)
Applicants	4.8 (5)	3.9 (4)	92.3 (36)
Selection committee	4.4 (4)	5 (5)	75 (6)
Residents	4.1 (4)	1.4 (0)	0 (0)

^a Scale: 5 = great/very well; 4 = good/well; 3 = neutral; 2 = poor/ly; 1 = very poor/ly.

Table 3 Preference for future interviews

	Virtual only % (n)	In-person only % (n)	Virtual only with campus visit as separate event % (n)	Option of in-person or virtual interview % (n)
Applicants	15.4 (6)	38.5 (15)	20.5 (8)	25.6 (10)
Faculty	37.5 (3)	50 (4)	0	12.5 (1)
Residents	0	85.7 (6)	0	14.2 (1)

In general, all groups thought that the technical components of the virtual platform were good (4) or very good (5) on a 1 to 5 scale ([Table 2]). Most applicants preferred to spend their breaks talking to residents (97.2%), temporarily logging off (79.5%), and talking to other applicants (53.8%). The majority of applicants believed that they had just the right amount of time to interact with residents (84.6%). Finally, 84.6% of applicants indicated that they were able to interview at more programs this year than if interviews were in-person.

Discussion

In this study, we evaluated the efficacy of our virtual ophthalmology residency interviews from the perspectives of applicants, selection committee members, and residents. We had an excellent survey response rate. Ours is the first quality improvement project to survey all parties involved in the interview process.

As visiting medical student electives were discontinued due to the COVID-19 pandemic, the applicant recruitment process started with public relations initiatives prior to interviews. Per applicants, the most popular resources were resident-made videos about our city and residency program (82.1%), conversations with residents via telephone or videoconferencing (46.2%), and a gift bag with personalized note (43.6%). Although 38.5% of applicants found the open houses to be helpful, not all respondents may have attended. For future virtual interviews, programs can consider using these methods to reach applicants.

Selection committee members and applicants overall perceived that the platform allowed for good communication and assessment of applicant fit for the program and vice-versa ([Table 2]). Unlike selection committee members, residents perceived that the quality of the communication and ability to assess fit was poor ([Table 2]). In free response format, residents submitted suggestions to implement more personal interactions through smaller group rotating waiting rooms of one to two residents per one to two applicants or hosting a virtual meet and greet the night before the interview. Several residents indicated that they did not have the opportunity to get to know applicants personally, as there was no allotment for one-on-one conversations. This indicates that videoconferenced interviews were a good platform for structured, small group conversations, and one-on-one conversations, but were poor for unstructured and large groups. This interaction is particularly meaningful in our interview process, as residents are able to submit the names of applicants they determine would be an appropriate fit with the program. In addition, historically with our in-person interviews, our program coordinators have been able to interact with applicants and gain insight into how applicants interact when there are no residents or faculty present. This year, that interaction was limited as our coordinators were tasked with moving applicants between break out rooms.

Advantages of virtual interviews include reduced cost and time for applicants and programs. Applicants can minimize costs and time for travel and lodging.[2] For programs, virtual interviews can minimize meal cost and lost clinical or surgical time for faculty and residents. University of Arizona's ophthalmology program has been conducting optional virtual interviews for 10 years and also reported on these advantages.[3] An anesthesiology program that offered either in-person or virtual interviews reported that they ranked and subsequently matched equally from both interviewee pools.[4] An internal medicine program in 2014 utilized virtual interviews with positive applicant feedback.[5] If virtual interviews continue to be utilized, we will be able to better determine if this format leads to successful residency matches.

There are possible disadvantages to virtual interviews such as limited ability to experience the city and program in person. A urology program conducted a crossover project of interviewing the same applicant both virtually and in-person with a panel of faculty, and found advantages in time efficiency and cost with virtual interviews. However, applicants expressed a reduced ability to represent themselves over videoconferencing.[6] Applicants to a different urology program thought that most aspects of the interview except resident interactions could be replicated virtually.[7] Formatting interviews to allow for structured one-on-one or small group conversations could better suit virtual interviews. More events to connect applicants to a program and its residents could address this issue.

This match cycle presented the unique challenge of quickly converting a well-established system of in-person ophthalmology residency interviews into the virtual format. About 46.3% of all survey respondents preferred in-person interviews. The majority of applicants preferred virtual only, virtual with optional in-person campus visit, or option of either in-person or virtual interview ([Table 3]). However, a minority of applicants (38%) preferred in-person interviews only. Although the virtual interview was successful technologically, the majority of residents and half of the faculty preferred in-person interviews as well. We did not ask the various stake holders why they preferred in-person interviews. We speculate that applicants may not have had a chance to openly communicate with faculty and residents or assess the city and hospital sites for themselves. Faculty and residents may be unfamiliar with virtual interviews. Residents were not able to assess applicant fit based on large group interactions. Given these potential limitations of virtual interviews, providing applicants with the option of in-person or virtual interviews could potentially bias programs toward selecting in-person interviewees for their residencies and applicants selecting programs and geographic locations that are more familiar to them. Additionally, if in-person interviewees were more likely to match, this could limit applicants with limited financial resources who could have difficulty with travel and lodging expenses.

Our project is limited by its sample size and largely qualitative nature. In critiquing a residency program of interest, applicants may have provided favorable responses. However, they were assured that surveys were anonymous. Respondents may have held subconscious bias regarding virtual interviews due to its novelty. Offering virtual interviews as an option can allow applicants to attend interviews at a greater variety of programs. Our interview evaluation helped us to better understand how virtual interviews were perceived by all parties involved. This perspective will allow us to continue to make improvements for potential future virtual interviews.

Conflict of Interest

None declared.

• References

• 1 Coalition for physician accountability. Final report and recommendations for medical education institutions of LCME-accredited, U.S. Osteopathic, and Non-U.S. medical school applicants, May 11, 2020. Accessed January 2, 2021 at: https://www.aamc.org/media/44736/download
• 2 Moore DB. Not a cheap investment: estimating the cost of the 2017 to 2018 ophthalmology residency match to the applicant and program. J Acad Ophthalmol 2018; 10: e158-e162
• 3 Pasadhika S, Altenbernd T, Ober RR, Harvey EM, Miller JM. Residency interview video conferencing. Ophthalmology 2012; 119 (02) 426-426.e5
• 4 Vadi MG, Malkin MR, Lenart J, Stier GR, Gatling JW, Applegate II RL. Comparison of web-based and face-to-face interviews for application to an anesthesiology training program: a pilot study. Int J Med Educ 2016; 7: 102-108
• 5 Williams K, Kling JM, Labonte HR, Blair JE. Videoconference interviewing: tips for success. J Grad Med Educ 2015; 7 (03) 331-333
• 6 Shah SK, Arora S, Skipper B, Kalishman S, Timm TC, Smith AY. Randomized evaluation of a web based interview process for urology resident selection. J Urol 2012; 187 (04) 1380-1384
• 7 Kenigsberg AP, Khouri Jr RK, Kuprasertkul A, Wong D, Ganesan V, Lemack GE. Urology residency applications in the COVID-19 Era. Urology 2020; 143: 55-61

Publication History

Received: 05 April 2021

Accepted: 22 June 2021

Article published online:
22 November 2021