Attending Perceptions on the Use of Preference Signaling in the Ophthalmology Residency Application Process

Abstract

Objective This article characterizes perspectives of ophthalmologists involved in the residency selection process regarding the potential impact of preference signaling on the ophthalmology residency match.

Methods An anonymous online questionnaire generated from SurveyMonkey was approved by the Association of University Professors of Ophthalmology (AUPO) Data Resource Committee for distribution to 391 individuals from the AUPO Departmental Chairs, Program Directors, and Directors of Medical Student Education email listservs in August 2022.

Results A total of 96 (24.6%) ophthalmology faculty completed the questionnaire. The majority (n = 76, 79.2%) agreed or strongly agreed that preference signaling should be implemented in the ophthalmology residency application system. Most respondents agreed or strongly agreed that preference signaling will allow for more holistic reviews of applications (n = 55, 57.3%), agreed or strongly agreed that it will benefit applicants who do not have connections to home programs or faculty that can reach out to desired programs (n = 81, 84.4%), and agreed or strongly agreed that it will improve the distribution of interviews to applicants (n = 76, 79.2%). Participants agreed or strongly agreed that applicants who have signaled interest in their program will receive preference when offering interviews (n = 59, 61.5%), and those signals will be used as a tiebreaker for similar applications (n = 75, 78.1%). The majority of participants believed that the ideal number of preference signals' applicants should be given three to four signals (n = 35, 36.0%) or five to six signals (n = 29, 30.2%).

Conclusion A majority of ophthalmology faculty surveyed support the integration of preference signaling into the ophthalmology residency match.

Keywords

ophthalmology match - medical education - holistic review - interview selection - residency selection - preference signaling - program directors

Background

Each year, the number of residency applications submitted per student has increased across all specialties including ophthalmology.[1] During the 2022 application cycle, successfully matched ophthalmology applicants applied to an average of 81 programs—a 12.5% increase compared to ophthalmology applicants during the 2017 application cycle.[2] On average, the number of applications submitted for successfully matched ophthalmology applicants has increased by 1.5 each year over the last 6 years. In a 2021 survey distributed to ophthalmology program directors (PDs) after the first application cycle in which interviews were held virtually, 72% of participating ophthalmology PDs reported an increased number of applications received by their respective programs compared to previous years.[3] The rise in applications that programs received has limited the time available for holistic review, and as a result, ophthalmology PDs rely on data including United States Medical Licensing Examination (USMLE) scores, class rank, grades, and Alpha Omega Alpha status.[4] While previously thought to be objective measures, these factors have been found to be subject to bias and racial disparities, and the transition of USMLE step 1 scores from numerical grades to pass/fail has made the review process more difficult.[5] [6]

Starting with the 2021 to 2022 application cycle, many other medical specialties including otolaryngology, dermatology, and urology have implemented preference signaling as a possible solution to application overload.[7] [8] [9] [10] [11] [12] Preference signaling is a standardized system in which residency applicants may indicate to particular programs a sincere interest before application review and interview selection.[9] Applicants are limited to the number of signals they are able to send. Signaling aims to both mitigate the disparities inherent in resource inequities between applicants while improving transparency by providing a formal, uniform system for expressing interest. While preference signaling may come with benefits, it has only been recently implemented in the residency application process.

The ideal number of signals per applicant remains unclear. Having too many tokens would decrease their value, while limiting them would force applicants to choose between their top programs. For the 2021 to 2022 application cycle, otolaryngology allowed four signals, dermatology allowed three signals, and internal medicine, surgery, and urology allowed five signals.[11] [12] [13] The number of programs and applicants varies between these specialties and ophthalmology, so it is difficult to discern an ideal number of signals for ophthalmology applicants based on these guidelines. This study aims to clarify the perception among ophthalmologists involved in residency selection of integrating preference signaling into the Ophthalmology Match.

Methods

An anonymous online 19-item questionnaire hosted by SurveyMonkey (Momentive Inc., Waterford, NY) was approved by the Association of University Professors of Ophthalmology (AUPO) Data Resource Committee for distribution to 391 individuals from the AUPO Chair, PDs, and Directors of Medical Student Education email listservs on August 1, 2022. The email included a description of the study and a secure link to the SurveyMonkey questionnaire. The study protocol was determined to be exempt from Institutional Review Board (IRB) approval by the University of Maryland Baltimore IRB and University of Miami IRB. The research performed adhered to the tenets of the Declaration of Helsinki.

Participants answered questions about demographic data, years of experience in residency selection, and rated statements characterizing how ophthalmology faculty perceived the use of preference signaling in the ophthalmology residency match ([Supplementary Material 1], available in the online version). Statements were rated based on a Likert scale (strongly disagree, disagree, neither disagree or agree, agree, or strongly agree). Responses to the questionnaire were collected over the course of 9 weeks between August 1 to October 1, 2022. Per AUPO Survey guidelines, no reminders for completing the questionnaire were sent after the initial email. Statistical analysis was performed using Microsoft Excel (Microsoft Corporation, Redmond, WA).

Results

Respondent Demographics

The questionnaire was sent to 391 individuals and a total of 96 (24.6%) ophthalmology faculty responded. Participants were allowed to choose more than one option when answering the question regarding positions held in the ophthalmology department. For example, a participant may select “program director” and “professor” if they fall under both the positions. Participants included PDs (n = 39, 40.6%), Chairs (n = 30, 31.3%), Directors of Medical Student Education (n = 26, 27%), Vice-Chairs (n = 12, 12.5%), and Associate/Assistant PDs (n = 9, 9.4%). Faculty rank included Professors (n = 15, 15.6%), Associate Professors (n = 13, 13.5%), and Assistant Professors (n = 12, 12.5%). Responses were distributed among programs from the North Central (n = 27, 28%), New England and Middle Atlantic (n = 24, 25%), South Atlantic (n = 19, 19.8%), South Central (n = 18, 18.8%), Pacific (n = 5, 5.2%), and Mountain (n = 3, 3%) regions. The median number of years for postresidency graduation was 15.5. The median number of years involved in both the interview selection process and ranking process was 10. Of the participants, 61.5% were male (n = 59), 36.5% were female (n = 35), and 2.1% preferred not to specify (n = 2; [Table 1]).

Table 1 Demographic data of survey participants from ophthalmology residency programs

Characteristics	Number of responses (%)
Position
Program Director	39 (40.6)
Associate/Assistant Program Director	9 (9.4)
Chair	30 (31.2)
Vice-Chair	12 (12.5)
Director of Medical Student Education	26 (27.1)
Professor	15 (15.6)
Associate Professor	13 (13.5)
Assistant Professor	12 (12.5)
Gender
Male	59 (61.5)
Female	35 (36.5)
Prefer not to specify	2 (2.1)
Years postresidency graduation, median (IQR)	15.5 (9.8–26.3)
Years involved in interview selection, median (IQR)	10 (5–15)
Years involved in ranking process, median (IQR)	10 (4–15)
Region
New England and Middle Atlantic	24 (25)
North Central	27 (28.1)
South Atlantic	19 (19.8)
South Central	18 (18.8)
Mountain	3 (3.1)
Pacific	5 (5.2)

Abbreviation: IQR, interquartile range.

Perceptions of Preference Signaling

A majority of participants agreed or strongly agreed that preference signaling should be implemented in the ophthalmology application system (n = 76, 79.2%; [Table 2]). They also believed that implementing preference signaling will allow for more holistic reviews of applications (n = 55, 57.3%), benefit applicants who do not have connections to home programs or faculty that can reach out to desired programs (n = 81, 84.4%), improve the distribution of interviews to applicants (n = 76, 79.2%), decrease geographical bias when offering interviews (n = 62, 64.6%), and will be a better alternative to program-specific essays for the initial application review phase (n = 59, 61.5%). Most participants agreed or strongly agreed that applicants who have signaled interest in their program will receive preference when offering interviews (n = 59, 61.5%) and those signals will be used as a tiebreaker for similar applications (n = 75, 78.1%). However, only 39.6% (n = 38) of participants felt that applicants who have signaled interest in their program will receive preference when determining a rank list and only 16.7% (n = 16) viewed absence of a signal as a sign of disinterest. Additionally, most participants agreed or strongly agreed that implementing preference signaling would de-emphasize numerical cutoffs in the application review process (n = 49, 51.0%), but a majority did not agree or strongly agree that signals would be effective alternatives to away rotations for applicants to demonstrate interest in a program (n = 43, 44.8%). In terms of number of preferences signals that applicants should receive, the majority of respondents answered either three to four signals (n = 35, 36.0%) or five to six signals (n = 29, 30.2%; [Fig. 1]).

Table 2 Survey participants' perspectives of preference signaling on the ophthalmology match

Statement	Strongly disagree	Disagree	Neither agree/disagree	Agree	Strongly agree
Preference signaling should be implemented in the ophthalmology application system	6 (6.3%)	4 (4.2%)	10 (10.4%)	25 (26.0%)	51 (53.1%)
Implementing preference signaling will allow for more holistic reviews of applications	6 (6.3%)	10 (10.4%)	25 (26.0%)	25 (26.0%)	30 (31.3%)
Implementing preference signaling will de-emphasize numerical cutoffs in the application review process	7 (7.3%)	20 (20.8%)	20 (20.8%)	21 (21.9%)	28 (29.2%)
Implementing preference signaling will benefit applicants who do not have connections to home programs or faculty that can reach out to desired programs	2 (2.1%)	4 (4.2%)	9 (9.4%)	39 (40.6%)	42 (43.8%)
Implementing preference signaling will improve the distribution of interviews to applicants (avoiding a subset of applicants receiving a larger number of interviews per person)	5 (5.2%)	1 (1.0%)	14 (14.6%)	40 (41.7%)	36 (37.5%)
Implementing preference signaling will decrease geographical bias when offering interviews	5 (5.2%)	13 (13.5%)	16 (16.7%)	33 (34.4%)	29 (30.2%)
Preference signaling will be an effective alternative to away rotations for applicants to demonstrate interest in a program	8 (8.3%)	20 (20.8%)	25 (26.0%)	30 (31.3%)	13 (13.5%)
Preference signaling will be a better alternative to program specific essays for the initial application review phase	1 (1.0%)	13 (13.5%)	23 (24.0%)	33 (34.4%)	26 (27.1%)
An applicant who has signaled interest in my program will receive preference when offering interviews	2 (2.1%)	3 (3.1%)	22 (22.9%)	54 (56.3%)	15 (15.6%)
An applicant who has signaled interest in my program will receive preference when determining a rank list	5 (5.2%)	10 (10.4%)	43 (44.8%)	34 (35.4%)	4 (4.2%)
Signals will be used as a tiebreaker for similar applications	6 (6.3%)	1 (1.0%)	14 (14.6%)	54 (56.3%)	21 (21.9%)
Absence of a signal will be viewed as a sign of disinterest	18 (18.8%)	31 (32.3%)	31 (32.3%)	13 (13.5%)	3 (3.1%)

Fig. 1 Percentage distribution of participant responses to the question, “How many preference signals should applicants be given?”.

Discussion

Overall, preference signaling is viewed favorably among ophthalmology faculty members that responded to this questionnaire. The results of this study align with existing literature suggesting that signaling is a way to provide applicants a standardized system for expressing interest to desired programs.[7] [8] [11] [13] [14] Survey studies in dermatology, otolaryngology, and urology have found considerable advantages to using signaling such as improving the distribution of interview offers among applicants and allowing seriously interested applicants to standout to desired programs.[7] [15] [16] PDs in otolaryngology reported that signals were used as a tiebreaker for similar applications and as part of the initial application review algorithm.[7] The rate at which applicants received interview offers was significantly higher for programs they signaled compared to nonsignaled programs in the otolaryngology, dermatology, and urology residency match.[7] [15] [16] These specialties reported that signals were most commonly used as part of the initial application review to extend interviews and as a tiebreaker for similar applications, but not for determining a rank list, which is consistent with our findings.[7] Additionally, orthopaedic PDs surveyed ranked signaling only below away rotation performance and personal knowledge of the applicant in terms of importance for residency selection.[17] Given the promising results in other specialties' matches, the American Orthopaedic Association's Council of Orthopaedic Residency Directors announced in March 2022 its endorsement for a preference signaling program for the 2022 to 2023 application cycle.

Traditionally, applicants would use opportunities such as away rotations, advocacy from faculty mentors, and preinterview communication (emails, phone calls, etc.) to demonstrate interest in desired programs.[7] While these methods can be effective, they may exacerbate inequities in the application process for those who do not have home ophthalmology programs or students with limited financial means who cannot afford the cost of an away rotation. A survey distributed to medical students who participated in away rotations reported that the average cost of a single rotation was $958.[18] Previous studies have shown that ophthalmology applicants without home residency programs are at a considerable disadvantage. The presence of a home ophthalmology residency program was associated with a 1.4-fold increase in likelihood of matching. However, only 53% of U.S. medical schools are associated with an ophthalmology residency program.[19] [20] Medical students who have an ophthalmology department may have advantages in terms of access to resources and mentors who can advocate on their behalf during the residency application process. For these reasons, it is notable that the statement “implementing preference signaling will benefit applicants who do not have connections to home programs or faculty that can reach out to desired programs” had the highest percentage of people agree or strongly agree (84.4%) in this questionnaire. This strongly supports that signaling may make the residency application process more equitable for these students. Previous studies also reported that approximately 60% of applicants matched to the same geographic region as their medical school.[20] A majority of participants in our study believed that signaling may decrease geographical bias when offering interviews, which means that it has the potential to bring more visibility to applicants looking to move to certain regions that may have been previously inaccessible to them. For example, a medical student may have chosen to go to an in-state school for the lower cost of tuition rather than a school in a desirable location that was out-of-state.

Our analysis also shows that signaling may streamline the application review and interview distribution process of the ophthalmology residency match. A majority of ophthalmology faculty believe that signaling will allow for more holistic review of applications and improve the distribution of interviews to applicants. A common problem outlined across different competitive surgical subspecialties was that a disproportionate number of interviews were being given to a subset of “high-tier” applicants.[9] [21] This may be in part due to the increasing number of applications submitted per student. To continue completing holistic review of applicants in a reasonable amount of time, residency programs may rely more on filters like USMLE scores and class rank which may exclude a cohort of applicants. Signaling may improve the efficiency of the interview invitation process by allowing programs to assign interviews to truly interested applicants and potentially decrease the number of interview cancellations, especially with the interview cap of 15 for the 2023 ophthalmology application cycle. Allowing a formal system for applicants to signal preferences may also decrease the amount of time spent by applicants sending and program leadership responding to informal communications like letters of interest.

It is important to note that the majority of participants reported that an absence of a signal would not be viewed as a sign of disinterest in their program. This is a valuable perspective to keep in mind when determining the number of signals that should be available for ophthalmology applicants because having too many signals may lead to programs viewing the lack of a signal as an indication of disinterest. From our survey, the ideal number of signals is between three and six, which is comparable to other competitive specialties.[11] [12] [13] During the 2021 to 2022 application cycle, there were 643 otolaryngology, 1,019 dermatology, 604 urology, and 748 ophthalmology applicants. The average applications per program were 427 for otolaryngology, 566 for dermatology, and 350 for urology.[22] Although there are no data for the average applications per programs for ophthalmology, the number of applicants is closest to otolaryngology which allowed four signals. Given this comparison and the results of our survey, four signals may be the ideal number for the ophthalmology residency match. The perception of signaling as an effective alternative to away rotations remains contentious among our respondents. This may be because away rotations require a much larger time and financial commitment. However, as board examinations move towards pass–fail scoring, ophthalmology PDs reported that they will place increased importance on applicants attending a rotation in their department and believed applicants will attend more away rotations.[23] As mentioned before, medical students with financial barriers may already be geographically limited due to attending an in-state versus an out-of-state school. These same applicants may also be unable to afford multiple away rotations which puts them at a significant disadvantage, so preference signaling may be an equitable alternative.

Strengths of this study include a relatively large, diverse sample size with participants across different positions, levels of experience, and geographical regions. Limitations of this study include potential responder bias as ophthalmology faculty with stronger opinions about the topic may be more likely to respond. Another limitation is that this survey only explored the potential benefits of preference signaling. Future studies will need to explore potential drawbacks and strategies applicants should consider when using the signaling system, such as using signals on less competitive programs to secure a “safety” interview or rank spot.[10]

Conclusion

In summary, a majority of ophthalmology faculty surveyed agreed that preference signaling should be implemented in the ophthalmology residency match. If implemented successfully, preference signaling has the potential to make it easier for programs to sort through applications while simultaneously increasing equity within the process for applicants. Based off of other specialties and the results from this survey, we recommend giving four signals to each applicant.

Conflict of Interest

J.S. is a consultant for Alcon, Allergan, Apellis, D.O.R.C., Genentech, OcuTerra, and Regeneron.

• References

• 1 Association of American Medical Colleges. ERAS Statistics. 2021. Accessed March 5, 2022 at: https://www.aamc.org/eras-statistics-2019
• 2 SF Match. Ophthalmology Residency Match Summary Report. Accessed May 5, 2022 at: https://www.sfmatch.org/specialty/ophthalmology-residency/Statistics
• 3 Venincasa MJ, Steren B, Young BK. et al. Ophthalmology Residency Match in the Covid-19 Era: Applicant and Program Director Perceptions of the 2020-2021 Application Cycle. Semin Ophthalmol 2022; 37 (01) 36-41
• 4 Nallasamy S, Uhler T, Nallasamy N, Tapino PJ, Volpe NJ. Ophthalmology resident selection: current trends in selection criteria and improving the process. Ophthalmology 2010; 117 (05) 1041-1047
• 5 Boatright D, O'Connor PG. E Miller J. Racial Privilege and Medical Student Awards: Addressing Racial Disparities in Alpha Omega Alpha Honor Society Membership. J Gen Intern Med 2020; 35 (11) 3348-3351
• 6 Ross DA, Boatright D, Nunez-Smith M, Jordan A, Chekroud A, Moore EZ. Differences in words used to describe racial and gender groups in Medical Student Performance Evaluations. PLoS ONE 2017; 12 (08) e0181659
• 7 Pletcher SD, Chang CWD, Thorne MC, Malekzadeh S. The Otolaryngology Residency Program Preference Signaling Experience. Acad Med 2022; 97 (05) 664-668
• 8 Chang CWD, Pletcher SD, Thorne MC, Malekzadeh S. Preference Signaling for the Otolaryngology Interview Market. Laryngoscope 2021; 131 (03) E744-E745
• 9 Fantasia J, Elsamra S, Thavaseelan S. Improving The Match: Use of Preference Signaling to Optimize the Urology Match Interview Process. Urology 2021; 154: 57-61
• 10 Gangal A, Blalock TW. A perfect match: pros and cons of preference signaling in dermatology. J Am Acad Dermatol 2021; 84 (05) 1504-1505
• 11 Traxel E, Richstone L, Brown J, Mirza M, Greene K, Thavaseelan S. Preference signaling pilot in the urology match: outcomes and perceptions. Urology 2022; 170: 27-32
• 12 Association of American Medical Colleges. Supplemental ERAS application guide. 2021. Accessed March 5, 2022 at: https://students-residents.aamc.org/media/12326/download
• 13 Otolaryngology Program Directors Association. Otolaryngology preference signaling: updates. 2021. Accessed March 5, 2022 at: https://opdo-hns.org/mpage/signaling-updates
• 14 Pelletier-Bui AE, Schnapp BH, Smith LG. et al. Making our preference known: preference signaling in the emergency medicine residency application. West J Emerg Med 2021; 23 (01) 72-75
• 15 Dirr MA, Brownstone N, Zakria D, Rigel D. Dermatology match preference signaling tokens: impact and implications. Dermatol Surg 2022; 48 (12) 1367-1368
• 16 Leopold Z, Rajagopalan A, Mikhail M. et al. Preference signaling in the 2022 urology residency match - the applicant perspective. Urology 2022; 170: 33-37
• 17 Mun F, Suresh KV, Li TP, Aiyer AA, LaPorte DM. Preference signaling for orthopaedic surgery applicants: a survey of residency program directors. J Am Acad Orthop Surg 2022; 30 (23) 1140-1145
• 18 Winterton M, Ahn J, Bernstein J. The prevalence and cost of medical student visiting rotations. BMC Med Educ 2016; 16 (01) 291
• 19 Ahmed H, Law JC, Felsted D. et al. Matching ophthalmology amidst coronavirus disease 2019 (COVID-19): lessons that went viral. J Acad Ophthalmol 2020; 12 (02) e200-e204
• 20 Loh AR, Joseph D, Keenan JD, Lietman TM, Naseri A. Predictors of matching in an ophthalmology residency program. Ophthalmology 2013; 120 (04) 865-870
• 21 Salehi PP, Benito D, Michaelides E. A novel approach to the national resident matching program-The Star System. JAMA Otolaryngol Head Neck Surg 2018; 144 (05) 397-398
• 22 Association of American Medical Colleges. ERAS Statistics. 2023. Accessed January 25, 2023 at: https://www.aamc.org/data-reports/interactive-data/eras-statistics-data
• 23 Cui D, Scott IU, Wingert HL. Ophthalmology Program Directors' Perspectives on the Impact of the United States Medical Licensing Examination Step 1 Change to Pass-Fail Scoring. J Acad Ophthalmol 2020; 12 (02) e277-e283

Publication History

Received: 23 February 2023

Accepted: 31 August 2023

Article published online:
25 September 2023