An Analysis of the Diversity and Inclusion Content Featured on Ophthalmology Residency Program Web Sites

Abstract

Introduction Ophthalmology departments have been stated to be among the least diverse clinical departments at United States medical schools. Improvement requires recruiting a pipeline of diverse trainees. Residency program Web sites represent a potential diversity and inclusion recruitment tool. This study assesses how ophthalmology residency program Web sites demonstrate a commitment to diversity and inclusion.

Methods We analyzed the diversity and inclusion content of 116 ophthalmology residency program Web sites in April 2021. Main outcome measures were the presence of 12 diversity and inclusion elements on program Web sites, based on prior work and Accreditation Council for Graduate Medical Education guidelines: nondiscrimination statement, diversity and inclusion message, community resources, extended faculty or resident biographies (including hobbies, etc.), faculty photos, resident photos, additional financial resources for trainees, wellness resources, mental health resources, health disparities/community engagement, and diversity council. We used Mann–Whitney U and Kruskal–Wallis tests to assess whether residency program characteristics such as rank, size, university affiliation, and geographic location were associated with commitment to diversity and inclusion on affiliated residency Web sites.

Results Ophthalmology residency program Web sites included a mean of 4.4 ± 2.1 diversity elements. Sixteen percent of programs featured more than half (7 + ) of the evaluated diversity elements. The most featured common diversity elements included resident photos (85.3%), faculty photos (78.4%), and community resources (64.3%). Extended faculty biographies (2.6%), mental health resources (9.5%), and diversity council information (11.2%) were less commonly showcased. Top-ranked programs (7.6 ± 1.8, p < 0.0001) and university-based/-affiliated programs (4.7 ± 2.8, p = 0.0039) displayed more diversity elements than lower-ranked (4.1 ± 1.8) and community-based programs (2.8 ± 1.7).

Conclusion Most ophthalmology residency program Web sites feature less than half of the 12 diversity and inclusion elements included in this study, suggesting room for improvement. By drawing attention to program diversity and inclusion efforts, Web sites offer a potential tool for residency programs to consider in their recruitment efforts for diverse trainees. Incorporating the diversity and inclusion elements analyzed in this study represents a low-burden way to signal a greater commitment to diversity that could help programs recruit diverse applicants.

Keywords

diversity - inclusion - residency - Web site - ophthalmology

A potential strategy to mitigate health care disparities in the United States is to create a more diverse physician workforce.[1] [2] Race concordance between patient and physician has been shown to result in longer visits, better communication, increased trust, and improved patient satisfaction.[3] [4] [5] In addition, physicians from backgrounds that are underrepresented in medicine (URiM) are more likely to serve racial and ethnic minority populations that suffer from physician shortages.[6] [7] Ophthalmology departments have been stated to be among the least diverse clinical departments at United States medical schools.[8] Recently, the American Council of Graduate Medical Education (ACGME) released updated ophthalmology residency program requirements and declared that residency programs must (1) recruit and retain a diverse workforce and that residents must (2) competently treat diverse patient populations.[9]

One tool that residency programs may consider in their recruitment efforts for diverse trainees is the residency program Web site. Applicants rate residency program Web sites as an important factor when deciding where to apply to residency,[10] [11] [12] and thus program Web sites could be a venue for broadcasting a commitment to diversity and inclusion. The Web site is especially critical today given that the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) pandemic has temporarily halted in-person interviews and limited visiting away rotations, both of which provide prospective residents a chance to learn about program values that may be difficult to discern online, such as commitment to diversity and inclusion. Previous reports indicate that female applicants and URiM applicants in particular value program diversity when considering where to apply to residency.[13] [14] [15]

Prior analyses of residency program Web sites demonstrate that Web site content is inconsistent across specialties.[16] [17] [18] In addition, there is limited published data on residency Web site content that highlights diversity and inclusion initiatives. One prior study described general surgery residency programs, but not ophthalmology programs.[19] In this study, we assessed U.S. ophthalmology residency program Web sites to evaluate presence and extent of content demonstrating program diversity and inclusion elements.

Methods

The Stanford School of Medicine's Institutional Review Board declared this study exempt from oversight because all data obtained for this study were publicly available.

Program Eligibility

We used FREIDA (Fellowship and Residency Electronic Interactive Database) to obtain a list of all ophthalmology residency programs accredited by the ACGME. Military-based programs (four) and those without a functional residency program Web site (three) were excluded from analysis. Web site content analysis was completed for the 116 remaining ophthalmology residency programs between April 1, 2021 and April 29, 2021. All Web sites were accessed and evaluated by two independent medical student reviewers with an interest in applying to ophthalmology residency. A Kappa coefficient was calculated for interrater agreement. Discrepancies between raters were marked and later resolved after a conversation between the two independent reviewers.

Diversity Criteria

All residency program Web sites were evaluated for the presence of 12 criteria that demonstrate a commitment to diversity and inclusion. Eight of the elements included in our analysis were based on established methods in a recent study examining the presence of diversity elements on general surgery residency program Web sites: specifically, nondiscrimination statement, diversity and inclusion message, community resources, extended faculty biographies, extended resident biographies, faculty photos, resident photos, and additional financial resources for trainees.[19] The four remaining criteria were based on ACGME guidelines specifying that residency programs should demonstrate a commitment to wellness, mental health, health disparities, and a diverse and inclusive workforce. Specific elements included in our study were: wellness resources, mental health resources, health disparities/community engagement, and diversity council.[9] [Table 1] provides a brief explanation of each diversity element that was evaluated on residency program Web sites.

Table 1 Definition of diversity elements investigated for inclusion into ophthalmology residency program Web sites

Diversity element	Definition of element
Nondiscrimination statement	Equal opportunity or nondiscrimination statement
Diversity and inclusion message	Individualized message from program separate from nondiscrimination statement regarding diversity
Community resources	Material about community resources, demographics, or diverse groups
Extended faculty biographies	Faculty biography includes personal information about interests or background
Extended resident biographies	Resident biography includes personal information about interests or background
Faculty photos	Individual photos of each faculty member
Resident photos	Individual photos of each current resident
Additional financial resources for trainees	Program details resources for financial assistance available for residents
Wellness resources	Wellness resources available for residents
Mental health resources	Mental health/counseling resources available for residents
Health disparities/Community engagement	Community engagement initiatives aimed at addressing health disparities available for residents
Diversity council	Council committed to diversity and inclusion consisting at least partially of residents

Data Collection

Residency programs earned “credit” for the presence of each of the 12 aforementioned diversity elements if a given element was directly mentioned on the residency program Web site or if there was a direct link to a departmental or institutional Web site with the relevant information. For example, many residency program Web sites had links to their associated Graduate Medical Education office, which provided information about financial resources for trainees. Residency program Web sites also often featured extended faculty biographies indicating nonacademic hobbies and interests, etc.; analogous extended resident biographies; faculty photos; and resident photos for some but not all eligible residents and/or faculty. In these situations, programs with extended resident/faculty biographies or resident/faculty photos for greater than 50% of eligible individuals received full “credit” for the applicable diversity element.

A binary scoring system was used by reviewers to assess the presence (+1) or absence (0) of each diversity element, consistent with similar research examining residency program Web site content, to maintain objectivity.[16] [17] [18] [20] After programs were evaluated for each of the 12 elements, a total “diversity score” was calculated based on the number of individual diversity elements that were present on the program Web site, with a maximum score of 12 and a minimum score of 0.

Data Stratification and Statistical Tests

Residency programs were grouped based on program rank, size, affiliation, and geographic location. For rank, three different measures of program rank were used to minimize bias. First, Doximity 2020 rankings were used to identify the top 25 ophthalmology residency programs.[21] Next, the Ophthalmology Times 2020 rankings were used to identify the 12 “Best Overall Programs” and the 12 “Best Residency Programs.”[22] For size, programs were divided into four groups based on total number of residents (postgraduate year [PGY]-2 to PGY-4): 6 to 9, 10 to 15, 16 to 21, and 22 + . For affiliation, programs were divided into two categories: university-based/university-affiliated versus community-based. For geographic location, programs were divided into Midwest, Northeast, South, and West based on regions defined by the United States Census Bureau.[23] We compared the number of diversity and inclusion elements incorporated into residency program Web sites between programs of various size, rank, affiliation, and location using the Mann–Whitney U test or Kruskal–Wallis test, as appropriate. If a statistically significant difference was observed based on program size or geographic region, a post hoc Dunn's test was used to determine which program sizes or regions differed from each other. All statistical analyses were conducted using Microsoft Excel Version 15.21.1 and SPSS Version 26.0.0.1. p-Values of less than 0.05 were deemed significant.

Results

A total of 116 nonmilitary ophthalmology residency programs with functional Web sites were assessed for the presence of 12 diversity and inclusion elements. There was high interrater reliability with 86% agreement and a Kappa coefficient of 0.69. Program Web sites included a mean of 4.38 (standard deviation [SD] 2.14) diversity elements. The most common diversity score was 5 elements (21.6% of programs). Only 25% of programs had 50% (6) or more diversity elements and only 16.4% of programs had more than half (7 or more) of the evaluated diversity elements ([Fig. 1]).

Fig. 1 Percentage of ophthalmology residency programs with each number of diversity elements. Of the 12 diversity elements studied, residency program Web sites showcased a mean of 4.38 diversity elements.

The most common diversity elements featured on ophthalmology residency program Web sites were, respectively, resident photos (85.3%), faculty photos (78.4%), and community resources (64.3%). The least common diversity elements included extended faculty nonacademic biographies (2.6%), mental health resources (9.5%), and information about a diversity council (11.2%) ([Fig. 2]).

Fig. 2 Number of ophthalmology residency programs that display each diversity element on their Web site (n = 116 programs). Programs most commonly featured resident and faculty photos as well as additional financial resources for trainees.

When comparing programs by rank, the top 25 Doximity programs incorporated a greater number of diversity elements (mean 6.3, SD 2.1) into their residency program Web sites compared with all other programs (mean 4.1, SD 1.9) (p = 0.0003). In addition, the top 12 overall programs (mean 7.6, SD 1.8) and the top 12 residency programs (mean 7.0, SD 2.4) based on Ophthalmology Times also demonstrated a greater commitment to diversity and inclusion than other ophthalmology residency programs (p < 0.0001). When comparing programs by affiliation, university-based/university-affiliated programs (mean 4.7, SD 2.1) had higher diversity scores when compared with community-based programs (mean 2.8, SD 1.7) (p = 0.0039). There was no significant difference in total diversity score when comparing residency programs based on program size or geographic location ([Table 2]).

Table 2 Stratified analysis comparing Web site diversity content based on rank, size, affiliation, and geographic location

Program characteristic	Number of programs	Diversity score, mean (SD)	p
Doximity rank			0.0001
Top 25	25	6.3 (2.1)
Not top 25	91	4.1 (1.9)
Ophthalmology Times best overall			< 0.0001
Top 12	12	7.6 (1.8)
Not top 12	104	4.1 (1.8)
Ophthalmology Times best residency			< 0.0001
Top 12	12	7.0 (2.4)
Not top 12	104	4.1 (1.8)
Program size			0.7076
6–9	31	4.2 (2.1)
10–15	59	4.4 (2.1)
16–21	17	5 (2.1)
22+	7	4.7 (2.4)
Affiliation			0.0039
University-based/affiliated	104	4.7 (2.1)
Community-based	12	2.8 (1.7)
Geographic region			0.1031
Northeast	36	4.0 (1.9)
South	34	4.3 (2.0)
Midwest	30	4.8 (2.1)
West	15	5.5 (2.1)

Abbreviation: SD, standard deviation.

Boldface values represent statistical significance at p < 0.05.

Discussion

Residency program Web sites provide important information to applicants for decision-making about applying to programs.[10] [11] [12] [14] In this study, we assessed 116 ophthalmology residency program Web sites for the presence of 12 criteria that showcase diversity and inclusion to potential program applicants. Our results indicate that only 16% of residency program Web sites contained more than half (seven or more) of the diversity elements examined—with the majority of programs ranging from 3 to 5 diversity elements. Higher ranked programs and university-based/-affiliated programs incorporated more diversity elements into their Web sites when compared with lower ranked programs and community-based programs, respectively. The wide variation and limited content in diversity and inclusion elements displayed on program Web sites suggests that applicants may benefit from more standardized residency program Web sites that allow for easier comparison of programs when deciding where to apply. Additionally, by highlighting program diversity and inclusion efforts, Web sites offer a potential tool for residency programs to consider in their recruitment efforts for diverse trainees.[13] [14] [15]

Our findings align with a previous study that examined the presence (or lack thereof) of diversity and inclusion elements on general surgery residency program Web sites.[19] Residency program Web sites for both ophthalmology and general surgery often lacked important diversity elements including a nondiscrimination statement, a diversity and inclusion message, and extended nonacademic faculty/resident biographies. Extended faculty/resident biographies may be particularly important to female and ethnic minority applicants, who rate the presence of gender and ethnic diversity among faculty and residents as a high priority.[13] [14] [15]

In our study, more than half (56%) of ophthalmology programs listed information about additional financial resources for trainees. With average debt among medical students rising dramatically and a majority of residency applicants either borrowing money to cover interview costs (71%) or declining interviews for financial reasons (41%), residency programs that demonstrate a willingness to provide financial assistance to trainees may help attract applicants from socioeconomically disadvantaged backgrounds who otherwise might not apply due to financial constraints.[24] [25]

Fewer than 25% of residency program Web sites listed wellness resources available to residents, and less than 10% provided details on special mental health resources or counseling services available to residents. Residency program Web sites that emphasize such resources may be appealing to applicants who hope to avoid burnout and depression, both of which are of increasing concern for trainees.[26] Given that racial/ethnic minority physicians-in-training have been shown to be more hesitant to seek out mental health counseling resources, residency program Web sites that emphasize destigmatizing mental illness may prove particularly beneficial for recruiting URiM applicants and trainees.[27] [28] [29] Additionally, a commitment to wellness and mental health resources on residency program Web sites may provide insight regarding a program's vision or philosophy, which has been rated as an important factor among applicants when deciding where to apply.[30]

Only 11% of ophthalmology residency program Web sites included in this study promoted a “Diversity Council” including residents and fellows with a primary aim of improving diversity, equity, and inclusion in graduate medical education. This is a potential area for improvement; committees focused on URiM recruitment have been shown to be an effective method for helping to recruit diverse applicants.[31] Furthermore, a diversity council can provide a support network that helps URiM trainees manage the additional academic and extracurricular demands that non-URiM trainees may not experience.[32] As such, programs with established diversity councils (or programs planning to implement diversity councils in the future) may benefit from highlighting this rare yet beneficial diversity and inclusion feature on the residency program Web site.

Our finding that both top-ranked and university-based/-affiliated programs displayed more elements indicating commitment to diversity and inclusion on their Web sites may represent greater availability of resources for (1) implementing diversity and inclusion initiatives and (2) developing residency Web sites to showcase initiatives.[33] [34] However, since resource availability may be cross-cutting, increasing financial resources for diversity and inclusion initiatives may help to recruit more diverse applicants and address current diversity limitations.[35] Regional analysis of residency program Web sites revealed that programs in the West contained the highest average number of diversity elements on program Web sites when compared with programs in the Northeast, Midwest, and South regions, though this difference was not statistically significant. Notably, a disproportionate percentage of Western residency program Web sites (87%) contained information about community resources, demographics, and/or diverse patient populations when compared with residency programs from non-Western regions (61%). Furthermore, a significantly greater percentage of Western residency program Web sites (47%) contained information about community engagement initiatives aimed at addressing health disparities when compared with non-Western programs (17%).

There are several limitations to our study. First, we were unable to determine the association between presence of diversity and inclusion content on residency program Web sites and diverse residents in the program. There is minimal public information about the breakdown of residency applicants by race, gender, ethnicity, and socioeconomic status. As such, further study is necessary to determine whether increased diversity and inclusion elements on residency program Web sites correlates to increasingly diverse applicants and matched residents. Additionally, Web site contents may over- or underrepresent actual program, departmental, or institutional diversity, equity, and inclusion efforts, and presence of programs and initiatives do not necessarily reflect (or lead to) greater diversity in the faculty or trainees. Further research on the association between publicly facing displays of commitment to diversity and inclusion by residency programs on their Web sites and program implementation and/or execution of those commitments is warranted. Another limitation is that we assessed for the presence or absence of diversity and inclusion elements on residency program Web sites but did not analyze the quality of the elements presented. However, both independent reviewers agreed that higher scoring programs tended to possess increased quantity and increased quality of diversity elements when compared with lower scoring programs—reducing the likelihood of a situation where a program inappropriately received a high score. Additionally, the binary scoring system used in this study is similar to other studies examining residency program content analysis.[16] [17] [18] [20] Finally, although the 12 diversity and inclusion elements included in this study are not a validated measure of program commitment to diversity and inclusion, these criteria have been utilized previously to evaluate residency program Web sites and are based on updated ACGME guidelines.[9] [19]

In summary, of the 116 nonmilitary ophthalmology residency program Web sites examined, only 16% included more than half of the evaluated diversity and inclusion elements in this study, suggesting potential for residency programs to further their commitment to diversity and inclusion initiatives online. Not only the existence of diversity and inclusion initiatives but also inclusion of information on program Web sites, such as a nondiscrimination statement, diversity and inclusion message, and mental health/wellness resources may help to better convey the program's commitment to diversity and inclusion, which can help to recruit more diverse applicants. Furthermore, the residency program Web site can be an especially vital communication tool with potential applicants as a result of the shift to virtual interviews due to the SARS-CoV-2 pandemic, where opportunities for applicants to engage with program residents, faculty, and staff to gauge a program's commitment to diversity and inclusion are limited. An increased emphasis and communication around diversity and inclusion among ophthalmology residency programs may help to facilitate productive discussions regarding the gender, socioeconomic, and racial/ethnic disparities that currently exist among ophthalmology residents and in the ophthalmology workforce at large.

Conflict of Interest

None declared.

Acknowledgments

None.

Financial Disclosures

No financial disclosures.

• References

• 1 Maldonado ME, Fried ED, DuBose TD, Nelson C, Breida M. The role that graduate medical education must play in ensuring health equity and eliminating health care disparities. Ann Am Thorac Soc 2014; 11 (04) 603-607
• 2 New policy aimed at increasing diversity in physician workforce. American Medical Association. Accessed February 10, 2021 at: https://www.ama-assn.org/press-center/press-releases/new-policy-aimed-increasing-diversity-physician-workforce
• 3 Street Jr RL, O'Malley KJ, Cooper LA, Haidet P. Understanding concordance in patient-physician relationships: personal and ethnic dimensions of shared identity. Ann Fam Med 2008; 6 (03) 198-205
• 4 Saha S, Komaromy M, Koepsell TD, Bindman AB. Patient-physician racial concordance and the perceived quality and use of health care. Arch Intern Med 1999; 159 (09) 997-1004
• 5 Takeshita J, Wang S, Loren AW. et al. Association of racial/ethnic and gender concordance between patients and physicians with patient experience ratings. JAMA Netw Open 2020; 3 (11) e2024583
• 6 Komaromy M, Grumbach K, Drake M. et al. The role of black and Hispanic physicians in providing health care for underserved populations. N Engl J Med 1996; 334 (20) 1305-1310
• 7 Moy E, Bartman BA. Physician race and care of minority and medically indigent patients. JAMA 1995; 273 (19) 1515-1520
• 8 Fairless EA, Nwanyanwu KH, Forster SH, Teng CC. Ophthalmology departments remain among the least diverse clinical departments at United States medical schools. Ophthalmology 2021; 128 (08) 1129-1134
• 9 Accreditation Council for Graduate Medical Education. Common Program Requirements. Accessed February 9, 2021 at: https://www.acgme.org/Specialties/Overview/pfcatid/13/Ophthalmology/
• 10 Jarman BT, Joshi ART, Trickey AW, Dort JM, Kallies KJ, Sidwell RA. Factors and influences that determine the choices of surgery residency applicants. J Surg Educ 2015; 72 (06) e163-e171
• 11 Embi PJ, Desai S, Cooney TG. Use and utility of Web-based residency program information: a survey of residency applicants. J Med Internet Res 2003; 5 (03) e22
• 12 Long T, Dodd S, Licatino L, Rose S. Factors important to anesthesiology residency applicants during recruitment. J Educ Perioper Med 2017; 19 (02) E604
• 13 Yousuf SJ, Kwagyan J, Jones LS. Applicants' choice of an ophthalmology residency program. Ophthalmology 2013; 120 (02) 423-427
• 14 Phitayakorn R, Macklin EA, Goldsmith J, Weinstein DF. Applicants' self-reported priorities in selecting a residency program. J Grad Med Educ 2015; 7 (01) 21-26
• 15 Aagaard EM, Julian K, Dedier J, Soloman I, Tillisch J, Pérez-Stable EJ. Factors affecting medical students' selection of an internal medicine residency program. J Natl Med Assoc 2005; 97 (09) 1264-1270
• 16 Svider PF, Gupta A, Johnson AP. et al. Evaluation of otolaryngology residency program websites. JAMA Otolaryngol Head Neck Surg 2014; 140 (10) 956-960
• 17 Oladeji LO, Yu JC, Oladeji AK, Ponce BA. How useful are orthopedic surgery residency web pages?. J Surg Educ 2015; 72 (06) 1185-1189
• 18 Skovrlj B, Silvestre J, Ibeh C, Abbatematteo JM, Mocco J. Neurosurgery residency websites: a critical evaluation. World Neurosurg 2015; 84 (03) 727-733
• 19 Driesen AMDS, Romero Arenas MA, Arora TK. et al. Do general surgery residency program websites feature diversity?. J Surg Educ 2020; 77 (06) e110-e115
• 20 Shaath DS, Whittaker TJ. Evaluation of ophthalmology residency program web sites. J Acad Ophthalmol 2019; 11: e44-e48
• 21 Doximity Residency Navigator - Ophthalmology Programs. Doximity. Accessed February 9, 2021 at: https://www.doximity.com/residency/programs?_remember_me_attempted=yes&location_type=region&residency_specialty_id=50&sort_by=reputation&x0026=
• 22 2020 surprise: Bascom Palmer, Wilmer Eye tie for top spot. Ophthalmology Times. Accessed February 9, 2021 at: https://www.ophthalmologytimes.com/view/2020-best-programs-bascom-palmer-wilmer-eye-tie-top-spot
• 23 United States, Bureau of the Census, Statistical Abstract of the United States, 1993. U.S. Dept. of Commerce, Bureau of the Census. Accessed June 3, 2021 at: https://www.census.gov/library/publications/1993/compendia/statab/113ed.html
• 24 Fogel HA, Liskutin TE, Wu K, Nystrom L, Martin B, Schiff A. The economic burden of residency interviews on applicants. Iowa Orthop J 2018; 38: 9-15
• 25 Craft III JA, Craft TP. Rising medical education debt a mounting concern. Graduates also face less favorable repayment terms, shortage of training positions. Mo Med 2012; 109 (04) 266-270
• 26 Brazeau CMLR, Shanafelt T, Durning SJ. et al. Distress among matriculating medical students relative to the general population. Acad Med 2014; 89 (11) 1520-1525
• 27 Ey S, Moffit M, Kinzie JM, Choi D, Girard DE. “If you build it, they will come”: attitudes of medical residents and fellows about seeking services in a resident wellness program. J Grad Med Educ 2013; 5 (03) 486-492
• 28 Vogel DL, Wester SR, Larson LM. Avoidance of counseling: psychological factors that inhibit seeking help. J Couns Dev 2007; 85: 410-422
• 29 Lambert MJ. Bergin and Garfield's Handbook of Psychotherapy and Behavior Change. Hoboken, New Jersey: John Wiley & Sons; 2013
• 30 Chu LF, Young CA, Zamora AK. et al. Self-reported information needs of anesthesia residency applicants and analysis of applicant-related web sites resources at 131 United States training programs. Anesth Analg 2011; 112 (02) 430-439
• 31 Auseon AJ, Kolibash Jr AJ, Capers Q. Successful efforts to increase diversity in a cardiology fellowship training program. J Grad Med Educ 2013; 5 (03) 481-485
• 32 Dickins K, Levinson D, Smith SG, Humphrey HJ. The minority student voice at one medical school: lessons for all?. Acad Med 2013; 88 (01) 73-79
• 33 Noble P, Ten Eyck P, Roskoski Jr R, Jackson JB. NIH funding trends to US medical schools from 2009 to 2018. PLoS One 2020; 15 (06) e0233367
• 34 Carney PA, Waller E, Green LA. et al. Financing residency training redesign. J Grad Med Educ 2014; 6 (04) 686-693
• 35 Walker V. The road to nonprofit diversity and inclusion. J Infect Dis 2019; 220 (220, Suppl 2): S86-S90

Publication History

Received: 04 September 2021

Accepted: 24 November 2021

Article published online:
19 April 2022