Standardized Training Examinations among Ophthalmology Residents and the American Board of Ophthalmology Written Qualifying Examination First Attempt: The Morsani College of Medicine Experience
23 December 2014 (online)
Purpose The aim of the study is to demonstrate whether resident performance on the United States Medical Licensing Exam (USMLE) Step 1 and the Ophthalmology Knowledge Assessment Program (OKAP) correlates to American Board of Ophthalmology Written Qualifying Examination (ABO-WQE).
Design This is a retrospective cohort study.
Participants Forty-one residents who completed their postgraduate training between 1999 and 2011 from our institution were included in the study.
Methods Percentiles on the USMLE Step 1, OKAP, and ABO-WQE first attempt were recorded and analyzed for possible correlations among scores across examinations using Pearson rank correlation, chi-square analysis, Fisher exact test, and Student t-test.
Results The percentile score on the USMLE Step 1 was the only standardized test that had a statistically significant association with the percentile score on the ABO-WQE (Pearson rank coefficient = 0.45, p = 0.01). Not achieving a score of 20th percentile on the OKAP examination among second-year residents had the highest statistical association with failing the ABO-WQE on first attempt (p = 0.001) with an odds ratio of 20.4, and the relative risk of failing the ABO-WQE was 11.3; however, the relative risk of passing if the cutoff was achieved was only 1.8, and a positive predictive value for failing the ABO-WQE on first attempt of 0.47.
Conclusions Percentile scores on the OKAP are not indicative of percentile scores on the ABO-WQE. While OKAP examinations in the second and third year appear to separate out residents at risk of failing the ABO-WQE, there is a large disparity between the risk of passing and failing which is likely due to the inadequate prognostic value of the test. It is likely OKAP examinations may be a useful tool to identify residents who are at low risk for failing the ABO-WQE on first attempt, but much less useful for identifying residents who are at high risk for failing the ABO-WQE.
- 1 Graff JM, Oetting TA, Lee AG. The Ophthalmic Knowledge Assessment Program (OKAP) Examination and global evaluation of resident performance. J Acad Ophthalmol 2008; 1 (1) 20-24
- 2 Chen PP, Bhandari A. Resident performance on the Ophthalmic Knowledge Assessment Program and the Written Qualifying Examination for board certification. J Acad Ophthalmol 2010; 3 (1) 11-14
- 3 Johnson GA, Bloom JN, Szczotka-Flynn L, Zauner D, Tomsak RL. A comparative study of resident performance on standardized training examinations and the American Board of Ophthalmology Written Examination. Ophthalmology 2010; 117 (12) 2435-2439
- 4 Lee AG, Oetting TA, Blomquist PH , et al. A multicenter analysis of the ophthalmic knowledge assessment program and American Board of Ophthalmology written qualifying examination performance. Ophthalmology 2012; 119 (10) 1949-1953
- 5 O'Day DM, Li C. First-time failure rates of candidates for board certification: an educational outcome measure. Arch Ophthalmol 2008; 126 (4) 548-553