Impact of inexperienced examiners on arterio-venous ratio of static vessel analysis

Background:

Nonmydriatic retinal imaging is widely used in large epidemiological studies because static retinal vessel analysis (SVA) is a promising concept for interdisciplinary research on ocular and cardiovascular diseases. Accuracy is important to correctly measure vessel diameters and to determine arterio-venous ratio (AVR). SVA emerges into non-reading center environments and health services without scientific background, in which inexperienced or inadequately instructed raters might bias results of SVA. Our study aims to assess the impact of inexperienced raters on SVA.

Methods:

One experienced professional rater and four inexperienced raters performed SVA with a set of digital fundus images from 25 subjects. All images were taken with a non-mydriatic camera. The test set was chosen randomly from pictures taken in the Study of Health in Pomerania, a large-scale population-based study. All raters used the computer-assisted ARIC protocol of the software Vesselmap (Imedos, Jena, Germany) to determine AVR. Interrater variability was evaluated by computing the intraclass correlation coefficient (ICC) for each inexperienced rater and the expert. Bland-Altman plots were used to graphically visualize variation between the expert and each of the inexperienced raters.

Results:

The average ICC for interrater variability between one expert and four inexperienced raters was 79%± 5% for AVR. The average difference was 2.6%± 1.4% in Bland-Altman plots. The specificity for an AVR cutoff of 0.8 was 100%, whereas the sensitivity ranged between 28.6% and 57.1%.

Conclusions:

Even a brief training in inexperienced raters was sufficient to obtain reliable AVR measurements. The continuous measurements may sensibly be used as a predictor or outcome in cardiovascular research. However misclassification, when using a clinically relevant cutpoint is an important issue and would require further training.