Download PDF
Methods Inf Med 1991; 30(02): 96-101
DOI: 10.1055/s-0038-1634822
Statistical Applications
Schattauer GmbH

Distribution-Free Confidence Bounds for ROC Curves

R. A. Hilgers
1   Department of Medical Statistics, University of Göttingen, Gottingen, Germany
› Author AffiliationsThe data of our practical example for normals and patients with thyroid autonomy are used with kind permission of Prof. Dr. Emrich, Head of the Department of Nuclear Medicine, University of Göttingen.
Further Information

Publication History

Publication Date:
07 February 2018 (online)

    Permissions and Reprints

Abstract

ROC curves are widely used for the evaluation of diagnostic tests to decide between “healthy” and “diseased” individuals when the measurements are on a continuous scale. These curves are graphical displays of the interdependence between specificity and sensitivity of the test varying with the cut-off point chosen for the decision. Up to now only point estimators derived from the empirical distribution functions are used which may be misleading if they are based on rather small samples. In this paper we propose reasonable confidence bounds for ROC curves and a corresponding point estimator. Our bounds are strongly related to two-sided distribution-free tolerance regions because they are can, structed from minimum and maximum coverages which at a given value.t can be guaranteed with a confidence (2π* - 1). The interpretation of the bounds is that if a cut-off-point is chosen on the basis of the ROC curve then with a nominal confidence of at least (2π* -1)2 the real sensitivity and specificity will be within a rectangle.

Key-Words

Diagnostic Test - Sensitivity - Specificity - ROC Curve - Nonparametric Confidence Bounds - Distribution-Free Tolerance Regions
 

  • REFERENCES

  • 1 Galen RS, Gambino SR. Beyond Normality: The Predictive Value and Efficiency of Medical Diagnosis. New York: John Wiley & Sons; 1975
    Google Scholar
  • 2 Egan JP. Signal Detection Theory and ROC Analysis. New York: Academic Press; 1975
    Google Scholar
  • 3 Wilks SS. Determination of sample sizes for setting tolerance limits. Ann Math Statist 1941; 12: 91-6.
    CrossrefPubMedGoogle Scholar
  • 4 Wald A. An extension of Wilk’s method for setting tolerance limits. Ann Math Statist 1943; 14: 45-55.
    CrossrefPubMedGoogle Scholar
  • 5 Fraser DAS. Sequentially determined statistically equivalent blocks. Ann Math Statist 1951; 22: 372-81.
    CrossrefPubMedGoogle Scholar
  • 6 Fraser DAS. Nonparametric tolerance regions. Ann Math Statist 1953; 24: 44-55.
    CrossrefPubMedGoogle Scholar
  • 7 Tukey JW. Non-Parametric Estimation II. Statistically equivalent blocks and tolerance regions - the continuous case. Ann Math Statist 1947; 18: 529-39.
    CrossrefPubMedGoogle Scholar
  • 8 Tukey Jw. Non-Parametric Estimation III. Statistically equivalent blocks and multivariate tolerance regions – the discontinuous case Ann Math Statist 1948; 19: 30-9.
    CrossrefPubMedGoogle Scholar
  • 9 Murphy RB. Non-Parametric Tolerance Limits. Ann Math Statist 1948; 19: 581-9.
    CrossrefPubMedGoogle Scholar
  • 10 Sandford MD. Nonparametric one-sided confidence intervals for an unknown distribution function using censored data. Technometrics 1985; 27: 41-8.
    CrossrefPubMedGoogle Scholar
  • 11 Kendall MG, Stuart A. The Advanced Theory of Statistics. Vol. 2 3rd ed. London: Griffin; 1975
    Google Scholar
  • 12 Abramowitz M, Stegun IA. eds. Handbook of Mathematical Functions with Formulas, Graphs, and Mathematical Tables (8th ed). New York: Dover Publ; 1972
    Google Scholar
  • 13 Majumder KL, Bhattarcharjee GP. AS64/AS109. Inverse of the incomplete Beta Function ratio. In: Griffiths P, Hill ID. eds. Applied Statistics Algorithms. Chichester. Ellis Horwood; 1985
    Google Scholar
  • 14 SAS Institute Inc. SAS Language Guide for Personal Computers (Version 6). Cary, NC: SAS Institute Inc; 1985
    Google Scholar