Download PDF
Methods Inf Med 1990; 29(02): 153-157
DOI: 10.1055/s-0038-1634777
Inter-observer Variation
Schattauer GmbH

Inter-Observer Variation in Decision-Making Regarding Patients on Chronic Hemodialysis: A Study Using the Kappa Index

T. Timpka
1   Departments of Computer and Information Science, Medical Informatics, and Community Medicine
,
T. Buur
2   Department of Nephrology, Linköping, University, Sweden
› Author Affiliations
Further Information

Publication History

Publication Date:
06 February 2018 (online)

    Permissions and Reprints

Abstract

To examine inter-observer variation in the monitoring of patients with chronic disease, four nephrologists independently assessed 62 patients on maintenance hemodialysis. Deviation from normal was determined for adequacy of dialysis, protein intake, and metabolic state. The kappa-index, which allows to adjust for chance agreement, was used to analyze each monitoring diagnosis. Low agreement was found on decisions concerning adequacy of dialysis (kappa 0.12-0.26), while agreement was higher about protein intake (kappa 0.21-0.46), and metabolic state (kappa 0.24-0.52). Two physicians classified no patient as overdialyzed, while 16-18% were thus categorized by the other two. Routines for review of recent medical history also differed significantly between the physicians. Measures are needed to increase the reliability of decisions regarding the monitoring of chronic hemodialysis. A long lasting physician-patient relationship is not a sufficient prerequisite for diminishing decision variation. Medical audit as part of the clinical routine, and use of additional sources of information, exemplified by urea kinetic modeling, are discussed.

Key-Words

Medical Decision Making - Inter-Observer Variation - Hemodialysis - Kappa Analysis - Decision Support

* The kappa-index values can be interpreted as uncertain agreement for values in range (0.00–0.19), low agreement (0.20–0.39), moderate agreement (0040–0.59), substantial agreement (0.60–0.79) and almost perfect agreement (> 0.80) [5].


 

  • REFERENCES

  • 1 Cohen J. A coefficient of agreement for nominal scales. Educ Psychol Meas 1960; 20: 37-46.
    CrossrefPubMedGoogle Scholar
  • 2 Koran LM. The reliability of clinical methods, data and judgement. N Engl J Med. 1975 293. 642-6, 695-700..
    PubMedGoogle Scholar
  • 3 Corazza GR, Bonvicini F, Frazzoni M, Gatto M, Gasbarrini G. Observer variation in assessment of jejunal biopsy specimen. A comparison between subjective criteria and morphometric measurement. Gastroenterology 1982; 83: 1217-22.
    PubMedGoogle Scholar
  • 4 Timpka T, Bjurulf P. The semantics of diagnosis and management of genitourinary infections: A cross-specialty study. Fam Pract 1989; 06: 279-85.
    CrossrefPubMedGoogle Scholar
  • 5 LandiS JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics 1977; 33: 159-74.
    CrossrefPubMedGoogle Scholar
  • 6 Hartling OJ, Höilund-Carisen PG, Hesse B. et al. Evaluation of intravenous radionuclide angiography in detection of occlusive disease of the iliac arteries. Europ J Radiol 1984; 04: 205-9.
    PubMedGoogle Scholar
  • 7 Gjörup T, Agner E, Jensen LB, Jensen AM, MöIlman KM. The endoscopic diagnosis of duodenal ulcer disease. Scand J Gastroenterology 1986; 21: 261-7.
    CrossrefPubMedGoogle Scholar
  • 8 Theodossi A, Knill-Jones RP, Skene A. et al. Inter-observer variation of symptoms and signs in jaundice. Liver 1981; 01: 21-32.
    PubMedGoogle Scholar
  • 9 Quinton WE, Dillard D, Scribner BH. Cannulation of blood vessels for prolonged hemodialysis. Trans Am Soc Artif Intern Organs 1963; 09: 67-72.
    PubMedGoogle Scholar
  • 10 Schupak E, Merrill JP. Experience with long-term intermittent hemodialysis. Ann Intern Med 1965; 62: 509-14.
    CrossrefPubMedGoogle Scholar
  • 11 Theodossi A, Skene AM, Portman B. et al. Observer variation in assessment of liver biopsies including analyses by kappa statistics. Gastroenterology 1980; 79: 232-41.
    PubMedGoogle Scholar
  • 12 Avram MM, Slater PA, Gan A. et al. PredialysisBUN and creatinine do not predict adequate dialysis, clinical rehabilitation, or longevity. Kidney Int 1985; 28 (Suppl. 17) S100-5.
    PubMedGoogle Scholar
  • 13 Shapiro JI, Argy wP, Rakowski TA, Chester A, Siemsen AS. Schreiner GE. The unsuitability of BUN as a criterion for prescription of dialysis. Trans Am Soc Artif Intern Organs 1983; 29: 286-92.
    PubMedGoogle Scholar
  • 14 Gotch FA, Sargent JA. A mechanistic analysis of the National Cooperative Dialysis Study (NCDS). Kidney Int 1985; 28: 526-34.
    CrossrefPubMedGoogle Scholar
  • 15 Dougirdas JT, Dumler F, Zasuwa GA, Levin Nw. Chronic hemodialysis prescription. In: Daugirdas IT, Ing TS. eds. Handbook of Dialysis. Boston: Little, Brown and Co; 1988: 72-86.
    Google Scholar
  • 16 Musch DC, Higgins ITT, Landis JR. Some factors influencing interobserver variation in classifying simple pneumoconiosis. Br J Ind Med 1985; 42: 346-9.
    PubMedGoogle Scholar
  • 17 Shaw CD, Costain DW. Guidelines for medical audit: seven principles. Br Med J 1989; 299: 488-9.
    PubMedGoogle Scholar
  • 18 Gjörup T, Hamberg O, Knudsen J. et al. Does the patient appear acutely or chronically ill? An interobserver study of global assessment of hospital patients. Acta Med Scand 1982; 212: 325-8.
    PubMedGoogle Scholar
  • 19 Gjorup T. The kappa coefficient and the prevalence of a diagnosis. Meth Inform Med 1988; 27: 184-6.
    PubMedGoogle Scholar
  • 20 Buur T. An advanced, user-friendly microcomputer program for hemodialysis kinetics. Adv Exp Med Bioi 1987; 223: 239-44.
    PubMedGoogle Scholar