Eine Alternative zur klassischen Testtheorie? – Eine exemplarische Anwendung der Generalisierbarkeitstheorie auf der Basis von Sekundärdaten

Zusammenfassung

Hintergrund Messungen bilden die Grundlage des wissenschaftsbasierten therapeutischen Vorgehens. Messfehler sind jedoch ein verbreitetes Problem. Um Fehlerquellen zu ermitteln, sind erweiterte statistische Modelle notwendig. Im Gegensatz zur klassischen Testtheorie ermöglicht die Generalisierbarkeitstheorie, mehrere Fehlerquellen gleichzeitig zu untersuchen.

Ziel Vergleich der klassischen Testtheorie und der Generalisierbarkeitstheorie anhand eines physiotherapeutischen Beispiels zur Reliabilitätsermittlung und deren Ergebnisse.

Methode Die Messeigenschaft „Reliabilität“ wurde anhand von Daten einer vorangegangenen Reliabilitätsstudie, in der die klassische Testtheorie zum Einsatz kam, untersucht. Dazu wurde exemplarisch eine Generalisierbarkeitsstudie durchgeführt.

Ergebnisse Die Variabilität ging hauptsächlich von den Probanden und der Probanden-Untersucher-Interaktion aus. Unabhängig von fixen oder zufälligen Facetten waren die Generalisierbarkeitskoeffizienten für alle Bedingungen (overall, inter-rater, intra-Messzeitpunkte) exzellent.

Schlussfolgerung Die Ergebnisse deuten darauf hin, dass die Generalisierbarkeitstheorie gegenüber der klassischen Testtheorie Vorteile hat. Diese ermöglicht es, einen wirkungsvollen und effizienten Einsatz von Messinstrumenten im klinischen Alltag zu ermitteln.

Abstract

Background Applying repeated measurements, errors can arise from multiple sources, which affects the outcome. In order to determine these sources of error, an expanded statistical model like the generalizability theory framework is required. In contrast to the classic test theory, it enables the simultaneous investigation of several sources of error.

Objective To demonstrate an exemplary application of generalizability theory in a physiotherapy setting based on an isometric muscle strength test as well as the comparison of two different approaches, the classical test theory and the generalizability theory.

Method Examinizing the clinimetric property „Reliability” based on data of a previous reliability study in which classic test theory was used. For this purpose, an exemplary generalizability study was conducted.

Results Variances resulted mainly from subjects and interactions between subjects and raters. Generalizability coefficients were excellent for all conditions (overall, inter-rater, inter-time point), regardless of the type of facets (fix or random facets).

Conclusion The results of the exemplary generalizability study shows that generalizability theory has advantages over classic test theory. Generalizability theory offers an approach to determine and implement effective and efficient use of measurement protocols for physiotherapy practice.

Schlüsselwörter

Generalisierbarkeitstheorie - Reliabilität - Generalisierbarkeitsstudie - Physiotherapie

Key words

generalizability theory - reliability - generalizability study - physiotherapy

• Literatur

• 1 Oesch P, Eberhardt R. Bewegungsapparat. 3., überarbeitete und ergänzte Aufl. Bern: Hogrefe; 2017
• 2 Vet HCWd. Measurement in Medicine: A Practical Guide. Cambridge: University Press; 2011.
• 3 Denegar CR, Ball DW. Assessing Reliability and Precision of Measurement: An Introduction to Intraclass Correlation and Standard Error of Measurement. 1993; 2: 35 . doi:10.1123/jsr.2.1.35
• 4 Shrout PE, Fleiss JL. Intraclass correlations: uses in assessing rater reliability. Psychol Bull 1979; 86: 420-428
• 5 Streiner DL. Health measurement scales: a practical guide to their development and use. 5th ed.. Oxford: University Press; 2015
• 6 Naizer G. Basic concepts in generalizability theory: a more powerful approach to evaluating reliability. Paper, präsented Anual Meeting Southwest Edu Research Asociation. Houston, TX: 1992
• 7 Shavelson RL, Webb NM. Generalizability Theory: A Primer. Thousand Oaks, CA: Sage; 1991
• 8 Brennan RL. Generalizability theory. New York: Springer; 2001
• 9 de Bruin ED, Rozendal RH, Stussi E. Reliability of phase-velocity measurements of tibial bone. Phys Ther 1998; 78: 1166-1174 . doi:10.1093/ptj/78.11.1166
• 10 Heitman RJ, Kovaleski JE, Pugh SF. Application of generalizability theory in estimating the reliability of ankle-complex laxity measurement. J Athl Train 2009; 44: 48-52 . doi:10.4085/1062-6050-44.1.48
• 11 Fleiss JL. Design and Analysis of Clinical Experiments. Wiley; 2011
• 12 Baschung Pfister P, de Bruin ED, Sterkele I. et al Manual muscle testing and hand-held dynamometry in people with inflammatory myopathy: An intra- and interrater reliability and validity study. PloS one 2018; 13: e0194531 . doi:10.1371/journal.pone.0194531
• 13 Bates D, Mächler M, Bolker B. et al Fitting Linear Mixed-Effects Models Using lme4. Journal of Statistical Software 2015; 67: 48 . doi:10.18637/jss.v067.i01
• 14 R Core Team. A language and environment for statistical computing. R Foundation for Statistical Computing. Vienna: 2019
• 15 Roebroeck ME, Harlaar J, Lankhorst GJ. The application of generalizability theory to reliability assessment: an illustration using isometric force measurements. Phys Ther 1993; 73: 386-395 ; discussion 396–401. doi:10.1093/ptj/73.6.386
• 16 Gagnon D, Nadeau S, Gravel D. et al Reliability and validity of static knee strength measurements obtained with a chair-fixed dynamometer in subjects with hip or knee arthroplasty. Arch Phys Med Rehabil 2005; 86: 1998-2008 . doi:10.1016/j.apmr.2005.04.013
• 17 Anthony A, Gatti PWS, Nicholas MB. et al. How to Optimize Measurement Protocols: An Example of Assessing Measurement Reliability Using Generalizability Theory.
• 18 Macintyre NJ, Bennett L, Bonnyman AM. et al Optimizing reliability of digital inclinometer and flexicurve ruler measures of spine curvatures in postmenopausal women with osteoporosis of the spine: an illustration of the use of generalizability theory. ISRN Rheumatol 2011; 571698 . doi:10.5402/2011/571698
• 19 Bauer CM, Rast FM, Ernst MJ. et al Concurrent validity and reliability of a novel wireless inertial measurement system to assess trunk movement. J Electromyogr Kinesiol 2015; 25: 782-790 . doi:10.1016/j.jelekin.2015.06.001
• 20 Bauer CM, Heimgartner M, Rast FM. et al Reliability of lumbar movement dysfunction tests for chronic low back pain patients. Man Ther 2016; 24: 81-84 . doi:10.1016/j.math.2016.02.013
• 21 Lafave M, Butterwick D. A generalizability theory study of athletic taping using the Technical Skill Assessment Instrument. doi:10.4085/1062-6050-49.2.22. Im Internet (Stand: 29.02.2020): www.cochranelibrary.com/central/doi/10.1002/central/CN-01115682/full
• 22 Levac D, Missiuna C, Wishart L. et al The motor learning strategy instrument: interrater reliability within usual and virtual reality physical therapy interventions. Pediatr Phys Ther 2013; 25: 53-60 . doi:10.1097/PEP.0b013e3182750c28
• 23 Watkins B, Darrah J, Pain K. Reliability of passive ankle dorsiflexion measurements in children: comparison of universal and biplane goniometers. Im Internet (Stand: 29.02.2020): www.cochranelibrary.com/central/doi/10.1002/central/CN-00182435/full
• 24 Ernst MJ, Sommer BB, Meichtry A. et al Intra-rater reliability of determining positions of cervical spinous processes and measuring their relative distances: An update to define rigid bodies of the cervical spine in a movement laboratory setting. BMC research notes 2019; 12: 265 . doi:10.1186/s13104-019-4299-8
• 25 Lariviere C, Ludvig D, Kearney R. et al Identification of intrinsic and reflexive contributions to low-back stiffness: medium-term reliability and construct validity. J Biomech 2015; 48: 254-261 . doi:10.1016/j.jbiomech.2014.11.036
• 26 Pryseley A, Ledent EY, Drewes AM. et al Applying concepts of generalizability theory on data from experimental pain studies to investigate reliability. Basic & clinical pharmacology & toxicology 2009; 105: 105-112 . doi:10.1111/j.1742-7843.2009.00408.x
• 27 Rast FM, Graf ES, Meichtry A. et al Between-day reliability of three-dimensional motion analysis of the trunk: A comparison of marker based protocols. J Biomech 2016; 49: 807-811 . doi:10.1016/j.jbiomech.2016.02.030
• 28 Swaine BR, Lortie E, Gravel D. The reliability of the time to execute various forms of the finger-to-nose test in healthy subjects. Physiotherapy theory and practice 2005; 21: 271-279 . doi:10.1080/09593980500321119
• 29 Fortin C, Feldman DE, Cheriet F. et al Reliability of a quantitative clinical posture assessment tool among persons with idiopathic scoliosis. Physiotherapy 2012; 98: 64-75 . doi:10.1016/j.physio.2010.12.006
• 30 Sommer BB, Weisenhorn M, Ernst MJ. et al Concurrent validity and reliability of a mobile tracking technology to measure angular and linear movements of the neck. J Biomech 2019; 96: 109340 . doi:10.1016/j.jbiomech.2019.109340
• 31 Vangeneugden T, Laenen A, Geys H. et al Applying concepts of generalizability theory on clinical trial data to investigate sources of variation and their impact on reliability. Biometrics 2005; 61: 295-304 . doi:10.1111/j.0006-341X.2005.031040.x
• 32 Vispoel WP, Morris CA, Kilinc M. Applications of generalizability theory and their relations to classical test theory and structural equation modeling. Psychol Methods 2018; 23: 1-26 . doi:10.1037/met0000107