Download PDF
physioscience 2021; 17(01): 25-33
DOI: 10.1055/a-1201-6872
Originalarbeit

Eine Alternative zur klassischen Testtheorie? – Eine exemplarische Anwendung der Generalisierbarkeitstheorie auf der Basis von Sekundärdaten

An Alternative to Classic Test Theory? – An Exemplary Application of Generalizability Theory-based on Secondary Data
Iris Sterkele
1   Physiotherapie Ergotherapie USZ, Universitätsspital Zürich, Zürich, Schweiz
,
Pierrette Baschung Pfister
1   Physiotherapie Ergotherapie USZ, Universitätsspital Zürich, Zürich, Schweiz
2   Forschungszentrum Physiotherapie Ergotherapie USZ, Universitätsspital Zürich, Zürich, Schweiz
,
Ruud Knols
1   Physiotherapie Ergotherapie USZ, Universitätsspital Zürich, Zürich, Schweiz
2   Forschungszentrum Physiotherapie Ergotherapie USZ, Universitätsspital Zürich, Zürich, Schweiz
3   Departement Gesundheitswissenschaften und Technologie, Institut für Bewegungswissenschaften und Sport, ETH Zürich, Zürich, Schweiz
,
Eling D de Bruin
3   Departement Gesundheitswissenschaften und Technologie, Institut für Bewegungswissenschaften und Sport, ETH Zürich, Zürich, Schweiz
4   Division of Physiotherapy, Department of Neurobiology, Care Science and Society, Karolinska Institute, Stockholm, Sweden
5   Research Line Functioning and Rehabilitation, Department of Epidemiology, School CAPHRI Care and Public Health Research Institute, Maastrich University, The Netherland
› Author Affiliations
› Further Information
   Permissions and Reprints

Zusammenfassung

Hintergrund Messungen bilden die Grundlage des wissenschaftsbasierten therapeutischen Vorgehens. Messfehler sind jedoch ein verbreitetes Problem. Um Fehlerquellen zu ermitteln, sind erweiterte statistische Modelle notwendig. Im Gegensatz zur klassischen Testtheorie ermöglicht die Generalisierbarkeitstheorie, mehrere Fehlerquellen gleichzeitig zu untersuchen.

Ziel Vergleich der klassischen Testtheorie und der Generalisierbarkeitstheorie anhand eines physiotherapeutischen Beispiels zur Reliabilitätsermittlung und deren Ergebnisse.

Methode Die Messeigenschaft „Reliabilität“ wurde anhand von Daten einer vorangegangenen Reliabilitätsstudie, in der die klassische Testtheorie zum Einsatz kam, untersucht. Dazu wurde exemplarisch eine Generalisierbarkeitsstudie durchgeführt.

Ergebnisse Die Variabilität ging hauptsächlich von den Probanden und der Probanden-Untersucher-Interaktion aus. Unabhängig von fixen oder zufälligen Facetten waren die Generalisierbarkeitskoeffizienten für alle Bedingungen (overall, inter-rater, intra-Messzeitpunkte) exzellent.

Schlussfolgerung Die Ergebnisse deuten darauf hin, dass die Generalisierbarkeitstheorie gegenüber der klassischen Testtheorie Vorteile hat. Diese ermöglicht es, einen wirkungsvollen und effizienten Einsatz von Messinstrumenten im klinischen Alltag zu ermitteln.

Abstract

Background Applying repeated measurements, errors can arise from multiple sources, which affects the outcome. In order to determine these sources of error, an expanded statistical model like the generalizability theory framework is required. In contrast to the classic test theory, it enables the simultaneous investigation of several sources of error.

Objective To demonstrate an exemplary application of generalizability theory in a physiotherapy setting based on an isometric muscle strength test as well as the comparison of two different approaches, the classical test theory and the generalizability theory.

Method Examinizing the clinimetric property „Reliability” based on data of a previous reliability study in which classic test theory was used. For this purpose, an exemplary generalizability study was conducted.

Results Variances resulted mainly from subjects and interactions between subjects and raters. Generalizability coefficients were excellent for all conditions (overall, inter-rater, inter-time point), regardless of the type of facets (fix or random facets).

Conclusion The results of the exemplary generalizability study shows that generalizability theory has advantages over classic test theory. Generalizability theory offers an approach to determine and implement effective and efficient use of measurement protocols for physiotherapy practice.

Schlüsselwörter

Generalisierbarkeitstheorie - Reliabilität - Generalisierbarkeitsstudie - Physiotherapie

Key words

generalizability theory - reliability - generalizability study - physiotherapy

Supplement



Publication History

Received: 17 June 2020

Accepted: 03 September 2020

Article published online:
03 February 2021

© 2021. Thieme. All rights reserved.

Georg Thieme Verlag KG
Rüdigerstraße 14, 70469 Stuttgart, Germany

 

  • Literatur

  • 1 Oesch P, Eberhardt R. Bewegungsapparat. 3., überarbeitete und ergänzte Aufl. Bern: Hogrefe; 2017
    Google Scholar
  • 2 Vet HCWd. Measurement in Medicine: A Practical Guide. Cambridge: University Press; 2011. DOI: 10.1017/CBO9780511996214
    CrossrefGoogle Scholar
  • 3 Denegar CR, Ball DW. Assessing Reliability and Precision of Measurement: An Introduction to Intraclass Correlation and Standard Error of Measurement. 1993; 2: 35 . doi:10.1123/jsr.2.1.35
    PubMedGoogle Scholar
  • 4 Shrout PE, Fleiss JL. Intraclass correlations: uses in assessing rater reliability. Psychol Bull 1979; 86: 420-428
    CrossrefPubMedGoogle Scholar
  • 5 Streiner DL. Health measurement scales: a practical guide to their development and use. 5th ed.. Oxford: University Press; 2015
    CrossrefGoogle Scholar
  • 6 Naizer G. Basic concepts in generalizability theory: a more powerful approach to evaluating reliability. Paper, präsented Anual Meeting Southwest Edu Research Asociation. Houston, TX: 1992
    Google Scholar
  • 7 Shavelson RL, Webb NM. Generalizability Theory: A Primer. Thousand Oaks, CA: Sage; 1991
    Google Scholar
  • 8 Brennan RL. Generalizability theory. New York: Springer; 2001
    CrossrefGoogle Scholar
  • 9 de Bruin ED, Rozendal RH, Stussi E. Reliability of phase-velocity measurements of tibial bone. Phys Ther 1998; 78: 1166-1174 . doi:10.1093/ptj/78.11.1166
    CrossrefPubMedGoogle Scholar
  • 10 Heitman RJ, Kovaleski JE, Pugh SF. Application of generalizability theory in estimating the reliability of ankle-complex laxity measurement. J Athl Train 2009; 44: 48-52 . doi:10.4085/1062-6050-44.1.48
    CrossrefPubMedGoogle Scholar
  • 11 Fleiss JL. Design and Analysis of Clinical Experiments. Wiley; 2011
    Google Scholar
  • 12 Baschung Pfister P, de Bruin ED, Sterkele I. et al Manual muscle testing and hand-held dynamometry in people with inflammatory myopathy: An intra- and interrater reliability and validity study. PloS one 2018; 13: e0194531 . doi:10.1371/journal.pone.0194531
    CrossrefPubMedGoogle Scholar
  • 13 Bates D, Mächler M, Bolker B. et al Fitting Linear Mixed-Effects Models Using lme4. Journal of Statistical Software 2015; 67: 48 . doi:10.18637/jss.v067.i01
    CrossrefPubMedGoogle Scholar
  • 14 R Core Team. A language and environment for statistical computing. R Foundation for Statistical Computing. Vienna: 2019
    Google Scholar
  • 15 Roebroeck ME, Harlaar J, Lankhorst GJ. The application of generalizability theory to reliability assessment: an illustration using isometric force measurements. Phys Ther 1993; 73: 386-395 ; discussion 396–401. doi:10.1093/ptj/73.6.386
    CrossrefPubMedGoogle Scholar
  • 16 Gagnon D, Nadeau S, Gravel D. et al Reliability and validity of static knee strength measurements obtained with a chair-fixed dynamometer in subjects with hip or knee arthroplasty. Arch Phys Med Rehabil 2005; 86: 1998-2008 . doi:10.1016/j.apmr.2005.04.013
    CrossrefPubMedGoogle Scholar
  • 17 Anthony A, Gatti PWS, Nicholas MB. et al. How to Optimize Measurement Protocols: An Example of Assessing Measurement Reliability Using Generalizability Theory. DOI: 10.3138/ptc-2018-0110
    CrossrefGoogle Scholar
  • 18 Macintyre NJ, Bennett L, Bonnyman AM. et al Optimizing reliability of digital inclinometer and flexicurve ruler measures of spine curvatures in postmenopausal women with osteoporosis of the spine: an illustration of the use of generalizability theory. ISRN Rheumatol 2011; 571698 . doi:10.5402/2011/571698
    PubMedGoogle Scholar
  • 19 Bauer CM, Rast FM, Ernst MJ. et al Concurrent validity and reliability of a novel wireless inertial measurement system to assess trunk movement. J Electromyogr Kinesiol 2015; 25: 782-790 . doi:10.1016/j.jelekin.2015.06.001
    CrossrefPubMedGoogle Scholar
  • 20 Bauer CM, Heimgartner M, Rast FM. et al Reliability of lumbar movement dysfunction tests for chronic low back pain patients. Man Ther 2016; 24: 81-84 . doi:10.1016/j.math.2016.02.013
    CrossrefPubMedGoogle Scholar
  • 21 Lafave M, Butterwick D. A generalizability theory study of athletic taping using the Technical Skill Assessment Instrument. doi:10.4085/1062-6050-49.2.22. Im Internet (Stand: 29.02.2020): www.cochranelibrary.com/central/doi/10.1002/central/CN-01115682/full
    PubMedGoogle Scholar
  • 22 Levac D, Missiuna C, Wishart L. et al The motor learning strategy instrument: interrater reliability within usual and virtual reality physical therapy interventions. Pediatr Phys Ther 2013; 25: 53-60 . doi:10.1097/PEP.0b013e3182750c28
    CrossrefPubMedGoogle Scholar
  • 23 Watkins B, Darrah J, Pain K. Reliability of passive ankle dorsiflexion measurements in children: comparison of universal and biplane goniometers. Im Internet (Stand: 29.02.2020): www.cochranelibrary.com/central/doi/10.1002/central/CN-00182435/full
    PubMedGoogle Scholar
  • 24 Ernst MJ, Sommer BB, Meichtry A. et al Intra-rater reliability of determining positions of cervical spinous processes and measuring their relative distances: An update to define rigid bodies of the cervical spine in a movement laboratory setting. BMC research notes 2019; 12: 265 . doi:10.1186/s13104-019-4299-8
    CrossrefPubMedGoogle Scholar
  • 25 Lariviere C, Ludvig D, Kearney R. et al Identification of intrinsic and reflexive contributions to low-back stiffness: medium-term reliability and construct validity. J Biomech 2015; 48: 254-261 . doi:10.1016/j.jbiomech.2014.11.036
    CrossrefPubMedGoogle Scholar
  • 26 Pryseley A, Ledent EY, Drewes AM. et al Applying concepts of generalizability theory on data from experimental pain studies to investigate reliability. Basic & clinical pharmacology & toxicology 2009; 105: 105-112 . doi:10.1111/j.1742-7843.2009.00408.x
    CrossrefPubMedGoogle Scholar
  • 27 Rast FM, Graf ES, Meichtry A. et al Between-day reliability of three-dimensional motion analysis of the trunk: A comparison of marker based protocols. J Biomech 2016; 49: 807-811 . doi:10.1016/j.jbiomech.2016.02.030
    CrossrefPubMedGoogle Scholar
  • 28 Swaine BR, Lortie E, Gravel D. The reliability of the time to execute various forms of the finger-to-nose test in healthy subjects. Physiotherapy theory and practice 2005; 21: 271-279 . doi:10.1080/09593980500321119
    CrossrefPubMedGoogle Scholar
  • 29 Fortin C, Feldman DE, Cheriet F. et al Reliability of a quantitative clinical posture assessment tool among persons with idiopathic scoliosis. Physiotherapy 2012; 98: 64-75 . doi:10.1016/j.physio.2010.12.006
    CrossrefPubMedGoogle Scholar
  • 30 Sommer BB, Weisenhorn M, Ernst MJ. et al Concurrent validity and reliability of a mobile tracking technology to measure angular and linear movements of the neck. J Biomech 2019; 96: 109340 . doi:10.1016/j.jbiomech.2019.109340
    CrossrefPubMedGoogle Scholar
  • 31 Vangeneugden T, Laenen A, Geys H. et al Applying concepts of generalizability theory on clinical trial data to investigate sources of variation and their impact on reliability. Biometrics 2005; 61: 295-304 . doi:10.1111/j.0006-341X.2005.031040.x
    CrossrefPubMedGoogle Scholar
  • 32 Vispoel WP, Morris CA, Kilinc M. Applications of generalizability theory and their relations to classical test theory and structural equation modeling. Psychol Methods 2018; 23: 1-26 . doi:10.1037/met0000107
    CrossrefPubMedGoogle Scholar