Challenges in planning and conducting diagnostic studies with molecular biomarkers

Abstract

Biomarkers are of increasing importance for personalized medicine in many areas of application, such as diagnosis, prognosis, or the selection of targeted therapies. In many molecular biomarker studies, intensity values are obtained from large scale –omics experiments. These intensity values, such as protein concentrations, are often compared between at least two groups of subjects to determine the diagnostic ability of the molecular biomarker. Various prospective or retrospective study designs are available for molecular biomarker studies, and the biomarker used may be univariate or even consist in a multimarker rule. In this work, several challenges are discussed for the planning and conduct of biomarker studies. The phases of diagnostic biomarker studies are closely related to levels of evidence in diagnosis, and they are therefore discussed upfront. Different study designs for molecular biomarker studies are discussed, and they primarily differ in the way subjects are selected. Using two systematic reviews from the literature, common sources of bias of molecular diagnostic studies are illustrated. The extreme selection of patients and controls and verification bias are specifically discussed. The pre-analytical and technical variability of biomarker measurements is usually expressed in terms of the coefficient of variation, and is of great importance for subsequent validation studies for molecular biomarkers. It is finally shown that the required sample size for biomarker validation quadratically increases with the coefficient of variation, and the effect is illustrated using real data from different laboratory technologies.

Keywords

biomarker - coefficient of variation - diagnostic study - molecular diagnostics - sampling bias, validation - verification bias

• References

• 1 Ziegler A, Koch A, Krockenberger K et al. Personalized medicine using DNA biomarkers: a review. Hum Genet 2012; 131: 1627-1638
• 2 Gallo V, Egger M, McCormack V et al. STrengthening the Reporting of OBservational studies in Epidemiology – Molecular Epidemiology (STROBE-ME): an extension of the STROBE Statement. PLoS Med 2011; 8: e1001117
• 3 Biomarkers Definitions Working Group. Biomarkers and surrogate endpoints: preferred definitions and conceptual framework. Clin Pharmacol Ther 2011; 69: 89-95
• 4 Kroll W. Biomarkers – predictors, surrogate parameters – a concept definition. In: Schmitz G, Endres S, Götte D, eds, Biomarker. Stuttgart: Schattauer; 2008: 1-14
• 5 Haddow JE, Palomaki GE. ACCE: A model process for evaluating data on emerging genetic tests. In: Khoury M, Little J, Burke W, eds, Human Genome Epidemiology: A Scientific Foundation for Using Genetic Information to Improve Health and Prevent Disease. Oxford: Oxford University Press; 2003: 217-233
• 6 DIN Deutsches Institut für Normung e.V.. DIN ISO 5725-1:1997-11 Genauigkeit (Richtigkeit und Präzision) von Messverfahren und Messergebnissen – Teil 1: Allgemeine Grundlagen und Begriffe. In: DIN Deutsches Institut für Normung e.V., ed, DIN-Taschenbuch 355: Statistik – Genauigkeit von Messungen – Ringversuche. Berlin: Beuth; 2004: 1-44
• 7 Evans JP, Skrzynia C, Burke W. The complexities of predictive genetic testing. BMJ 2001; 322: 1052-1056
• 8 Jensen K, Abel U. Methodik diagnostischer Validierungsstudien. Fehler in der Studienplanung und Auswertung. Med Klin (Munich) 1999; 94: 522-529
• 9 Pepe MS. The statistical evaluation of medical tests for classification and prediction. New York: Oxford University Press; 2003
• 10 Zhou X-H, Obuchowski NA, McClish DK. Statistical methods in diagnostic medicine. New York: John Wiley & Sons; 2001
• 11 Pepe MS, Etzioni R, Feng Z et al. Phases of biomarker development for early detection of cancer. J Natl Cancer Inst 2001; 93: 1054-1061
• 12 Abel U, Jensen K. Klinische Studien außerhalb des Arzneimittelgesetzes: Diagnosestudien. Bundesgesundheitsbl 2009; 52: 425-432
• 13 Buyse M, Michiels S, Sargent DJ et al. Integrating biomarkers in clinical trials. Expert Rev Mol Diagn 2011; 11: 171-182
• 14 Buyse M, Sargent DJ, Grothey A et al. Biomarkers and surrogate end points – the challenge of statistical validation. Nat Rev Clin Oncol 2010; 7: 309-317
• 15 Sargent DJ, Conley BA, Allegra C et al. Clinical trial designs for predictive marker validation in cancer treatment trials. J Clin Oncol 2005; 23: 2020-2027
• 16 Mandrekar SJ, Grothey A, Goetz MP et al. Clinical trial designs for prospective validation of biomarkers. Am J Pharmacogenomics 2005; 5: 317-325
• 17 Mandrekar SJ, Sargent DJ. Clinical trial designs for predictive biomarker validation: theoretical considerations and practical challenges. J Clin Oncol 2009; 27: 4027-4034
• 18 Mandrekar SJ, Sargent DJ. Clinical trial designs for predictive biomarker validation: one size does not fit all. J Biopharm Stat 2009; 19: 530-542
• 19 Mandrekar SJ, Sargent DJ. Predictive biomarker validation in practice: lessons from real trials. Clin Trials 2010; 7: 567-573
• 20 Schäfer H. Anforderungen an einen patientenorientierte klinisch-therapeutische Forschung. Dtsch Med Wochenschr 1997; 122: 1531-1536
• 21 Weinstein S, Obuchowski NA, Lieber ML. Clinical evaluation of diagnostic tests. AJR Am J Roentgenol 2005; 184: 14-19
• 22 Obuchowski NA. How many observers are needed in clinical studies of medical imaging?. AJR Am J Roentgenol 2004; 182: 867-869
• 23 Egerer K, Feist E, Burmester GR. The serological diagnosis of rheumatoid arthritis: antibodies to citrullinated antigens. Dtsch Arztebl Int 2009; 106: 159-163
• 24 Ziegler A, König IR. Leitlinien für Forschungsberichte: Deutschsprachige Übersetzungen von CONSORT 2010, PRISMA und STARD. Dtsch Med Wochenschr 2011; 136: 357-358
• 25 Fontela PS, Pant Pai N, Schiller I et al. Quality and reporting of diagnostic accuracy studies in TB, HIV and malaria: evaluation using QUADAS and STARD standards. PLoS ONE 2009; 4: e7753
• 26 Lijmer JG, Mol BW, Heisterkamp S et al. Empirical evidence of design-related bias in studies of diagnostic tests. JAMA 1999; 282: 1061-1066
• 27 Rutjes AW, Reitsma JB, Di Nisio M et al. Evidence of bias and variation in diagnostic accuracy studies. Can Med Assoc J 2006; 174: 469-476
• 28 Sica GT. Bias in research studies. Radiology 2006; 238: 780-789
• 29 Centre for Review and Dissemination. Systematic reviews: CRD's guidance for undertaking reviews in health care. 2009 www.york.ac.uk/inst/crd/SysRev/!SSL!/WebHelp/TITLEPAGE.htm Accessed 26.04.2013.
• 30 Blackmore CC. The challenge of clinical radiology research. AJR Am J Roentgenol 2001; 176: 327-331
• 31 Brealey S, Scally AJ. Bias in plain film reading performance studies. Br J Radiol 2001; 74: 307-316
• 32 Whiting P, Rutjes AW, Reitsma JB et al. Sources of variation and bias in studies of diagnostic accuracy: a systematic review. Ann Intern Med 2004; 140: 189-202
• 33 Guyatt G, Rennie D, Meade MO, Cook DJ eds. Users' Guide to the Medical Literature: A Manual for Evidence-Based Clinical Practice. 2.. ed. Minion: McGraw-Hill; 2008
• 34 Thomson DM, Krupey J, Freedman SO et al. The radioimmunoassay of circulating carcinoembryonic antigen of the human digestive system. Proc Natl Acad Sci U S A 1969; 64: 161-167
• 35 Zielinski C. Aussagekraft des carcinoembryonalen Antigens. Dtsch Med Wochenschr 1995; 120: 893
• 36 Lachs MS, Nachamkin I, Edelstein PH et al. Spectrum bias in the evaluation of diagnostic tests: lessons from the rapid dipstick test for urinary tract infection. Ann Intern Med 1992; 117: 135-140
• 37 Cicero S, Rembouskos G, Vandecruys H et al. Likelihood ratio for trisomy 21 in fetuses with absent nasal bone at the 11-14-week scan. Ultrasound Obstet Gynecol 2004; 23: 218-223
• 38 Knottnerus JA, Muris JW. Assessment of the accuracy of diagnostic tests: the cross-sectional study. J Clin Epidemiol 2003; 56: 1118-1128
• 39 PIOPED Investigators. Value of the ventilation/perfusion scan in acute pulmonary embolism. Results of the prospective investigation of pulmonary embolism diagnosis (PIOPED). The PIOPED Investigators. JAMA 1990; 263: 2753-2759
• 40 Punglia RS, D'Amico AV, Catalona WJ et al. Effect of verification bias on screening for prostate cancer by measurement of prostate-specific antigen. N Engl J Med 2003; 349: 335-342
• 41 de Groot JA, Bossuyt PM, Reitsma JB et al. Verification problems in diagnostic accuracy studies: consequences and solutions. BMJ 2011; 343: d4770
• 42 Martus P, Schueler S, Dewey M. Fractional flow reserve estimation by coronary computed tomography angiography. J Am Coll Cardiol 2012; 59: 1410-1411 author reply 1411
• 43 Hanrahan CF, Westreich D, Van Rie A. Verification bias in a diagnostic accuracy study of symptom screening for tuberculosis in HIV-infected pregnant women. Clin Infect Dis 2012; 54: 1377-1378 author reply 1378-1379
• 44 Richardson ML, Petscavage JM. Verification bias: an under-recognized source of error in assessing the efficacy of MRI of the meniscii. Acad Radiol 2011; 18: 1376-1381
• 45 Begg CB, Greenes RA. Assessment of diagnostic tests when disease verification is subject to selection bias. Biometrics 1983; 39: 207-215
• 46 Diamond GA. "Work-up bias". J Clin Epidemiol 1993; 46: 207-209
• 47 Rutjes AW, Reitsma JB, Coomarasamy A et al. Evaluation of diagnostic tests when there is no gold standard. A review of methods. Health Technol Assess 2007; 11: iii, ix-51
• 48 Dehdashti H, Dehdashtian M, Rahim F et al. Sonographic measurement of abdominal esophageal length as a diagnostic tool in gastroesophageal reflux disease in infants. Saudi J Gastroenterol 2011; 17: 53-57
• 49 Sarkhy AA. Methodological issues in diagnostic studies. Saudi J Gastroenterol 2011; 17: 161-162
• 50 Harvey RF. Indices of thyroid function in thyrotoxicosis. Lancet 1971; 2: 230-233
• 51 Andersen B. Methodological errors in medical research. An incomplete catalogue. Oxford: Blackwell; 1990
• 52 Begg CB. Biases in the assessment of diagnostic tests. Stat Med 1987; 6: 411-423
• 53 Simel DL, Feussner JR, DeLong ER et al. Intermediate, indeterminate, and uninterpretable diagnostic test results. Med Decis Making 1987; 7: 107-114
• 54 Ronco G, Montanari G, Aimone V et al. Estimating the sensitivity of cervical cytology: errors of interpretation and test limitations. Cytopathology 1996; 7: 151-158
• 55 Ramos JM, Robledano C, Masia M et al. Contribution of Interferon gamma release assays testing to the diagnosis of latent tuberculosis infection in HIV-infected patients: A comparison of QuantiFERON-TB gold in tube, T-SPOT.TB and tuberculin skin test. BMC Infect Dis 2012; 12: 169
• 56 Begg CB, Greenes RA, Iglewicz B. The influence of uninterpretability on the assessment of diagnostic tests. J Chronic Dis 1986; 39: 575-584
• 57 Philbrick JT, Horwitz RI, Feinstein AR et al. The limited spectrum of patients studied in exercise test research. Analyzing the tip of the iceberg. JAMA 1982; 248: 2467-2470
• 58 Detrano R, Gianrossi R, Mulvihill D et al. Exercise-induced ST segment depression in the diagnosis of multivessel coronary disease: a meta analysis. J Am Coll Cardiol 1989; 14: 1501-1508
• 59 Bossuyt PM, Reitsma JB, Bruns DE et al. The STARD statement for reporting studies of diagnostic accuracy: explanation and elaboration. Ann Intern Med 2003; 138: W1-12
• 60 Reid MC, Lachs MS, Feinstein AR. Use of methodological standards in diagnostic test research. Getting better but still not good. JAMA 1995; 274: 645-651
• 61 Sardanelli F, Di Leo G. Biostatistics for Radiologists. Mailand: Springer; 2009
• 62 Rutjes AW, Reitsma JB, Di Nisio M et al. Evidence of bias and variation in diagnostic accuracy studies. CMAJ 2006; 174: 469-476
• 63 Wilczynski NL. Quality of reporting of diagnostic accuracy studies: no change since STARD statement publication – before-and-after study. Radiology 2008; 248: 817-823
• 64 Hollingworth W, Jarvik JG. Technology assessment in radiology: putting the evidence in evidence-based radiology. Radiology 2007; 244: 31-38
• 65 Gemeinsamer Bundesausschuss. Verfahrensordnung des Gemeinsamen Bundesausschusses. http://www.g-ba.de/downloads/62-492-654/VerfO_2012-10-18.pdf. Fassung vom: 18.12.2008. BAnz. Nr. 84a (Beilage) vom 10.06.2009. Letzte Änderung: 18.10.2012. BAnz AT 05.12.2012 B3. In Kraft getreten am 06.12.2012. Letzter Zugriff 26.04.2013.