Intervention Effects in the Case of Heterogeneity between Three Subgroups

Summary

Background: Usually, statistical tests for interactions are applied to investigate potential effect modifiers. If an effect modifier, consisting of three categories, is found to be statistically significant, the application of pairwise interaction tests is indicated. In this case, the problem of non-transitive relations may occur if the significance level is fixed at 0.05 for all tests.

Objective: To develop an algorithm for which non-transitive relations do not occur.

Methods: A hierarchical testing procedure is applied, based on the heterogeneity statistic Q. In a first step the interaction will be tested for the three trial subgroups altogether, applying the significance level α = 0.05 (global test). If a significant interaction is proven in the first step, pairwise tests for interaction will be applied in a second step. Theoretical data scenarios will be considered and p-values will be calculated for the pairwise tests. Based on these results the significance level for pairwise testing will be determined.

Results: Fixing the significance level at 0.05 for all tests, the problem of non-transitive relations is mostly relevant, if the difference in the effects between the three trial subgroups is approximately 3.5 standard errors and the effect of the ‘middle’ trial subgroup is not close to one of the other two effects. This problem vanishes when the significance level is set to α = 0.22. We propose to select α = 0.20 to get a more ’even’ and simple value.

Conclusions: By increasing the significance level for the pairwise tests to 0.20, non-transitive relations are virtually avoidable. The proposed hierarchical testing procedure represents a clear practical guidance to perform subgroup analyses in the framework of systematic reviews.

Keywords

Subgroups - heterogeneity - systematic review - interaction - meta-analysis

• References

• 1 Deeks JJ, Altman DG, Bradburn MJ. Statistical methods for examining heterogeneity and combining results from several studies in meta-analysis. In: Egger M, Davey Smith G, Altman DG editors. Systematic Reviews in Health Care: Meta-analysis in Context (2nd edition). London (UK): BMJ Publication Group; 2001
• 2 Yusuf S, Zhao F, Mehta SR, Chrolavicius S, Tognoni G, Fox KK. Effects of clopidogrel in addition to aspirin in patients with acute coronary syndromes without ST-segment elevation. N Engl J Med 2001; 345 (Suppl. 07) 494-502.
• 3 Institute for Quality and Efficiency in Health Care.. Clopidogrel plus acetylsalicylic acid in acute coronary syndrome: final report; commission A04-01B (Internet). Cologne: IQWiG; Jan 28, 2009 (cited Nov 5, 2009). Available from: http://www.iqwig.de.download/A04-01B_AB_Clopidogrel_plus_ASS_bei_akutem_Koronarsyndrom.pdf
• 4 Alosh M, Huque MF. A flexible strategy for testing subgroups and overall population. Stat Med 2009; 28 (Suppl. 01) 3-23.
• 5 Cochran WG. The Combination of Estimates from Different Experiments. Biometrics 1954; 10: 101-129.
• 6 Koch A, Ziegler S. Metaanalyse als Werkzeug zum Erkenntnisgewinn. Med Klin 2000; 95: 109-116.