Download PDF
Methods Inf Med 2006; 45(04): 435-440
DOI: 10.1055/s-0038-1634101
Original Article
Schattauer GmbH

A Note on Testing for Intervention Effects on Binary Responses

T. Friede
1   Biostatistics and Statistical Reporting, Novartis Pharma AG, Basel, Switzerland
,
R. Henderson
2   Mathematics and Statistics, University of Newcastle upon Tyne, Newcastle, UK
,
C.-F. Kao
3   Mathematics and Statistics, Lancaster University, Lancaster, UK
› Author Affiliations
Further Information

Publication History

Publication Date:
06 February 2018 (online)

    Permissions and Reprints

Summary

Objectives: In some circumstances controlled trials are not feasible and treatments can only be evaluated using clinical databases. Here we consider the situation where treatment is introduced at a particular calendar time and can only be evaluated by comparison with historical controls. In these circumstances Heuer and Abel recommended using change-point methods to search for change in characteristics over the whole study period rather than simply comparing treated and untreated patients. Their recommendation is to only conclude that the intervention had an effect if a change-point could be demonstrated close in time to the introduction of the new treatment. This reduces the risk of false positives caused by confounding changes in population characteristics or changes in patient management. For binary data we develop a method that follows their philosophy and apply it to an observational study in the treatment of pin sites after orthopaedic surgery.

Methods: Tests for change in binomial probabilities based on Brownian bridge and Hansen’s approximation for maximally selected X 2 statistics are compared to an exact test by Worsley. The approximate method is generalized to logistic regression models allowing for covariates.

Results: The agreement of the exact and approximate method is good for sample sizes of 100 or more. The actual test size of the Hansen approximate test allowing for covariates is close to the nominal level, whereas the Brownian bridge approximation is slightly conservative. The change in pin site treatment significantly reduces the risk of infection for both adults and children.

Conclusions: We consider the Hansen approximation to provide a very good and very simple method for obtaining the p-value when testing for a change in binary data event probabilities, with or without covariates.

Keywords

Change-point - observational study - logistic regression - pin site infection data
 

  • References

  • 1 Friede T, Henderson R. Intervention effects in observational studies, with an application in total hip replacements. Statistics in Medicine 2003; 22: 3725-37.
    CrossrefPubMedGoogle Scholar
  • 2 Heuer C, Abel U. The analysis of intervention effects using observational data bases. In Abel U, Koch A. (eds.) Nonrandomized comparative clinical studies. Duesseldorf: Symposium Publishing 1998; pp 101-7.
    PubMedGoogle Scholar
  • 3 Page ES. Continuous inspection schemes. Biometrika 1954; 41: 100-15.
    CrossrefPubMedGoogle Scholar
  • 4 Page ES. A test for a change in a parameter occurring at an unknown point. Biometrika 1955; 42: 523-7.
    CrossrefPubMedGoogle Scholar
  • 5 Sen A, Srivastava MS. On tests for detecting change in mean. Annals of Statistics 1975; 3: 98-108.
    CrossrefPubMedGoogle Scholar
  • 6 Worsley KJ. The power of likelihood ratio and cumulative sum tests for a change in a binomial probability. Biometrika 1983; 70: 455-64.
    CrossrefPubMedGoogle Scholar
  • 7 Worsley KJ. Confidence regions and tests for a change-point in a sequence of exponential family variables. Biometrika 1986; 73: 91-104.
    CrossrefPubMedGoogle Scholar
  • 8 Hinkley DV. Inference about the change-point in a sequence of random variable. Biometrika 1970; 57: 1-17.
    CrossrefPubMedGoogle Scholar
  • 9 Hinkley DV, Hinkley EA. Inference about the change-point in a sequence of binomial random variables. Biometrika 1970; 57: 477-88.
    CrossrefPubMedGoogle Scholar
  • 10 Yao YC. Approximating the distribution of the maximum likelihood estimate of the change-point in a sequence of independent random variables. Ann Statist 1987; 15: 1321-8.
    CrossrefPubMedGoogle Scholar
  • 11 Siegmund D. Boundary crossing probabilities and statistical applications. Annals of Statistics 1986; 14: 361-404.
    CrossrefPubMedGoogle Scholar
  • 12 Chernoff H, Zacks S. Estimating the current mean of a normal distribution which is subjected to a change in time. Annals of Mathematical Statistics 1964; 35: 999-1018.
    CrossrefPubMedGoogle Scholar
  • 13 Smith AFM. A Bayesian approach to inference about a change-point in a sequence of random variables. Biometrika 1975; 62: 407-16.
    CrossrefPubMedGoogle Scholar
  • 14 Bhattacharya PK, Frierson D. A nonparametric chart for detecting small disorders. Annals of Statistics 1981; 9: 544-54.
    CrossrefPubMedGoogle Scholar
  • 15 Carlstein E. Nonparametric change-point estimation. Annals of Statistics 1988; 16: 188-97.
    CrossrefPubMedGoogle Scholar
  • 16 Zacks S. Classical and Bayesian approaches to the change-point problem: fixed sample and sequential procedures. Statistique et Analyse des Données 1982; 1: 48-81.
    PubMedGoogle Scholar
  • 17 Henderson R, Matthews JNS. An investigation of changepoints in the annual number of cases of hemolytic-uremic syndrome. Applied Statistics 1993; 42: 461-71.
    CrossrefPubMedGoogle Scholar
  • 18 Henderson R. Change-point problem with correlated observations, with an application in material accountancy. Technometrics 1986; 28: 381-9.
    CrossrefPubMedGoogle Scholar
  • 19 Kim HJ. Change-point detection for correlated observations. Statistica Sinica 1996; 6: 275-87.
    PubMedGoogle Scholar
  • 20 Matthews DE, Farewell VT. On a singularity in the likelihood for a change-point hazard rate model. Biometrika 1985; 72: 703-4.
    CrossrefPubMedGoogle Scholar
  • 21 Henderson R. A problem with the likelihood ratio test for a change-point hazard rate model. Biometrika 1990; 77: 835-43.
    CrossrefPubMedGoogle Scholar
  • 22 Gijbels I, Gurler U. Estimation of a change point in a hazard function based on censored data. Lifetime Data Analysis 2003; 9: 395-411.
    CrossrefPubMedGoogle Scholar
  • 23 Miller R, Siegmund D. Maximally selected chi square statistics. Biometrics 1982; 38: 1011-6.
    CrossrefPubMedGoogle Scholar
  • 24 Halpern J.. Maximally selected chi square statistics for small samples. Biometrics 1982; 38: 1017-23.
    CrossrefPubMedGoogle Scholar
  • 25 Davies RB. Hypothesis testing when a nuisance parameter is present only under the alternative. Biometrika 1987; 74: 33-43.
    PubMedGoogle Scholar
  • 26 Andrews DA, Ploberger W. Optimal tests when a nuisance parameter is present only under the alternative. Econometrica 1994; 62: 1383-1414.
    CrossrefPubMedGoogle Scholar
  • 27 Betensky R, Rabinowitz D. Maximally selected χ2 statistics for k × 2 tables. Biometrics 1999; 55: 317-20.
    CrossrefPubMedGoogle Scholar
  • 28 Halpern AL. Minimally selected p and other tests for a single abrupt changepoint in a binary sequence. Biometrics 1999; 55: 1044-5.
    CrossrefPubMedGoogle Scholar
  • 29 Hansen BE. Approximate asymptotic p values for structural-change tests. Journal of Business & Economic Statistics 1997; 15: 60-7.
    PubMedGoogle Scholar
  • 30 Lausen B, Schumacher M. Maximally selected rank statistics. Biometrics 1992; 48: 73-85.
    CrossrefPubMedGoogle Scholar
  • 31 Lausen B, Hothorn T, Bretz F, Schumacher M.. Assessment of optimal selected prognostic factors. Biometrical Journal 2004; 46: 364-74.
    CrossrefPubMedGoogle Scholar
  • 32 Halpern AL. Multiple-changepoint testing for an alternating segments model of a binary sequence. Biometrics 2000; 56: 903-8.
    CrossrefPubMedGoogle Scholar