In March 2015, the Australian National Health and Medical Research Council (NHMRC) published an Information Paper on homeopathy. This document, designed for the general public, provides a summary of the findings of a review of systematic reviews, performed by NHMRC, to assess the evidence base for effectiveness of homeopathy in humans.
‘The Australian report’ concludes that “…there are no health conditions for which there is reliable evidence that homeopathy is effective…no good-quality, well-designed studies with enough participants for a meaningful result reported either that homeopathy caused greater health improvements than placebo, or caused health improvements equal to those of another treatment.”
Such overly-definitive negative conclusions are immediately surprising, being inconsistent with the majority of comprehensive systematic reviews on homeopathy.
In-depth analysis has revealed the report's multiple methodological flaws, which explain this inconsistency. Most crucially, NHMRC's findings hinge primarily on their definition of reliable evidence: for a trial to be deemed ‘reliable’ it had to have at least 150 participants and a quality score of 5/5 on the Jadad scale (or equivalent on other scales). Trials that failed to meet either of these criteria were dismissed as being of ‘insufficient quality and/or size to warrant further consideration of their findings’.
Setting such a high-quality threshold is highly unusual, but the n = 150 minimum sample size criterion is arbitrary, without scientific justification, and unprecedented in evidence reviews.
Out of 176 trials NHMRC included in the homeopathy review, only 5 trials met their definition of ‘reliable’, none of which, according to their analysis, demonstrated effectiveness of homeopathy. This explains why NHMRC concluded there is ‘no reliable evidence’ that homeopathy is effective.
A distillation of other detailed findings, presented at conference, reveals further significant flaws in this highly influential report, providing critical awareness of its misrepresentation of the homeopathy evidence base.
Keywords: Homeopathy, NHMRC, overview report, review of systematic reviews, reporting bias
