The Guardian Science Blog on 5th February has an entry by Martin Robbins that maligns and misrepresents the BHA’s submission to the Commons Science & Technology Committee. We set the record straight by clarifying the facts:
Robbins claims the BHA is wrong to state that four comprehensive reviews of homeopathy “have reached the qualified conclusion that homeopathy differs from placebo”. Quoting from the conclusions of each of the four original papers:
Kleijnen 1991: “At the moment the evidence of [mainly placebo-controlled] clinical trials is positive but not sufficient to draw definitive conclusions”.
Boissel 1996: “From the available evidence it is likely that among the tested homoeopathic approaches some had an added effect over nothing or placebo.”
Linde 1997: “The results of our meta-analysis are not compatible with the hypothesis that the clinical effects of homoeopathy are completely due to placebo”. In 1999, the same group of authors investigated the influence of indicators of methodological quality on study outcome and found that studies with higher-quality scores tended to yield less positive results than those with lower-quality scores. After discarding most of the lower-quality trials, homeopathic treatment remained more effective than placebo – though less strikingly so than in their 1997 analysis. In both of Linde’s systematic reviews, insufficient evidence was found to draw conclusions about the efficacy of homeopathy for any specific medical condition.
Cucherat 2000: “There is some evidence that homeopathic treatments are more effective than placebo.” This paper differs markedly from the report of the same group of authors (Boissel 1996), and indeed contains an update of that review with at least two additional papers’ data cited and analysed.
The main thrust of the BHA’s submission to parliament was actually a summary of condition-specific systematic reviews of randomised controlled trials of homeopathy. Robbins states we incorrectly reflect two of those 24 reviews. Again, quoting from the original papers:
Ernst 1999: “There is evidence that homeopathic treatment can reduce the duration of ileus after abdominal or gynecologic surgery. However, several caveats preclude a definitive judgment. These results should form the basis of a randomized controlled trial to resolve the issue.”
Jonas 2000: “Overall, it appears that homeopathic remedies work better than placebo in studies of rheumatic syndromes, but there are too few studies to make definitive conclusions about the efficacy of any one type of homeopathic treatment for any one condition.”
The clinical research evidence in homeopathy is not currently definitive – which is why we need more research – but there are positive findings from randomised controlled trials in a number of medical conditions, some of that evidence supported in systematic reviews.
Vickers 2006: Despite being advised to the contrary (see below), Robbins accuses the BHA of confusion about the Cochrane review of influenza. Depending on the judgmental criteria used, the review may be regarded as positive or inconclusive regarding the treatment of influenza with the homeopathic preparation Oscillococcinum. Vickers concluded that Oscillococcinum reduced the length of influenza illness by 0.28 days on average; the effect was statistically significant, and so the review is included in the BHA website list as a positive finding. For the Committee review, we acknowledged this clearly in a footnote: “The review reported that Oscillococcinum reduced the length of influenza illness by 0.28 days (95% confidence interval, 0.50 to 0.06). The authors concluded “though promising, the data are not strong enough to make a general recommendation to use Oscillococcinum for first-line treatment of influenza”. Robbins has chosen to focus on the Cochrane review’s discussion of influenza prevention: “Current evidence does not support a preventative effect of Oscillococcinum-like homeopathic medicines in influenza and influenza-like syndromes”.
A transcript of email dialogue with Mr Robbins on 3rd February illustrates the biased nature of his reporting:
Robbins: Can you clarify what you feel the state of the evidence for homeopathy is? Your spokesman in a recent Telegraph article on 10:23 talked of a growing body of evidence, while your recent press release and statement of evidence to the select committee suggest that the body of evidence is generally poor in quality and quantity.
Mathie: There is research evidence in a number of medical conditions that homeopathic intervention is effective. Our written and oral evidence to the select committee made that clear. To date, a total of only 142 clinical trials in homeopathy have been published in peer-reviewed journals (compared with an estimated half million in conventional medicine), and so of course more research is required.
Robbins: Why do you think it’s been so difficult to produce consistently good results in controlled trials?
Mathie: Actually, 44% of the 142 controlled trials in homeopathy have been positive (see same sources as above). That research is spread rather thinly over 80 different medical conditions, and there are only 28 conditions in which there has been at least two trials. Differing study designs and the small size of many trials means that there are few conditions where there has been an opportunity to achieve consistent results. Nevertheless, repeatably positive results have been obtained for example in hayfever, fibromyalgia, sinusitis and vertigo.
Robbins: Some of the papers included in your evidence seem odd. Just taking the first few for example, you cite a Cochrane review (5,Vickers) as supporting homeopathy, yet the authors very clearly state in the paper (and to me in correspondence) that “Current evidence does not support a preventative effect of Oscillococcinum-like homeopathic medicines in influenza and influenza-like syndromes.” I found similar apparent errors through-out this document – how does the BHA explain these anomalies?
Mathie: There is nothing inconsistent or anomalous about the entries in the list. The evidence in systematic reviews is a synthesis of all the research available at the time. We have highlighted the individual trials as well as the systematic reviews. In the case of influenza, the Cochrane review by Vickers also reported that, when used as treatment, the homeopathic medicine Oscillococcinum reduced the length of influenza illness by 0.28 days on average; the effect was statistically significant, and so the review is included in our list as a positive finding.
Boissel 1996: Boissel JP, Cucherat M, Haugh M, Gauthier E (1996). Critical literature review on the effectiveness of homoeopathy: overview of data from homoeopathic medicine trials. In: Homoeopathic Medicine Research Group, Report of the Commission of the European Communities, Directorate-General XII – Science, Research and Development, Directorate E – RTD Actions: Life Sciences and Technologies – Medical Research, Brussels, Belgium.
Cucherat 2000: Cucherat M, Haugh MC, Gooch M, Boissel JP (2000). Evidence of clinical efficacy of homeopathy – A meta-analysis of clinical trials. European Journal of Clinical Pharmacology, 56: 27–33.
Ernst 1999: Barnes J, Resch K-L, Ernst E (1997). Homeopathy for postoperative ileus? A meta-analysis. Journal of Clinical Gastroenterology, 25: 628–633.
Jonas 2000: Jonas WB, Linde K, Ramirez G (2000). Homeopathy and rheumatic disease. Rheumatic Disease Clinics of North America, 26: 117–123.
Kleijnen 1991: Kleijnen J, Knipschild P, ter Riet G (1991). Clinical trials of homoeopathy British Medical Journal, 302: 316–323.
Linde 1997: Linde K, Clausius N, Ramirez G, et al (1997). Are the clinical effects of homoeopathy placebo effects? A meta-analysis of placebo-controlled trials. Lancet, 350: 834–843.
Linde 1999: Linde K, Scholz M, Ramirez G, et al (1999). Impact of study quality on outcome in placebo controlled trials of homeopathy. Journal of Clinical Epidemiology, 52:631–636.
Vickers 2006: Vickers A, Smith C (2006). Homoeopathic Oscillococcinum for preventing and treating influenza and influenza-like syndromes (Cochrane review). In: The Cochrane Library. Chichester, UK: John Wiley & Sons, Ltd. CD001957.