The meta-analysis by Shang et al (2005)

A systematic review of randomised controlled trials (RCTs), including a meta-analysis of the summarised data, is considered the highest grade of evidence with regard to a given clinical intervention. A meta-analysis is a quantitative summary of the outcomes of two or more RCTs that have been carried out on the same topic; by statistical pooling of data in what is known as a forest plot, a meta-analysis increases confidence in the findings by including a greater total number of patients than each of the original studies alone.

In our own programme of systematic reviews, we place emphasis on RCTs that have gone through the ‘peer review’ process, which strives to maintain standards of quality and thus ensures the credibility of scientific publications. During peer review of a manuscript submitted to an academic journal, independent researchers from the same field (‘peers’) judge its intrinsic quality, offering improvements as may be appropriate, and recommend to the journal editor whether it should be published or not.

Shang et al. 2005

In August 2005, the renowned peer-reviewed journal The Lancet published a systematic review and meta-analysis, reported by a research group from the University of Berne, Switzerland (Shang et al 2005a). The researchers examined a total of 110 placebo-controlled RCTs of homeopathy, with particular focus on 8 trials they categorised as ‘larger trials of higher methodological quality’. Without reference to the other (smaller) high-quality trials, they concluded that there was ‘weak evidence for a specific effect of homoeopathic remedies’ and that their findings were ‘compatible with the notion that the clinical effects of homoeopathy are placebo effects’.

In December 2005, the website of the University of Berne published a forest plot of the 110 placebo-controlled trials of homeopathy (Shang et al. 2005a), thus rectifying the omission of these data from the paper they had published in The Lancet four months earlier (Shang et al. 2005b).

Our observations

In the connected PDF file, prepared in October 2014, we examine the data from all the peer-reviewed RCTs included in Shang’s report, and observe some novel facts about the pooled results and also about the findings from the individual RCTs contained in their meta-analysis. For example, Shang’s data for peer-reviewed RCTs were actually collectively positive. Also, each of 3 of their ‘higher-quality’ trials statistically favoured homeopathy over placebo: this fact refutes the notion, promulgated by some, that no peer-reviewed and high-quality RCT of homeopathy has ever obtained a positive result.

In another connected PDF file, also prepared in October 2014, we examine the data from the peer-reviewed RCTs of individualised homeopathic treatment that were included in Shang’s report. There were 8 RCTs in Shang’s report that also comprised a sub-group analysis in the British Homeopathic Association’s meta-analysis of this type of trial, published in 2014: our finding collectively for those 8 RCTs (which was positive) is almost identical to Shang’s unreported finding for that same set of trials in individualised homeopathy.

Thus, the Shang meta-analysis contains previously unnoticed data for individualised homeopathy, whose positive collective finding does not differ from the British Homeopathic Association’s finding, published in 2014, for the same set of trials.

References

Shang A, Huwiler-Muntener K, Nartey L, et al (2005a). Are the clinical effects of homoeopathy placebo effects? Comparative study of placebo-controlled trials of homoeopathy and allopathy. Lancet366: 726–732.

Shang A, Huwiler-Muntener K, Nartey L, et al (2005b). Study characteristics of homoeopathy studies. http://www.ispm.ch/index.php?id=lancet
http://www.ispm.ch/fileadmin/doc_download/1433.Study_characteristics_of_homoeopathy_studies_corrected.pdf