Lancet review was ‘flawed’

Two recent studies show that a widely publicised negative review of homeopathy in The Lancet in 2005 was seriously flawed in its conclusions.

Papers published last month in the Journal of Clinical Epidemiology and Homeopathy journal have re-constructed the analysis carried out by the authors of the Lancet review, which concluded that there was only weak evidence for homeopathy.

The Lancet analysis

The authors started with 110 trials each in homeopathic and conventional medicine. 21 homeopathic and 9 conventional trials were judged as being of ‘higher quality’. The reviewers then narrowed these trials down to eight of homeopathy and six of conventional medicine. These were deemed to be ‘larger trials of higher quality’.

The resulting analysis of these 14 trials came out with a negative conclusion for homeopathy.

What the reconstruction of the analysis shows

It is now clear that if all 21 ‘higher quality’ trials had been included in the analysis, the result would have been positive for homeopathy.

If fewer than 21 trials are analysed, a positive or negative conclusion for homeopathy is crucially dependent on the exact number of trials selected.

A firm positive conclusion is found, for example, merely by omitting four trials that showed Arnica is ineffective for muscle soreness after long-distance running.

Whether the conclusions of the analysis are positive or negative is therefore highly sensitive to the choice of which “higher quality” trials are included. The clear implication is that homeopathy for certain conditions is not placebo.