Tuesday, July 1, 2014

Yay it's time for homeopathy research with Dana Ullman!

So the good ol' Dana Ullman was arguing about homeopathy on twitter again. (I know tell you something new right?) and I asked for what he calls “good science” published in a peer review journal. He provided two links so for:
Which is a “Review of 4 high quality studies on respiratory allergies” according to him and “To test the hypothesis that homoeopathy is a placebo by examining its effect in patients with allergic rhinitis and so contest the evidence from three previous trials in this series.” according to the study objective. Sure why not, totally a review of 4 studies. Lets look at the paper though:

The intervention was “Random assignment to an oral 30c homoeopathic preparation of principal inhalant allergen or to placebo.” Alright, makes sense. What next? My favorite part:

This trial, the fourth in a series, was designed in response to a challenge from an independent clinical team to contest the evidence from the three preceding trials that homoeopathic dilutions seem to differ from placebo

Designed to contest evidence from THREE preceding trials. So why link to this one? No idea.

These were not trials of treatments; they were designed to address the placebo hypothesis, using allergy as a model. In this study, as before, patients with atopic inhalant allergies received, randomly and double blind, either an oral 30c homoeopathic preparation of their principal allergen or a placebo. The previous trials studied effects in atopic patients with hay fever1,2 and asthma,3whereas this study focused on perennial allergic rhinitis. We report the results of this fourth trial and an overview of the series.

Ohhhh so it also has an overview of the other studies. Good to know. Still not exactly a review of 4 studies but still. Now we know that the study was double blind and randomized right? Welll..... it's more complicated than that.

At the start of the qualification period the doctor assessed each patient's history, allergy status, and nasal obstruction. The principal allergen determining the prescription was then chosen on the basis of the largest skin test weal concordant with the allergy history. In seven cases in which the prescriber had difficulty in determining the principal allergen, telephone advice was given by a doctor experienced in prescribing homoeopathic immunotherapy. The first of three phials of placebo corresponding to the principal allergen was then administered by the doctor on to the patient's tongue. Patients were unaware that the phials contained placebo, although the researchers were not blinded.
The following two weeks served jointly as the qualification and baseline period. At the second visit, qualifying patients were randomised by a restricted technique of permuted blocks of two,7 generated from random number tables and stratified for the indicated allergen. A double blinded prescription was immediately dispensed. The trial ended with a follow up visit four weeks later.
(Bolding mine) So... not entirely double blinded then right? Why would you, in a double blinded study single blind the first round? Again, no idea. Also while the qualifications make sense why did they keep in the 7 cases that were difficult to determine? How valid was this telephone advice? I am wary of that personally but moving on.
At the same time each morning and evening patients recorded three successive nasal inspiratory peak flow measurements. Before making these measurements patients oriented themselves by noting each morning (on a 0 to 4 integer scale) how their symptoms had interfered with their sleep and, each night, rating blocked, runny, or itchy nose symptoms, sneezing, and any eye and chest symptoms. Patients then recorded their daily overall visual analogue scale score. To allow comparison with our previous trials13 the identical wording was used: “Overall today I felt . . .” on a scale of 0-100 mm, where 0 is fine and 100 is terrible. Visual analogue scale scores are a recommended measure of the severity of rhinitis.16 Adverse events, including initial aggravations of symptoms as observed in our previous rhinitis trial,2 were documented by the patients, clarified by the doctor, and recorded on standard adverse experience report forms. Any use of conventional drugs was also noted.
Any use of “conventional drugs” being noted? Why is that? Oh yeah because patients were allowed to use conventional medicine during the trial. Maybe that made some sort of difference? I would think so but then again, I am not a doctor, I am a psychologist.

Moving on to the analysis I loved this part:
With a choice of 5% significance and 80% power, we estimated that 60 patients would be required in each group to avoid false negative results.60 patients would be required. But wait... lets go back up to the top of this very study to find... “51 patients with perennial allergic rhinitis.” Well. No avoiding false negative results then. Moving on “To summarise our results, the original data19 from all four trials were pooled and analysed by an independent worker using Cochrane Collaboration meta-analysis software” Wait. So this wasn't an independent study? It was more of an, we didn't like the results of the first trials so we're going to jimmy in some more patients rather than use the data like we said we would? “Statistical heterogeneity was assumed to be present when the P value for heterogeneity was less than 0.10, and therefore the random effects model was usedOhhhh well. Lets use a P=.10 because that's totally acceptable in most trials where at least P=.05 is the norm. Alright, whatever you say.

Into the results section. Maybe they explain why they didn't even get their own stated significance levels:
Fifty one patients successfully completed qualification screening and were randomised (fig (fig1).1). Because of the exacting screening, strict qualification criteria, and the prospectively defined requirement to stop enrolment before the pollen season, we did not recruit the number of patients that the power calculation had estimated we required.
We did not recruit the number of patients that we required. So why are you continuing the study? Why didn't you screen enough people to get at least the 60 people you knew you required. Nonsense.
Initial aggravations of rhinitis symptoms were provoked more by homoeopathy than by placebo. By 48 hours after randomisation seven (29%) patients in the homoeopathy group reported a worsening of rhinitis, two with wheeze, compared with two (7%) patients in the placebo group, neither of whom had wheezing (P=0.04, Fisher's exact test). By 14 days, 11 (46%) patients in the homoeopathy group had reported adverse events, 10 of whom had rhinitis related aggravations, compared with seven (26%) in the placebo group, five of whom had rhinitis related aggravations (χ2=3.28, P=0.07). In general, most aggravations were short lived, averaging four days, and all had resolved by day 16. Aggravations of rhinitis in patients who received homoeopathy seemed to point to a good outcome. Initial deterioration was followed by subjective improvement and a corresponding improvement in nasal inspiratory peak flow. Only one patient in each group resorted to conventional rhinitis drugs, and both took them for less than four days.
Believe it or not I can see a reason this pattern may have occurred. Why? Because the first randomization set was not double blinded. It makes sense that the homeopathy group would report a worsening of symptoms since that is what patients are told to expect with homeopathy. The standard line is “you'll get worse before you get better” I should know. I used that line many times while I was in school for this stuff. Also the whole “aggravations of rhinitis in patients who received homeopathy seemed to point to a good outcome” thing makes perfect sense because it's likely that short lived events like rhinitis aggravations would regress to the mean quickly and because the initial sweep of treatments was not double blinded so over time the self induced symptoms would disappear just like the way homeopaths say they should since the person likely believes in it. Why do I say that? Because way back up at the top of the study there was this little line that mentioned that they were recruiting people that were familiar with homeopathy. Hmmmm that may have an impact when the blinding isn't exactly as is claimed.
On average, over the last two weeks after randomisation, patients who received homoeopathy had a 28% improvement compared with 3% among those in the placebo group
Well, given that in the first two weeks, as we have already seen, the homeopathy group had an increase in symptoms they had further to go to improve than the placebo group did. This change makes sense and DOESN'T make homeopathy different than placebo.
I am going to quote out much of the discussion here because it's fantastic:
Although the objective measure consistently improved in all five centres, the subjective results were better than placebo in only four of the five centres and overall there was no difference between the groups. If the objective results are valid the discrepancy in the subjective measurements may be partly due to under recruitment compounded by aggravations and possible initial placebo responses during the run-in period in both groups, perhaps reflecting the positive expectations of the participants.23 Patients with rhinitis are keen to enter studies in their quest for better symptom control.24Subjective improvement began before the end of the placebo run-in phase in both groups, and this lessened the chance of distinguishing between the groups.25,26 A larger sample size may have shown a subjective difference between the groups. More initial aggravations occurred in patients who received homoeopathy, and this may have further complicated the subjective results. The pattern of temporary worsening followed by improvement, seen in this trial and observed in clinical homoeopathy for over 200 years,27,28 is not typical of placebo.
So... objective measures are the same while subjective measures show no difference. Alright, I can buy that ignoring my previous issues with this. Did you catch the fantastic part? “ If the objective results are valid the discrepancy in the subjective measurements may be partly due to under recruitment compounded by aggravations and possible initial placebo responses during the run-in period in both groups, perhaps reflecting the positive expectations of the participants.23 Patients with rhinitis are keen to enter studies in their quest for better symptom control” That's right. The results may be partially due to under recruitment. Which the study had despite that last line there. Patients are keen to enter studies. So why did this one have under recruitment I wonder? Hmm.

There are no declared conflicts of interest for this study and I am curious to see if there is any link to homeopathy that I can find with a quick google search of the authors. Here I go:

Morag A Taylor, research associate:
I found a Morag Taylor that is a Neuroscience research assistant that seems to do falls and balance research. Doesn't seem right, but then neither does the only other Morag Taylor I can find involved in research: the one from the University of Leeds who, according tho their linked-in profile “I work as part of a large team within the colorectal cancer (CRC) research team. I am involved in several different projects looking at biomarkers of CRC both pre and post patient diagnosis, to better understand both disease progression and response to treatment respectively.” Again doesn't exactly seem like they would do a homeopathy study like this. Curious.

David Reilly, honorary senior lecturer in medicine:
Ohhh Score here:
Dr David Reilly, Scottish Government's Lead Clinician for Integrative Care & Director of TheWEL and The Healing Shift Programmes.
Honorary Senior Lecturer in Medicine, The University of Glasgow.
Consultant Physician, The NHS Centre for Integrative Care. - See more at: http://www.gcph.co.uk/events/125#sthash.yNnc2GiH.dpuf
I am leaving that link in because frankly it's too fantastic to pass up. He works in integrative care, I poked at his webpage and low and behold... he's a homeopath!

He's a psychologist focusing on dementia, depression and other mood disorders What this has to do with allergies I have no idea. Apart from the two homeopathy trials he participated in I don't see anything else in his work that points to any reason to be participating in this research.
 Charles McSharry, principal immunologist :
Oh hey someone relevant! An immunologist that deals with asthma!
Tom C Aitchison, senior lecturer in statistics :
I can't find much on this guy except links to stuff he's published which seem to be on a range of mobility issues. Maybe he works with the Morag Taylor that works in falls and balance research. Or Robert Llewellyn-jones's work with aging people with dementia. No idea to be honest.

Overall I am not impressed with good ol' Dana Ullman's “research” here. He sent me a second one but I feel no need to take that piece of work apart as it was done for me by the lovely guys over at Skeptics with a K way back in about the second episode of their podcast. Bah Dana. Do better. 

Oh, here is the second link. Just for fun. Review of 3 high quality studies on childhood diarrhea: