It is understandable that when you find yourself in that situation, you will be prepared to clutch at straws. If someone offers you a hope of a cure, you’d take it, wouldn’t you, even if you knew that the chances of a cure were slim?

Well, that is what the Burzynski Clinic is offering. They are offering a treatment called “antineoplaston therapy”, for which they charge tens, if not hundreds, of thousands of dollars. That would all be worth doing if antineoplaston therapy had shown evidence of being able to cure brain cancers.

But, despite the marketing efforts of the Burzynski Clinic, there is sadly no evidence that that is true. And it’s not because antineoplastons are a new treatment that have not had time to be properly researched. The Burzynski Clinic have been running trials with antineoplastons since the 1980s, but despite that, they have yet to provide any evidence from randomised controlled trials that the treatment is effective. While I realise that absence of evidence is not the same thing as evidence of absence, I really believe that if no evidence has been found after all that time, then the only reasonable conclusion to draw is that antineoplaston treatment does not work. I would be delighted to be proved wrong, of course, but it’s hard to see that happening after 3 decades of research have failed to come up with anything convincing.

There are further reasons to doubt the bona fides of the Burzynski Clinic. Dr Burzynski has previously been successfully sued for fraud, having attempted to claim reimbursement from a health insurance company for antineoplaston treatment when it was not covered by the insurance. It is also hardly the act of a reputable institution that, when faced with blogs criticising antineoplaston treatment, the clinic uses vague and unprofessional legal threats, rather than attempting to explain why the critical bloggers were mistaken.

I’m afraid the only conclusion I can draw is that the Burzynski Clinic, whether knowingly or perhaps through simply not understanding the limitations of their treatment, are taking advantage of vulnerable and desperate people, charging them life-changing sums of money for treatments that have no realistic hope of any benefit.

There are some other excellent blogs about the Burzynski Clinic, which I encourage you to read:

False hope

How much does hope cost?

The weird world of US ethics regulation

The false hope of the Burzynski Clinic

The Burzynski Clinic

Controversy surrounding Burzynski’s ‘pioneering’ cancer therapy should be reported in newspapers

And finally, Ducks are nuthin’ but trouble, which contains a list of links to other blogs on the subject.

The problems with the Burzynski Clinic are thus very well documented in the blogosphere. What I don’t understand is why the mainstream media have been silent about, or even complicit in, this scandal.

The Burzynski Clinic Dianthus Medical - An innovative company providing outside the box analytical solutions for the pharmaceutical industry and other organisations working in the biomedical sector.

This is important. Malaria is a terrible disease, which kills almost a million people a year, most of them children, and almost all of them in developing countries. And over 200 million a year suffer non-fatal, but still thoroughly miserable malaria infection. Although various treatments for malaria exist, they are not always available to everyone who needs them in resource-poor countries, and drug-resistant strains of malaria represent a huge challenge for treatment. While mosquito control measures can have highly beneficial effects, they have not been anywhere near sufficient to provide adequate control of the disease in practice.

Clearly, a vaccine that would protect against malaria would be a huge advance, but an effective vaccine has long eluded researchers.

Until now.

The results of this new study, with the not-very-snappily named RTS-S/AS01 vaccine, developed by GSK Biologicals with assistance from the Bill and Melinda Gates Foundation, are exciting because they show that the vaccine works.

The results of the trial do clearly show efficacy. It was a well conducted trial: it was randomised and double-blind, and used standardised definitions of malaria infection that included laboratory confirmation. The sample size of 6000 children gave good statistical power. The analysis was done in various different ways (using slightly different definitions of malaria infection, analysis of both intent-to-treat and per-protocol populations, and statistical adjustment for potential confounding variables) and the results of all analyses were remarkably consistent and highly statistically significant. It really doesn’t look possible that the lower rate of malaria infections seen in the malaria vaccine group could have been a chance effect.

Now, as a statistician, my first instinct is to be cautious. So that is what I’m going to do first. You’ll have to wait until the end of this post to find out what I’m going to do second.

One caveat to bear in mind is that, although the vaccine was effective, it did not offer complete protection against malaria. The vaccine efficacy was just a little over 50%, meaning that it only halved the risk of malaria, and did not eliminate the risk. Nonetheless, not only would a 50% reduction be clinically meaningful in itself, this is also a very important proof of concept study, showing that it is possible to protect against malaria. It is now reasonable to expect that improved vaccines, offering better efficacy, will be developed in the future.

It is also important to realise that this is a single study. It is seldom wise to get too excited about the results of a single study. Although the results are consistent with small, phase II studies of the same vaccine (for example this one), independent replication of the results in another large study, ideally in a different study population, would greatly increase our confidence in the results.

It’s also worth noting that this study did not show any reduction in deaths from malaria. That doesn’t mean that it would not be expected to reduce malaria deaths, it’s just that deaths were rare in the investigated population, and the study simply didn’t have enough statistical power to investigate whether the lower incidence of malaria infection translates into fewer deaths. It would be good to have evidence that it does before getting too excited about the vaccine.

My final note of caution is about the safety of the vaccine. Although the vaccine seemed reasonably safe in this study, it is hard to be too confident based on a study of this size. A much larger safety database will be necessary to confirm that there are no rare but nasty side effects lurking. The study did hint towards a possible association between the vaccine and meningitis, which is probably just a chance effect, but will certainly need to be monitored carefully in future studies before we can be sure. There also seemed to be a small risk of seizures following vaccination (about 1 per 1000 doses). All children in this study recovered from the seizures uneventfully, but again, we’ll need a larger safety database to be sure that there isn’t a meaningful risk of harm here.

Anyway, that’s the cautious stuff done. Now for my second reaction to the study.

WHOOOOOO HOOOOOOOO! A malaria vaccine that actually works! This is AWESOME!

Malaria vaccine Dianthus Medical - An innovative company providing outside the box analytical solutions for the pharmaceutical industry and other organisations working in the biomedical sector.

The research, published in PLoS One, describes a radical new approach to antiviral treatment.  It relies on the fact that most viruses produce long sequences of double-stranded RNA, which is rare in mammalian cells: our cells generally only produce short sequences of double-stranded RNA. In an ingenious technique, the researchers have found a way of killing cells with long sequences of double-stranded RNA.

This, at least in theory, has the advantage of being a very general method of attacking viruses, which means that not only should it be effective against a wide range of viruses, but it should also be difficult for viruses to develop resistance (although viruses are devious little buggers, so no doubt they’d find a way eventually).

The research is at an early stage, and is some way off clinical trials in humans. There are many things that could go wrong between here and eventual clinical use. If you asked me to guess, I’d have to say the chances are that in the end it won’t work, at least as currently envisaged.

However, results in cell culture and even in mice do look extremely promising. If this does end up working in humans as well, it will be unbelievably awesome, and potentially the most exciting advance in medical science for decades.

Watch this space.

Exciting development in antiviral research Dianthus Medical - An innovative company providing outside the box analytical solutions for the pharmaceutical industry and other organisations working in the biomedical sector.

However, I’m not convinced.

The finding is based on a relatively small number of studies. Only 6 studies were used in the meta-analysis of the duration of symptoms. 4 of those studies were rather unimpressive, showing either non-significant reductions in the duration of symptoms, or a reduction of less than 1 day. The overall finding of a significant reduction in symptom duration was largely driven by 2 small studies (48 and 50 patients respectively), which found much larger reductions.

This is consistent with a pattern known as publication bias: small studies with significant positive results get published, small studies with negative results don’t get published. It’s possible that other small studies have been done, but not published, because their results were negative, and that inclusion of those studies would move the overall results to being negative.

Assessment of publication bias is an important part of a systematic review. Oddly, the methods section of the Cochrane review tells us that they looked for publication bias, but I can’t find anywhere in the results section a description of what they found.

I think that’s quite important here. Without knowing more about how likely publication bias is (and a quick glance at the results suggests it is quite likely), it’s hard to know whether to believe these results. I am not convinced that zinc supplementation is truly a useful treatment for the common cold. Further research, as they say, is needed.

Zinc and the common cold Dianthus Medical - An innovative company providing outside the box analytical solutions for the pharmaceutical industry and other organisations working in the biomedical sector.

The podcast is not intended as a comprehensive news roundup (the ICR website does that job nicely), but a chance to have a more in-depth look and a bit of a chat about a few of the news stories that particularly caught our attention. In the first episode, we talk about active controls in clinical trials, the Academy of Medical Sciences report on clinical research, a new FDA requirement for informed consent forms, the North-West Exemplar Project, a drug scandal involving a diabetes drug and the French regulator, and the sad news of the closure of Pfizer’s Sandwich plant.

You can find the podcast, and subscribe to future ones via the RSS feed, on the CRFocus page of the ICR website, or if you want to go straight to the podcast, you can find it here.

I shall certainly be listening to the podcast regularly, and hope to be back as guest presenter from time to time as well.

New podcast from the Institute of Clinical Research Dianthus Medical - An innovative company providing outside the box analytical solutions for the pharmaceutical industry and other organisations working in the biomedical sector.

Does it?

Well, it was certainly an excellent piece of research, which tells as a great deal about the effect of daily aspirin on the risk of cancer death. It seems pretty clear-cut in that respect: taking aspirin every day reduced the risk of dying of cancer (not all types of cancer, but enough types of cancer that the overall death rate from cancer was significantly reduced). It’s important to note that this was based on data from randomised controlled trials, and is therefore much more reliable than epidemiological studies. It was also based on multiple large trials, including over 25,000 patients, so the study was large enough to be statistically robust.

However, I don’t think this is the the end of the story for aspirin. We already knew that aspirin reduces the risk of cardiovascular disease, and we now know that it reduces the risk of cancer. But aspirin also has harms, which need to be weighed against those benefits, the most important of which is an increased risk of gastrointestinal bleeding.

The balance of benefits and harms is likely to vary from one person to another. Someone with risk factors for cancer and heart disease but not for gastrointestinal bleeding is likely to benefit from aspirin, whereas someone with a history of gastrointestinal bleeds would be more likely to be harmed by aspirin. I don’t see anything in this new paper that gives an unambiguous answer to how we determine who would benefit from aspirin and who wouldn’t.

One important statistic in the paper was the effect on all-cause mortality, in other words not just deaths from cancer, but deaths from anything else as well. If all-cause mortality were reduced in the aspirin patients, then that would give a pretty good clue that the overall benefits outweigh the overall harms, at least in patients similar to the ones studied (although it’s still not a perfect measure, as it ignores any benefits or harms that don’t affect mortality, such as non-fatal gastrointestinal bleeding, which can seriously ruin your day even if it doesn’t kill you).

The results here were a little inconsistent. In-trial all-cause mortality (ie deaths among patients who were still taking part in the trials at the time of death) was slightly, but significantly reduced, from 11.1% to 10.2%., P = 0.047. Taking account of deaths occurring after the trials as well, mortality was significantly reduced after 15 years of follow-up (hazard ratio 0.92, 95% CI 0.86 to 0.99, P = 0.03), but not after 20 years of follow-up (hazard ratio 0.96, 0.90 to 1.02, P = 0.37). On the whole, it does appear that aspirin reduced all-cause mortality, but the results are suggestive, rather than compelling.

It’s also important to note that many of the patients in the trials had cardiovascular risk factors, and their all-cause mortality may have been more favourably affected than the general population, because of aspirin’s effect on cardiovascular mortality. So there is no guarantee that those figures for all-cause mortality would generalise to everyone.

I don’t think we are quite at the point where aspirin should be recommended for everyone over the age of 50 who doesn’t have any contra-indications (such as a history of gastrointestinal bleeding). We still need a better estimate of the effect of aspirin on all-cause mortality, particularly in different groups of patients with different combinations of risk factors, as well as a better assessment of exactly what the trade-off is between non-fatal benefits and risks. Maybe when further research has given us those answers it will become the default position for everyone to take aspirin, although I suspect it’s always going to remain a personal choice, based on an individual assessment of the importance of different outcomes.

As for me, I’m not going to start taking aspirin every day, but I will certainly keep watching the literature with interest. Another positive paper for aspirin might just convince me that I should start taking it.

Should we all take a daily dose of aspirin? Dianthus Medical - An innovative company providing outside the box analytical solutions for the pharmaceutical industry and other organisations working in the biomedical sector.

Oh dear.

These EDMs are based on 3 published papers in the peer reviewed literature, which claim to show homoeopathy is effective. As anyone who has taken the workshop that I run for EMWA on critical reading of medical literature will know, just because something is published in a peer reviewed journal does not mean it is true.

So let’s look at the papers.

The first is a randomised, double-blind, placebo-controlled study of homoeopathy in the treatment of insomnia. It appears to show a significant benefit for homoeopathy over placebo. However, there are a number of problems with the study. First, it is based on a small sample size (N = 30), so even if the results are legitimate, the difference could very easily be a statistical type I error (ie obtaining a P value of < 0.05 just by chance, which happens quite often). But there are also some deficiencies in the paper that make the results less credible. First, although they report results from 30 patients, 33 patients started the study and 3 dropped out and weren’t analysed. We are not told what happened to those 3 patients (2 of whom were in the homoeopathy group). OK, it’s only 3 patients, but in a study of this size, excluding them from the analysis could easily skew the results, turning a non-significant result into an apparently significant one.

The statistical methods are also a little opaque. It seems odd that they used the Kruskal-Wallis test, at test generally used for comparisons of 3 or more groups, when only 2 groups were compared. OK, the Kruskall-Wallis test can be used to compare 2 groups, but generally it isn’t, so its use here starts to ring alarm bells about the level of statistical expertise behind the paper.

There is also a lot of emphasis placed on significant improvements from baseline within the homoeopathy group. That’s not surprising: patients enrolled in a clinical trial generally get better. The important statistical test is between the homoeopathy group and the placebo group at endpoint, and although that’s also significant, it is less so, and receives less emphasis in the paper.

But perhaps most worryingly, the extent of data presentation is very limited, so there is no way to know whether the authors’ conclusions are plausible. It would be usual to present some measure of variation of the outcomes (for example a standard deviation), but this does not appear anywhere in the paper. All we are shown is an average value, and we are not even told whether those average values are means or medians.

When the quality of reporting is that poor, it is hard to trust the analysis presented.

The second paper is a double-blind randomised trial of homoeopathy vs fluoxetine in the treatment of moderate to severe depression. This has already been discussed comprehensively on Michael Grayer’s blog. The big problem with the study is that it was designed as a non-inferiority study, and such a design means that you need to be absolutely sure that the comparator treatment is effective. Well, there are no doubt some patients who benefit greatly from fluoxetine, but in unselected patients, most antidepressants are not all that much more effective than placebo. There was no attempt to exclude placebo responders from the study, so what we are probably seeing is that there was simply a large placebo effect in both groups. Another problem with the study is that there were substantial numbers of dropouts in each group, and the way in which the missing data was handled is unclear in the paper, but likely to be of great importance.

The third study is an in-vitro study of the activity of a homoeopathic preparation against breast cancer cells. Well, that’s been discussed pretty comprehensively elsewhere (here and here), and I have little to add to that, except to say that I too am dismayed that a paper with no statistical analysis whatsoever somehow managed to slip through peer review.

Let’s just hope that most of our elected representatives are not taken in by all this daft mumbo-jumbo.

Worrying scientific illiteracy among our elected representatives Dianthus Medical - An innovative company providing outside the box analytical solutions for the pharmaceutical industry and other organisations working in the biomedical sector.

So I thought I’d go and read the paper that claims to have investigated the effects on 5-a-day and come and report back to you. But I didn’t. The paper is behind a paywall, and although I’d be interested to read it, I’m not interested enough to want to fork out $32 of my hard earned cash to read it.

This is outrageous.

This research is potentially of great public health importance, and deserves to be freely available in the public domain. More importantly, the research, which was part of the EPIC project, was largely funded from government money. I have therefore already paid for this research through my taxes.

So why am I, as someone who has paid for the research, not allowed to read it? This is not acceptable. If any of the candidates at the forthcoming election tells me that they will make it compulsory for publicly funded research to be available in open access journals, they are considerably more likely to get my vote.

In the meantime, if anyone happens to have a copy of the paper and would like to email it to me, I shall report back on whether I believe the headlines are a good reflection of the research.

Open access publishing Dianthus Medical - An innovative company providing outside the box analytical solutions for the pharmaceutical industry and other organisations working in the biomedical sector.

Just to recap, a paper was published last week claiming that the benefits of breast cancer screening comfortably outweigh the harms. This paper was picked up by the media, who reported its conclusions almost entirely without any critical evaluation, simply taking the authors conclusions as established fact.

However, as I previously pointed out, the conclusions are based on some extremely precarious calculations about how often breast cancer screening leads to harm. IMHO, the conclusions simply can’t be trusted.

The paper is, in fact, so deeply flawed, one might be tempted to ask how it got through peer review?

I have a theory about how that happened.

The calculations of breast cancer harms are based on some complicated mathematical modelling. I didn’t understand how they arrived at the equations they used. I showed the paper to 3 of my colleagues (including another statistician), all of whom also failed to understand where the equations came from. So the logic of the modelling is far from having been clearly explained.

My guess is that the peer reviewers of the paper did not follow the logic leading to the equations, but didn’t like to admit that. There is some evidence that when people are faced with impressive-sounding but actually meaningless information, they often don’t like to admit that they don’t understand it. Perhaps that happened here? The first equation in the paper contains an obvious typo (the right hand side of the equation is a negative quantity, which is clearly impossible for an incidence rate), so if the peer reviewers had been following the maths, they would surely have spotted this.

So, my theory, for which I admit I don’t have proof, is that the paper was not peer reviewed in any meaningful sense, because the peer reviewers simply couldn’t follow the logic of the paper but didn’t like to admit it, and so stayed silent.

We all know that peer review is a deeply flawed process.  This may be another example of how it can go wrong.

Breast cancer screening and peer review Dianthus Medical - An innovative company providing outside the box analytical solutions for the pharmaceutical industry and other organisations working in the biomedical sector.

Anyway, the research has now been published, so as promised, here are some thoughts.

Bottom line: I’m not convinced.

There are two parts to the paper, one estimates the benefits, and one estimates the harms. The part that estimates the benefits seems reasonable up to a point, although there are some problems. Using data from a randomised controlled trial of screening, they calculate that 323 women would need to be screened every 2-3 years for 7 years to prevent one death from breast cancer. That figure is probably OK, given that it comes from a randomised trial. However, they then go on to estimate that extending the screening period from 7 years to 20 years would have a proportionally greater effect, and hence prevent one breast cancer death for every 113 women screened. That’s called extrapolating beyond the limits of your data, and is a bad thing. It makes an assumption that hasn’t been tested.

It’s also worth noting that they only look at breast cancer deaths. That’s probably a reasonable measure of the benefits, but it would be more convincing if they could show a benefit on overall mortality. Classification of death isn’t always as simple as it should be.

The harms from breast cancer screening, as I mentioned yesterday, result from false-positive diagnoses: in other words, being told you have breast cancer when in reality you don’t. This can lead to anxiety, distress, and possibly unnecessary treatment. It is not a trivial matter. So the authors of the new paper tried to estimate how common false-positive diagnoses were.

Sadly, when we look at how they estimated the risk of false-positive diagnoses, the problems really start.

I found their methods totally unconvincing. They did not directly measure false-positive diagnoses (which, to be fair, would be quite hard to do), but attempted to estimate them based on the number of cases diagnosed compared with the number of cases that they would have expected to be diagnosed. As you can imagine, there are a great many assumptions involved. How do you know how many cases are “expected”? Some assumptions were probably reasonable, but were nonetheless guesses. And the real problem is that the final answer was highly sensitive to the assumptions they used.

A technique that is often used in mathematical modelling in medical statistics is called sensitivity analysis. This is where you allow your estimates to vary over plausible ranges and see what effect it has on your conclusions. They didn’t do this.

So I’ve had a go at doing it for them. And the results are not encouraging.

One of the items they estimate is the relative incidence in breast cancer 7 years after the randomised trial started compared with at the beginning, taking into account age and time trends. They estimate it as 1.35. Well, it turns out that the precise value used has a huge effect on the outcome, assuming that their equations are correct (which I’m actually not too sure about anyway, as I found it hard to follow the logic they used to come up with their equations, and the obvious typos in their first equation didn’t help). If in fact the real value were 1.25 instead of 1.35, then you would calculate about 4 times as many false-positive diagnoses. When there is that much sensitivity to the inputs of the calculations, then it’s foolish in the extreme to rely on the results. And that’s just one example. There are other assumptions involved as well.

Now, don’t get me wrong. I’m not saying that breast cancer screening is necessarily a bad thing. But it undoubtedly has harms as well as benefits, and despite all the positive stories in the news yesterday, I don’t believe we are even remotely any wiser about how common those harms are.

I shall never be invited to attend breast cancer screening myself, for obvious reasons. But if I were a woman, I genuinely don’t know whether I would accept an invitation or not. I hope some better research will be done that will make the decision easier for those who are faced with it. Breast cancer screening has been around for a long time, and it’s shocking that we still know so little about its outcomes.

Breast cancer screening part 2 Dianthus Medical - An innovative company providing outside the box analytical solutions for the pharmaceutical industry and other organisations working in the biomedical sector.