Discussing Bad Pharma on Canadian TV
Yesterday I made a brief appearance on the Canadian TV channel BNN, in which I provided a counterpoint to an interview with Ben Goldacre, who was talking about his book Bad Pharma.
As regular readers of this blog will know, this is a subject I have written about more than once before, and will no doubt do so again.
You can watch the video clip of my appearance here, and if you want to see what Ben said first, you can see that here.
← NHS Choices puts special interest groups ahead of accurate information Clinical trials and supermarket beef lasagne →
I have just looked at your answers to Ben Goldacre book and especially the one about interim analysis (link: http://dianthus.co.uk/interim-analyses)
You are discussing about the Bassler paper in JAMA.
When I first read this paper, I thought that the conclusion was obvious. With less patients or events, you need a bigger effect size to get a significant difference. It was just nice to show that it was the case “in real life”. Of course, interim analysis should be planned and the alpha threshold should be adequately decreased.
Unfortunately, the problems with truncated trial cannot be summarized to this point for various reasons
Most of the time, in a paper about a truncated trial, only intent to treat population is described. The per-protocol population is not described (if exists); neither is of course the population of patients not finishing the study. So, we don’t know if some specific kinds of patients were followed up for a longer or a shorter duration and evaluation of the beneficial effect after different time on treatment cannot be done. Timing of censoring vs drop-outs is also not often described.
For all events that can occur at a higher frequency with time, the truncated trial will lessen the difference with the placebo group. It can be the case for drop-outs or adverse events.
On the contrary, if the beneficial effects of the treatment are decreasing with time, calculation of the mean effect on the population can be unduly increased when compared to not truncated trials or to real-life benefit. This can be the case for trials where double blind is not that sure because of specific short term side effects.
Truncated trials are dealing with long duration therapies; should the beneficial effect size of the treatment in real life be the one obtained on the intend-to-treat population or on the per-protocol population?
How should truncated trials be included in meta-analysis, what duration, what number of patients, what P value, what effect size should be considered (ITT or PP)?
All the above points are only dealing with logical possible differences between truncated and not truncated trials and obviously you cannot know if the benefit of the product will apply to the full randomised population for the full duration.
At the end of this link, you can find a good example of potential bias in a truncated trial. Patients followed for a short time benefit from the placebo while others benefit from the verum, results on ITT or PP population quite different for the two outcome criteria. Would you consider this trial as biased or fair?
This trial was the only trial for approval. The product was considered as the best in class....and is a real blockbuster.
http://www.la-press.com/redirect_file.php?fileId=4504&filename=Approved-Beta-Interferons-in-Relapsing-Remitting-Multiple-Sclerosis_Odd_One_Out2&fileType=pdf
Interview masterfully navigated, sir!