Dianthus Medical Blog Archive

More zombie statistics

There is an oft-quoted figure that 50% of all clinical trials are never published. It's surprisingly popular for a figure that has no evidence, as I've written about before. And since I wrote that post, another study has been published showing disclosure rates of 89%.

So I was a little dismayed when I saw an article in Nature News with the headline "Half of US clinical trials go unpublished". My original thought was that it was simply repeating the same old zombie statistic, but after reading the article, it turned out it was talking about a new study, this one to be specific.

The only problem is that the study did not show that half of US clinical trials go unpublished. To do that, it would need to look at a random sample of US clinical trials. What it did instead was it looked at a sample of 600 clinical trials that had already had results posted on clinicaltrials.gov, and it did indeed find that half of that sample had not been published.

That is really not the same thing as saying half of all trials go unpublished. This is a specific sample of trials that had already had their results disclosed on a website. So although it's true that half of them had not been published in a peer-reviewed journal, 100% of them had had their results disclosed in the public domain.

Some may argue that there is no need to publish results in a journal as well if they are in the public domain anyway. I'm not entirely sure I agree with that argument, but that's a subject for another day. However, it is entirely plausible to suggest that some of those triallists who had posted their results on clinicaltrials.gov may have considered that their job of disclosure was already done, and perhaps didn't make publication in a peer reviewed journal as much of a priority as those triallists who had not posted results on clinicaltrials.gov.

Anyway, given that the latest study tells us nothing about what proportion of studies overall have their results published, can we please knock that "half of all trials published" statistic on the head with a large baseball bat before it rises up and goes on its zombie-like way?

Update 5 December 2013, 7.15 am:

Paul Ivsin (who has also left a perceptive comment below) has written a more thorough blogpost about this study than my few brief notes above. I'd encourage you to read it.

← Sponsored dietary advice in the NHS Good pharma →

12 responses to "More zombie statistics"