Conspiracy theories, transparency, and confirmation bias
Regular readers of this blog will no doubt be familiar with the All Trials campaign. They are campaigning for greater transparency in clinical trials, and call for all clinical trials to be published. Although in the case of licensed medicine, regulators get to see all the data from clinical trials, All Trials make the point that it's not satisfactory that doctors simply have to take the regulators' word for it that a drug is effective. They should be able to see the data for themselves. All Trials claim, and I think all right-thinking scientists would agree, that transparency is at the heart of the scientific method.
Against this background, I was fascinated to read a recent article in the Guardian. This article claims that Evil Big Pharma are involved in a Grand Evil Conspiracy in which they "mobilise" patient groups to help in their Evil Struggle against transparency.
Some pretty bold claims.
Those claims are, according to the article, based on a leaked memo, in which details of the Evil Conspiracy are set out. Oddly, however, the article does not reveal the contents of this leaked memo. It simply asks us to believe, based on the journalist's assessment of the leaked memo, that there is definitely an Evil Conspiracy.
I can think of only 4 reasons why the article might not have revealed the memo:
- There was no space to do so. Well, I know the Guardian have been struggling financially of late, but I find it really hard to believe they couldn't have found another couple of kb of web server space from somewhere.
- The journalist felt he owed a duty of confidentiality to the Evil Big Pharma Cabal who wrote the memo. That hardly seems compatible with writing the article in the first place.
- The journalist doesn't have access to the memo. In that case, the whole article is presumably pure speculation and is not to be trusted.
- The memo doesn't actually reveal anything quite as exiciting as a Grand Evil Conspiracy, and the journalist didn't want to let the facts get in the way of a good story.
There is a huge irony here. The article talks about the importance of transparency, and how we can only make an informed judgement about something if we have access to the relevant data. The article then hides the relevant data that would allow us to make a judgement about the plausibility of the alleged Grand Evil Conspiracy.
Now, I don't know whether there is really a Grand Evil Conspiracy or not. But I do know that I can't reliably conclude there is on the basis of an article that doesn't reveal its data.
But interestingly, looking at the reaction to the article on Twitter, some people seem to be convinced that the allegations in the article are spot on. One of those people is Ben Goldacre, one of the chief architects of the All Trials campaign. Looking at his tweets, it's pretty clear he believes the allegations to be accurate.
Goldacre has largely become famous by virtue of his incisive critical thinking skills, and certainly understands the importance of transparency and having access to relevant data. So you'd have thought he'd be suspicious of the claims in the article, given that the relevant data are hidden, and demand that more evidence be provided, wouldn't you?
Well, confirmation bias is a very powerful thing.
Goldacre starts from a position of being broadly anti-industry. So when an article comes along that is written from an anti-industry perspective, it seems that all those critical thinking skills go straight out of the window.
Update 26 July, 15:20
There's a fascinating article discussing this same topic at eyeforpharma, which seems to cast serious doubt on the accuracy of the Guardian article.
Having worked within 'Big Pharma' myself, I have seen a very large, Basel-HQ'd company hide data. It's shameful.
I don't think this is a fair comparison. Without wanting to get into a big debate about 'what is science?' it's fair to say medicine is a science. Journalism is most certainly not a science, and whilst it is based upon information that a journalist will somehow acquire, the source is often secret. The confidentiality of the source is sometimes necessary to ensure that they are safe and that the public can still be informed of the information. If the primary document identifies the source then it's fair enough to hold it back.
Considering that Ben Goldacre has a column in the Guardian, it's not unreasonable to suggest that he might have had some kind of role in the story coming about and might have seen the memo for himself. I suspect that he has.
I don't want to seem overly critical, but this seems like a bit of a reactionary poke at Goldacre that doesn't seem justified.
Tom, I think you may have misunderstood. I totally appreciate that the journalist needs to protect his sources, and I'm not for a minute suggesting that he should reveal the name of the person who leaked the memo.
All I'm saying is he should report, in full, what is in the memo, rather than expecting us to rely on his interpretation of it. If the memo has identifying information that might compromise the source, for example if it's addressed to an individual, then that could easily be redacted.
Thanks for posting. Indeed it would be scandalous when industry mobilizes patient groups, but what to think about unsubstantiated allegations?
There's a big difference between a personal comment and a bit of scientific or even journalistic work. If Ben Goldacre had written a lengthy blog endorsing the story without checking then I'd agree with you, but experience indicates that tweets can be premature and opinions can change as an issue is investigated.
If he's still tweeting it in a month and hasn't checked it out then it would be entirely fair to call him on it.
Fair point, Guy, although I think when someone has as many Twitter followers as Ben has, it's probably worth being a little bit careful before tweeting something.