Professional medical writers’ organisations such as EMWA, AMWA, and ISMPP have done their bit by publishing guidelines and position statements and educating their members about ethical publication practices, and there is some evidence that those efforts are pushing things in the right direction, but it’s clear that they are not going to solve the problem by themselves and that more needs to be done.

So what more can be done?

Well a paper recently published in PLoS Medicine by Bosch et al argues that the best way to deal with it is through legal action. Of the paper’s 3 authors, 1 is a lawyer and another declares an interest in doing consulting work for a law firm, so they would say that, wouldn’t they? It appears to be hard to talk about ghostwriting without conflicts of interest rearing their ugly head.

Now, I don’t want to completely pour scorn on the idea of legal action. It’s an interesting idea, which I suspect could be appropriate in some of the most egregious cases of publication malpractice, and would certainly make people sit up and take notice if it were successful. The deterrent effect of a couple of high profile cases could be quite valuable. Bosch et al suggest that authors of dishonestly written papers could be named as defendants in any legal action relating to harms to patients that result from physicians relying on those papers in their prescribing decisions, and if harms do result in that way, then it’s hard to argue that the authors of the papers aren’t responsible. I wonder if the action could even be extended to the journal that published the paper, although Bosch et al don’t mention that possibility.

However, I think suggesting that legal action is a sensible way forward in the majority of cases is misguided (unless, of course, you’re a lawyer or a consultant to a law firm, in which case I dare say you’ll do very nicely thank you). Legal action is time consuming and extremely expensive, and is going to be completely impracticable as a way of dealing with most unethically written papers.

I know I’ve said this before, but I think it bears repeating: we are never going to get rid of ghostwritten papers unless journal editors start to take the problem seriously (and by taking the problem seriously, I mean actually DOING something about it, not just publishing articles moaning about how terrible the problem is). But as Bosch et al rightly point out, journal editors have not shown a willingness to tackle ghostwriting. And that leaves us with a problem.

I don’t know how we can encourage journal editors to do their bit (although if they did find themselves on the wrong end of a couple of high profile lawsuits involving ghostwritten papers that they published in their journals, I suspect it might focus a few minds), but the question of editorial responsibility is something that’s cropped up in a couple of other contexts recently, and I think it’s worth drawing some parallels.

The first context is my unpleasant experience of finding defamatory material published about my company in the Journal of Bioethical Inquiry. It turned out that, unless I was willing to go down the legal route of suing for libel, which I wasn’t (did I mention that legal action is expensive and impracticable in most cases?), my attempts to have the record corrected were dependent entirely on the good will of the journal editor. If that good will was absent, as it turned out that it was, there was nothing I could do about it as there is simply no mechanism for enforcing ethical behaviour on journal editors who misbehave. Although COPE claim that they investigate complaints against journal editors, their total lack of transparency in their complaint procedure makes it hard to have confidence in the process.

My friend Karen Shashok wrote a detailed article analysing the Journal of Bioethical Inquiry case, in which she argued that a mechanism for making journal editors accountable for their actions is badly needed.

So, if there were a body that was able to hold editors of biomedical journals accountable for their actions, would they be able to take action against journals who publish ghostwritten papers? I don’t know the answer to that, as it’s all a bit hypothetical. But doesn’t it sound like an idea worth pursuing?

It’s also hard to talk about editorial accountability without thinking of the Leveson Inquiry, currently ongoing in the UK. This inquiry is focused on mass media, rather than scientific journals, but perhaps some of the ideas of editorial accountability that will come out of that inquiry could be applicable to journals too?

Legal remedies for ghostwriting Dianthus Medical - An innovative company providing outside the box analytical solutions for the pharmaceutical industry and other organisations working in the biomedical sector.

My friend Karen Shashok has recently written a very detailed account of the episode, and draws some wider conclusions about editorial accountability in general. It’s well worth a read.

Journal of Bioethical Inquiry and editorial accountability Dianthus Medical - An innovative company providing outside the box analytical solutions for the pharmaceutical industry and other organisations working in the biomedical sector.

OK, are you back? Right, let’s carry on. One of the points I made was that it doesn’t seem to me that medical journals are doing all they could to combat ghostwriting. While it’s probably not realistic to expect journals to eradicate the practice all by themselves, I do think they could very easily ask a few questions of people submitting articles to them. If journals did this routinely, it should eliminate all ghostwriting by those who are not dishonest but simply not aware of ethical standards in this area, who I suspect account for the majority of ghostwritten articles. Blatant dishonesty is a lot less common than ignorance of the latest guidelines. I and some colleagues have published a checklist that should help journals to do this, although to my knowledge, no journals have taken this up.

Anyway, my blog post prompted a discussion on Twitter, in which Trish Groves, deputy editor at the BMJ (tweeting as @trished) explained what the BMJ does in this area. It’s probably worth making clear at this point that I think the BMJ is doing vastly more than many other journals, for which it deserves considerable credit.

Trish explained that the BMJ has a series of 3 questions which they ask authors, which are designed to make it harder for ghostwriters. The questions can be found towards the bottom of this page, under the section “Who prompted this submission?”

These are good questions to ask, and should go a long way towards eliminating ghostwritten articles. However, I was a bit concerned that the BMJ page said “We may ask authors…” (my emphasis), implying that they don’t always. Trish explained that they are mainly for review papers rather than original research, and are only asked at the discretion of the BMJ editors. The reason why they are not always asked, apparently, is that there would be a significant resource implication for doing so.

Now, I don’t want to have a go at the BMJ, who, as I said earlier, are streets ahead of most other journals in this regard. But I do find it disappointing that the questions aren’t always asked, which suggests that the commitment to eradicating ghostwriting may be a little half-hearted. Surely it would actually be easier to send the same forms to everyone, rather than having to spend time deciding who gets asked these questions and who doesn’t?

I appreciate that there is a resource implication to someone having to read the answers, but if all 3 questions are answered “no”, then that really ought to take someone no more than a couple of seconds to do. If the questions are not answered “no”, then that suggests that they really needed to be asked, and there is a substantial risk of ghostwritten papers getting through if the questions are not asked.

It would be nice if the BMJ would rethink this policy and ask the questions of all their authors. It would be better still if other medical journals would follow the BMJ’s lead and start asking these questions as well (some already do, but really not many). Otherwise, I fear we are still going to be discussing ghostwriting in medical journals for many years to come.

What are medical journals doing to combat ghostwriting? Dianthus Medical - An innovative company providing outside the box analytical solutions for the pharmaceutical industry and other organisations working in the biomedical sector.

So, on to the 3rd session of the day, “Good Practice Guidelines. A Triumph of Hope over Experience?”, which was led by Charlie Buckwell from Complete Medical Group. There are many guidelines which define good practice in pharma industry publications (which for the most part are remarkably consistent with each other), and yet there is still a widespread perception that pharma industry publications are thoroughly evil. Charlie showed us a YouTube clip which gave a good idea of how some journalists, either because they are seriously uninformed or because they think conspiracy theories always sell, continue to tell untruths about how the pharma publishes its research.

Nonetheless, although that YouTube clip is an extreme example of uninformed nonsense, it is still a fact that bad behaviour by the industry in the past has left a lasting reputation of shady dealings. So a question asked by this session was whether there is still a gap between the ideal, as defined in guidelines, and the reality of how we are publishing our data. The honest answer to that is that we don’t really know, as there is a lack of data. Charlie did point to my own data which shows that ghostwriting is becoming less common, although we still don’t really know how common it is.

Many people in the room said that they always advised clients on good practice and would turn down work from a client who wanted to work in an unethical way, and I have no doubt that that is true. What worries, me, however, is that the audience consisted of people who are highly engaged the mainstream medical communications community and are up to speed with current ethical standards, whereas there are no doubt other people working at the fringes of the industry who take such things less seriously. Are there many companies out there working unethically? And if there are, how do we reach them and encourage them to work more ethically? I don’t know the answers, but as I mentioned during the session and have argued elsewhere, I really think that medical journals could do more to enforce ethical publication standards.

The final session was called “Does Transparency Lead to Trust? Or Just Loss of Control?”, and we were fortunate to have Dr Ben Goldacre on the panel. Ben comes at the issues around publishing clinical trials from a very different perspective to most people in the room, and is not afraid to challenge other people’s opinions. That’s a thoroughly good thing, as it guards against sliding into lazy groupthink. Having said that, I was convinced that we are all on the same side. We all want to see clinical trial results published in full and in a transparent and ethical manner.

However, we didn’t seem to be agreed about how much progress had been made. Most people in the room believed that, although we have probably still not reached the ideal of publishing all our trials in a fully transparent manner, we have made huge progress towards it in recent years and that the trend is moving inexorably in the right direction. That’s a view I hold myself, partly reassured by recent research showing that, although the pharma industry can be slow to publish results, it does eventually publish results more completely than non-industry researchers do.

Ben, on the other hand, did not agree. He started off by telling us the story of reboxetine, in which 74% of the data (most of which was negative) had not been published. This was a clear case of unacceptable publication bias, but largely involving trials done in the 1990s, when standards of completeness of publication were different to how they are now. I find it hard to believe that a similar situation could occur now, but Ben told us he didn’t believe anything had really changed. He used the phrase “transparency theatre”, to describe the pharma industry putting on an outward show of being transparent, while continuing to hide unfavourable data in reality.

This isn’t a description I recognise from working with our clients, who are mostly keen to ensure they are complying with legal requirements to post study results on the clinicaltrials.gov website within the 12 month time limit, and many other comments from the assembled delegates expressed similar surprise. However, Ben described a piece of research that he has done recently, but not yet published, which he claims shows that only 20% of pharma-sponsored clinical trials comply with the requirement to post results within 12 months of study completion. Without seeing that publication, it’s impossible to comment further on those results, but I look forward to seeing them when they’re published.

Despite some disagreements and some reason to think that there may still be serious problems with the way pharma publishes its trials, I thought the day ended on a very optimistic note. There was a real feeling of momentum towards striving for ever greater transparency and high ethical standards. I and many of the others in the room (including, wonderfully, both Ben Goldacre and Phil Hammond) have agreed to continue working towards these goals. Exactly what form that work will take will no doubt have to be thrashed out over the next few weeks and months, but I suspect it will have to focus first on identifying exactly where the main problems still exist, and then how we are going to solve them.

I am looking forward to being a part of those activities. Watch this space. And, as ever, if you have any thoughts on how we can improve transparency in clinical trials publications, do feel free to add your thoughts via the comments box below.

Strategic MedComms Forum 2011 part 2: guidelines and transparency Dianthus Medical - An innovative company providing outside the box analytical solutions for the pharmaceutical industry and other organisations working in the biomedical sector.

The day was split into 4 sessions, each with an expert panel and a session leader. Thankfully, use of PowerPoint was kept to a bare minimum, and most of the day was spent in discussion, rather than listening to someone’s presentation. The day was also widely discussed on Twitter, and Peter had the fantastic idea of preserving the day’s tweets for posterity, which are well worth a read.

The first session—“Innovation, Transparency and Excellence from Early Phase Onwards. Partners or foes?”—covered a range of issues including motivations for drug development and the role of pharma company marketing departments. One important theme that came out is that the old models of pharmaceutical development, characterised by a top-down, command and control approach, are looking increasingly outdated and doomed to failure. The future is more networked.

One revelation for me from this session is that I became convinced that pharmaceutical marketing is being done dramatically wrong. Marketing departments are currently very much involved in products that are already licensed, but have far less involvement with drug discovery. That’s completely the wrong way round. In an ideal world, drug companies, through their marketing departments, would be engaging in meaningful dialogue with patients right at the start of the process. They should be listening to patients far more before they ever go near a lab, to find out what the needs of patients really are. Then they can go off to their labs to try to meet those needs. No doubt some companies are already working this way, but it seemed to most people in the room that that is the exception rather than the rule.

Our chairman for the day, Dr Phil Hammond, made a fascinating point that sounded totally valid to me. He pointed out that the HIV patient community have traditionally been highly engaged with those who treat them, and that progress in HIV drugs since HIV first emerged in the 1980s has been phenomenal. Perhaps that’s not a coincidence.

I had the privilege of leading the second session, entitled “Sharing data. Opening the Pandora’s Box?”. We asked what data should be shared and what data actually are shared, which of course are two very different things. Most people in the room agreed that sharing data, in the form of making raw data from clinical trials publicly available, was a good thing. It facilitates evidence-based-medicine by making life far easier for any meta-analysts who may be interested in the data, and it also discourages dishonest analyses. Anyone who knows that their data could be independently re-analysed has a powerful disincentive to do anything suspect when analysing the data.

However, despite this widespread agreement on the benefits of data sharing, few were actually doing it in practice. We discussed some of the barriers, and I was heartened that the argument of protecting intellectual property did not come up. I’ve never really believed that one: if you are publishing a study anyway, then it’s hard to see why there could be any intellectual property concerns about making the data available.

Nonetheless, there are indeed many reasons why people don’t make their data available: some good, some not so good. For me, the most serious one is the ethical issue of consent and patient confidentiality. Clearly, no-one would release raw data that included patient names. However, I don’t think it’s safe to assume that simply omitting names removes all problems of confidentiality. It would often not be impossible to identify an individual patient if you knew their age, sex, diagnosis, and which hospital was treating them: all data that are likely to be easily available if raw data are released. To release such raw data without patients’ explicit consent would therefore be ethically questionable.

This is, in principle, an easy problem to solve: when writing consent documents for clinical trials, a paragraph could be added to explain the intention to share the data from the trial, and patients’ explicit consent could be requested. I imagine most patients would readily agree. That consent, however, should not be assumed. Sadly, no-one in the room was aware of it being standard practice in their companies to ask for such consent. I sit on a research ethics committee and read a lot of patient consent documents, and I don’t remember ever seeing this mentioned. Therefore, one action point from the meeting for me is to recommend that this be mentioned in any consent document in trials in which I am involved from now on.

This will make it easier to share raw data in the future, but it is still not clear to me how easy it is going to be to share the mountains of data from existing trials.

This was not the only reason why data are not being shared, of course. It seemed that the other main one was simply one of inertia: most companies do not have procedures in place for doing so, and in the absence of suitable incentives, few companies seem keen to invest the (potentially significant) resources needed for putting such procedures in place. One suitable incentive might be if some journals required raw data to be shared as a condition of acceptance. We learned at the meeting that this is already routine in some other scientific fields, such as genomics, although I’m not aware of any journals doing this in clinical medicine. Are any willing to take this on?

I seem to have written quite a long article so far just on the first 2 sessions, so I shall spare you having to read about sessions 3 and 4 for now. I’ll write about them soon, so do come back to find out, among other things, what Ben Goldacre thought of those of us working for the pharmaceutical industry.

As always, I would love to read your comments via the form below.

Strategic MedComms Forum 2011 part 1: marketing and data sharing Dianthus Medical - An innovative company providing outside the box analytical solutions for the pharmaceutical industry and other organisations working in the biomedical sector.

The objective of this short survey is to gain feedback as to how much trial data pharmaceutical companies should make public. The types of questions asked within the survey include, “Should pharma make all their data public, and if so, how where should they publish their data”, and “What are the limitations on publishing all data?”. The survey is aimed at professionals involved in the development, publishing or planning of medical publications.

If you are involved in publishing medical research, I’d be really grateful if you could spare a few minutes to take part in the survey, which you can do by clicking here.

And please feel free to tell any of your colleagues about it too.

Should pharmaceutical companies publish all their data? Dianthus Medical - An innovative company providing outside the box analytical solutions for the pharmaceutical industry and other organisations working in the biomedical sector.

Details here.

Medical writing training, October 2011 Dianthus Medical - An innovative company providing outside the box analytical solutions for the pharmaceutical industry and other organisations working in the biomedical sector.

I have not blogged about it before, because such things are better dealt with in private. This is particularly true if legal action is pending, although in the end, despite the fact that the journal refused to correct the untrue statements published about us, I decided that the cost of mounting a libel action against the journal would be prohibitive. As one great legal mind once put it, “Justice is open to everyone, in the same way as the Ritz Hotel is”.

Sadly, despite my best efforts, the astonishingly irresponsible and inaccurate journal article remains uncorrected, and it is now time to tell the story.

I have written about this little episode in some detail for The Write Stuff (journal of the European Medical Writers Association), and you can read the full article here. Briefly, the journal article in question accused us of writing ghostwritten articles in the medical literature. These articles, it was alleged, presented a distorted and biased view of clinical data to fit in with company marketing messages. No specific examples were given, which of course is inevitable, because there aren’t any. We take publication ethics very seriously, and we would never participate in the kind of behaviours described. To suggest that we do is totally false.

Most of us regard peer-reviewed scientific journals as a source of trustworthy information. We expect the information in them to be factually accurate, even if wise readers know that conclusions drawn from data may not always be reliable. We expect that if information is discovered to be incorrect, then a journal will issue a correction, or possibly a retraction.

But what happens if a journal editor just chooses to allow incorrect information to stand? It turns out that, at least for a small organisation that can’t afford the costs of libel lawyers (could this be why the authors of the article picked on us, and not on one of our better known and richer competitors?), there is little one can do about it.

I had hoped that the Committee on Publication Ethics would be able to rule on the matter. Sadly not. They did agree to look at the case, but gave an utterly bizarre response. They claim that they investigate breaches of their code of conduct, but they also told me that they will not comment on the facts of the case. And apparently, without considering the facts of the case, they determined that their code had not been breached. I cannot see how they could possibly know whether their code had been breached without looking at what actually happened. Presumably they just took the journal’s word for it.

In any case, as their process was conducted behind closed doors and no detailed reasons for their decision were given, how they arrived at their decision is destined to remain a mystery. For an organisation that claims to be concerned with publication ethics, of which transparency is such an important part, to have such an opaque decision making process is deeply worrying.

We rely on journal editors to maintain the accuracy of the scientific record. I find it worrying that if they deliberately shirk that responsibility, there is apparently no-one in a position to do anything about it.

Just as one final irony, the article that prompted all this was about the accuracy of the medical literature.

Update, 24 June:

It turns out that COPE’s lack of transparency is worse than I thought. On their website, they state “All the cases COPE has discussed since its inception in 1997 have been entered into a searchable database” and make those cases available on their website. Except that they don’t include complaints. Details of complaints are not made public at all.

I really cannot understand why an organisation that claims to be an authority on publication ethics feels the need to conduct their affairs with such secrecy. Doing things in secret is not good publication ethics.

Quis Custodiet Ipsos Custodes? Dianthus Medical - An innovative company providing outside the box analytical solutions for the pharmaceutical industry and other organisations working in the biomedical sector.

EMWA conference 2011 Dianthus Medical - An innovative company providing outside the box analytical solutions for the pharmaceutical industry and other organisations working in the biomedical sector.

Why?

Well, if you are going to write a scientific paper criticising someone for introducing bias into the scientific literature, would it be too much to ask that you should do it in an unbiased way? What makes me grumpy is when people write papers about how evil and biased the pharmaceutical industry is (and this is certainly not the first such paper), but then themselves distort the facts to make a point.

Now, I’m not saying that the pharmaceutical industry is populated entirely by saints. It is true that some of the things that some people in the industry have done have indeed distorted the scientific literature. That is undeniably a bad thing. However, I do think it’s important to stick to the facts, and not to embellish them to make things sound worse than they are. If a paper misrepresents some facts, how can I trust anything else in the same paper? There may be some important points to be made, but by mixing them up with distortions and untruths, the whole message is weakened.

And that’s a shame, because the integrity of the medical literature is important.

So, which paper am I talking about, and why do I think it’s distorted? The paper, by Joel Lexchin, is entitled “Those who have the gold make the evidence: how the pharmaceutical industry biases the outcomes of clinical trials of medications”. Not that that’s an emotive title or anything.

I haven’t extensively fact-checked the paper, but it did cite a couple of pieces of evidence with which I was already familiar, and I couldn’t help but notice that those pieces of evidence had been badly misrepresented.

One of the things we are told in the paper is that industry-sponsored papers have a discordance between their results and conclusions, and that results are “spun” to yield favourable conclusions. The evidence cited to support this is a paper published in the BMJ in 2007 by Yank et al. That is what Yank et al concluded, but sadly their methods were deeply flawed, as I pointed out at the time. They assessed spin with a single reviewer, who was aware of the study hypothesis, and was not blinded to whether papers were sponsored by industry or not. The potential for bias in a study like that is huge, but Lexchin made no attempt to acknowledge the limitations of Yank et al’s study, presumably because it supported his argument.

In fact there is some other evidence, not cited in Lexchin’s paper, that argues against the idea that industry papers “spin” the results in their conclusions (at least not any more so than anyone else). A far more careful study by Boutron et al did not find any difference between industry-sponsored studies and independent studies in the amount of spin in their conclusions (this wasn’t reported in the paper, for some strange reason, but was clarified in subsequent letters.

Interestingly, one of the ways in which it’s possible to spin results is to write a paper that pretends that a favourable secondary outcome was the primary outcome all along. Why did Lexchin not mention that? Could it be that a recent paper by Bourgeois et al found that a mismatch between pre-specified and reported outcomes was significantly less likely in industry-sponsored studies? That’s a highly relevant piece of information if you’re trying to answer the question of whether the industry are any worse than anyone else at spinning results, and yet it was entirely left out of Lexchin’s paper. Ignoring evidence that doesn’t fit with your preconceived notion of what the results should be is itself a pretty serious form of spin.

The paper also talks about ghostwriting. As regular readers of this blog will know, that’s something of a specialist subject of mine. What really annoys me is when people write about ghostwriting and don’t make the distinction between ghostwriting and legitimate, transparent assistance from professional medical writers. Guess what? Lexchin doesn’t once mention the latter. Much of the discussion about ghostwriting is therefore rather confused.

But one bit struck me as a particularly heinous crime against accuracy: a claim that “evidence points to [ghostwriting] being widespread and systematic”. The “evidence” (and trust me, I use that term loosely) cited in support of that claim was a paper by Ross et al published in JAMA in 2008.

So, how widespread exactly did Ross et al find that ghostwriting was? Well, I don’t know, because Ross et al did not present any quantitative data whatsoever on the prevalence of ghostwriting. They found, using a very broad definition of ghostwriting (which again, confuses ghostwriting with legitimate and transparent writing assistance) that “some” papers were ghostwritten. Now, perhaps as I statistician I have greater expectations than most people that data should come with numbers, but I don’t see how anyone can conclude that something is “widespread” just because it happens “sometimes”. Oh, and Ross et al’s paper looked only at publications on a single drug sponsored by a single company, so even if it did present quantitative data, it would be pretty bizarre to try to extrapolate its results to the entire pharmaceutical industry.

When Lexchin cites such mind-crushingly weak evidence to make his points, it’s clear that either he has made no attempt whatsoever to critically evaluate the literature he cites, or he is deliberately misrepresenting it just to fit his agenda. I honestly don’t know which would be worse. Either way, it makes it very hard to believe anything I read in the rest of the paper.

I could go on about this paper for much longer, but I imagine you’re getting slightly bored of reading this by now, so I think I’ll stop there.

All that remains is to say that the integrity of the medical literature is hugely important. It is literally a matter of life or death. We know that much of the medical literature is biased or otherwise imperfect, and that’s something that deserves to be taken extremely seriously. It’s certainly possible that the pharmaceutical industry may be responsible for biasing the literature, although as I’ve argued elsewhere, the evidence that the pharmaceutical industry does any worse than anyone else in this regard is pretty weak. It is important that we all—industry or otherwise—make an effort to improve standards in reporting of clinical trials, and highlighting any shortcomings in the way things are done at the moment is an important part of that.

But if we’re going to look at shortcomings, it is essential to do so in an unbiased and scientific manner. Distorting the evidence to fit a political agenda is not helpful. You would hope that people who write papers about other people distorting the evidence would know better.

Bias in papers about bias Dianthus Medical - An innovative company providing outside the box analytical solutions for the pharmaceutical industry and other organisations working in the biomedical sector.