Breast cancer screening and peer review
I've been thinking some more about the paper on breast cancer screening that I blogged about last week.
Just to recap, a paper was published last week claiming that the benefits of breast cancer screening comfortably outweigh the harms. This paper was picked up by the media, who reported its conclusions almost entirely without any critical evaluation, simply taking the authors conclusions as established fact.
However, as I previously pointed out, the conclusions are based on some extremely precarious calculations about how often breast cancer screening leads to harm. IMHO, the conclusions simply can't be trusted.
The paper is, in fact, so deeply flawed, one might be tempted to ask how it got through peer review?
I have a theory about how that happened.
The calculations of breast cancer harms are based on some complicated mathematical modelling. I didn't understand how they arrived at the equations they used. I showed the paper to 3 of my colleagues (including another statistician), all of whom also failed to understand where the equations came from. So the logic of the modelling is far from having been clearly explained.
My guess is that the peer reviewers of the paper did not follow the logic leading to the equations, but didn't like to admit that. There is some evidence that when people are faced with impressive-sounding but actually meaningless information, they often don't like to admit that they don't understand it. Perhaps that happened here? The first equation in the paper contains an obvious typo (the right hand side of the equation is a negative quantity, which is clearly impossible for an incidence rate), so if the peer reviewers had been following the maths, they would surely have spotted this.
So, my theory, for which I admit I don't have proof, is that the paper was not peer reviewed in any meaningful sense, because the peer reviewers simply couldn't follow the logic of the paper but didn't like to admit it, and so stayed silent.
We all know that peer review is a deeply flawed process. This may be another example of how it can go wrong.