An economic analysis of which journal to choose for your publication
I saw a fascinating question asked on Twitter the other day about choosing a journal for submission of your latest research paper. The question was asked by @deevybee (aka Prof Dorothy Bishop from Oxford University), who had been discussing the best target journal with a colleague (let’s call him “Al”, as we all know that a great many co-authors on papers seem to have that name).
Dorothy wanted to submit to PLoS One, whereas Al wanted to submit to a more prestigious journal. The advantage of publishing in PLoS One is that they have a very high acceptance rate and are fast (oh boy, are they fast: they took less than 2 minutes to reject my latest paper), so the paper is likely to be published quickly. Submitting to a more prestigious journal may achieve a higher-value result in the end (if you believe that the value of a publication is related to the prestige of the journal), but the paper may be rejected, causing delays while another journal is found, and even if accepted it is likely to take longer than the speedy PLoS One. Dorothy wanted to know how to convince Al that the advantages of rapid publication outweighed any potential extra kudos of a higher-ranked journal.
So, who is right? I don’t know the answer to that, as it depends on various assumptions as we shall see below, but it occurred to me that we can borrow some tricks from economists to arrive at a rational answer of which is the better journal to choose.
The important economic concept here is that of discounting. Sometimes people find this confusing once you start thinking of it in mathematical formulas, but the underlying concept is remarkably straightforward and intuitive. Discounting is a way of measuring how we attach less value to events that happen in the future compared with the same events that happen now. It’s easy to illustrate the concept. Suppose I asked you whether you would prefer, other things being equal, to have your paper published today or next year. That would be a no-brainer. Discounting is simply a way of measuring the strength of that preference.
A discount rate, normally expressed as a percentage, is the decrease in value associated with a unit time (normally a year). If you had decided to use a discount rate of 10%, and I asked you whether you would rather have your paper published in a reasonably prestigious journal in 1 year from now, or in a journal with only 90% of that prestige, but published today, then in theory you should have no preference between those options. If you did have a preference, then you have probably chosen the wrong discount rate.
Let us assume that we have 2 options: we can submit our paper directly to PLoS One, or we can submit to a more prestigious journal, and then subsequently submit to PLoS One if the first journal rejects it. We can calculate a net present value (NPV) for each of those courses of action, and the strategy with the higher NPV is the rational choice.
(In practice, we could submit to another, intermediate prestige journal if we are rejected by our first journal, and only submit to PLoS one if rejected there as well. That would make the calculations more complicated, but no different in principle. For the sake of simplicity, I have assumed that we’re only submitting to one other journal before reverting to PLoS One.)
Now, to calculate the NPV, we need some data. Some data are reasonably objective and easy to measure, others are more a matter of judgement. For a start, we need to decide what discount rate to use. NICE uses a discount rate of 3.5% when looking at cost effectiveness over time in its health technology appraisals, but I suspect that is far too low for our purposes. The discount rate probably depends on how rapidly moving a particular scientific field is: in some fields, a paper may be almost worthless if it is delayed by a year, and then it would be appropriate to use a very high discount rate. Determining the appropriate discount rate is not easy, it is subjective, and it depends on individual circumstances. I’m going to use a discount rate of 20% here, but I should stress that that figure is no more than a completely arbitrary guess.
The next thing we need to do is to rate the value attached to the different publications. Let’s assign an arbitrary value of 100 to PLoS One. The number you would attach to the more prestigious journal will be more than 100, and again is a matter of judgement. How much value do you attach to being published in a “better” journal? Please don’t tell me that you can measure it by impact factor, as that is a very crude and imperfect measure of the worth of a journal. It really comes down to individual judgement. For the sake of illustration, let’s say our other journal has a value of 150. But in reality, that is a difficult number to pin down, and probably the most subjective part of this whole calculation. I expect if you asked 5 researchers to judge the value of the same journal you’d get 5 completely different answers.
The other bits of data are easier to determine: we need to know the time to either rejection or publication in each journal, and the probability of acceptance. For simplicity, I shall assume that the probability of acceptance in PLoS One is 100%, although in practice it is less (I know it’s less than 100%, because they rejected my paper!) Let’s also assume that PLoS One takes one month to publish. Finally, let’s assume that our more prestigious journal takes 9 months from submission to publish a paper if it accepts it, and 3 months to make a decision to reject, and that we have a 20% chance of being accepted.
If we plug in all those numbers, we see that if we submit to PLoS One our paper is worth 92 points in year 1 (not discounted, but we only have it published for 11 months of the year), 80 points in year 2 (published for the whole year, but discounted by 20%), 64 points in year 3, etc. If we add the numbers over a 5 year time horizon (let’s assume that everyone will have forgotten about our paper in 5 year’s time) we get a total NPV of 328 points.
If our paper is submitted to, and accepted by, the better journal, it is worth 37.5 points in year one (worth 150 points for a whole year, but only published for 3 months of that year), 120 points in year 2 (150 points discounted by 20%), etc, and worth 392 points over the full 5 years. However, if it is rejected, it is worth 67 points in year 1 (published in PLoS One for 8 months of the year), and then the same as if it had been published in PLoS One all along for subsequent years, giving a total of 303 points. Since the probability of acceptance is 20%, we take a weighted average of those figures (0.2 × 392 + 0.8 × 303), giving an overall NPV of 321 points.
There are, of course, a number of simplifications in that calculation, and more importantly, some wild guesses about the numbers to input into it. However, if those wild guesses turn out to be right, then the rational strategy for Dorothy et Al should be to submit their paper to PLoS One.
If anyone would like to play around with their own figures for looking at this, I have an Excel spreadsheet which I’ll be happy to email to you if you get in touch.