This piece in the Times Higher Education is just too important not to re-post the link here!
I spend a lot of time doing reviews, receiving reviews (including nasty ones), and exploring the process of peer review in workshops with research students. It is something I care passionately about, both for its value and because it is far from perfect.
This piece in the THE is wonderful for the way in which senior, experienced academics show there is nothing to be ashamed of in receiving a nasty review from a peer. I often say that if you ask a senior professor if they have had a really unpleasant, overly harsh, aggressively worded review then (1) they are lying, (2) they are deluded, (3) they must never have published anything that has gone through peer review.
So, I’m happy to declare that I’ve received plenty of nasty reviews. One questioned my professional integrity, accusing me of wasting participants’ time (when all my evidence suggested the contrary was going on). Another unhelpfully criticised me for only providing 8 references, when this was a rule stipulated by the journal for the kind of piece I had submitted. Another suggested my research was half baked and I should come back when it was done. Ouch.
Personally, I love Andrew Oswald’s “strategy that I would recommend to any young scholar, I do not commit nasty comments to memory”. My version of this involves taking a thick black marker pen and blotting out all the unnecessary and unhelpful vitriol. Like the CIA do when they release classified documents.
I also like the idea, when you get a nasty review, of showing it to as many people as possible, being proud: “Hey! Look at this gem I got! It’s even worse than the one you showed me last week! Clearly this referee has some major issues…”
Do I respect the idea that peer review has a place in academia? Absolutely. There is a need to both police and improve knowledge that has the benefits, impact and potential danger of scholarly legitimacy.
Do I recognise that the process has become, perhaps has always been, flawed? Absolutely. Which is why I spent lots of time helping students and early career researchers to ‘read’ reviews in a productive way, to understand the bizarre, messy and unprofessional things that go on, and to respond agentically to it, rather than review being something that is done on or happens to them.
Would paying reviewers help? Probably not. Would publishing their reviews, with their names next to what they wrote? Well that might remove the cloak of anonymity and some of the mischief that goes on. But then people might not say yes so often.
Better editing might also help in some cases. Rather than copying and pasting, or just pressing ‘forward’, good editors should intervene, and mediate the communication between reviewers and authors, and this can extend to not passing on unhelpful, unprofessional, or unethical comments.
We have to remember peer review is not done by the best experts who know most about a particular topic. It is done by who will say yes to doing it (usually for free) when they have lots of other things to do. While journals may have guidelines they ask reviewers to follow, there are few well-developed pedagogies for reviewers (I submit some of the work I’ve been doing with research students as a humble attempt to address this in a small way).
As the THE piece states, peer review is flawed because it is done by human beings. But this is also what makes it possible to improve the knowledge we present to the public and on their behalf.