Peer Review Week 2018

This week is Peer Review Week, an event which aims to promote the vital importance of good peer review in scholarly communications. Peer review is central to what we do as academic publishers, and over the years we’ve written about this topic on the blog a number of times. In celebration of Peer Review Week, here are three of our top blog posts on the subject…

The Worthwhile Challenge of Peer Review

A post giving an overview of what peer review is and why we need it, and how the peer review process works here at Channel View/Multilingual Matters.

Peer Review Guidelines

Peer review is central to academic publishing, yet many academics receive no training on how to do it. In this post our Editorial Director, Anna, offers some guidance on the whats, hows, dos and don’ts of peer review.

Peer Review and the Research Excellence Framework (REF)

A post by one of the editors of our Aspects of Tourism series, Chris Cooper, in which he discusses why scholarly peer review is so important when assessing research.

For more information about Peer Review Week, check out the website here.

Peer Review and the Research Excellence Framework (REF)

In this blog post, one of the editors of our Aspects of Tourism series, Chris Cooper, discusses peer review, writing books and chapters and research assessment exercises.

I am embarrassed to say that this is my first ever blog post, and that is only because I was persuaded by Sarah at Channel View to write on peer review over a very nice lunch at the Trout Inn by the river in Oxford! This followed a discussion on the fact that career academics are often dissuaded from writing books or book chapters because they are not seen as being peer reviewed and therefore do not count in any research assessment exercise such as the UK REF (Research Excellence Framework).

This is a simple fact of working in higher education in the 20th century; governments are looking for value for money from the investment they make in higher education and they do this by assessing an institution’s research – and funding then flows from that assessment. Logically then for a Dean or Head of Department their research funding depends upon the quality and productivity of published research from their academics and so they persuade their researchers to publish in top, peer-reviewed journals because they generate the most cash for the department. Which brings us to the conundrum: what is the best approach for an academic? Quantity of publication or quality of publications? As a former dean and head of department the answer is simple – quality – and lots of it!

So why is scholarly peer review so important when assessing research? It submits a publication to the scrutiny of other experts in the field, often part of a community of practice of say tourism, hospitality or event management. Following the review (which is advisory) editors then make the decision to publish, reject or ask for changes. The process is normally anonymous and can be done by one, two or three persons, but not usually more than that.

Scholarly peer review has become the gold standard for assessing research outputs and is most commonly used in journal publishing – but it is not without its critics. They say that the process can suffer from unconscious bias and where reviewers are chosen from a community of practice, the use of the peer review process strengthens the status quo and suppresses new ideas, innovation and creativity. And of course, like any process, it is open to abuse. Finally, with the advent of technology new approaches to scholarly peer review are emerging, including the use of social media to crowd source or have open peer reviewing.

So scholarly peer review is important, but it is less overt in book publishing than in journals, hence the in-built bias of research exercise assessments against books and for journals. For example, in the 2014 UK REF the business panel received 353 books/chapters to assess set against 11,660 journal papers, whilst the Sport, Exercise Science and Tourism panel received only 76 books/chapters and 2,685 journal papers to assess.

A number of commentators on the 2014 REF have called for a more sympathetic consideration of books and chapters. I believe that if publishers follow – and overtly publicise – a scholarly peer review approach, then books and chapters will be taken seriously in research assessment exercises and we will begin to change the views of academic managers of their value. In Channel View’s Aspects of Tourism series for example, the commissioning editors always use peer review of manuscripts and also scrutinise initial proposals carefully to preempt reviewers’ comments where possible. The peer review process is rigorous and many books in the series have gone back for revision following reviewers’ comments. So, use of the scholarly review process by academic book publishers could enhance the perceived academic value of books and chapters, so making them more acceptable to academic managers and boosting the funding to departments.

Chris Cooper, Oxford, June 2018

We are currently in the process of developing a peer review certification – watch this space! If you found this interesting, you might also enjoy our blog post Peer Review Guidelines.

The worthwhile challenge of peer review

Scholarly publishing is built on peer review –we just couldn’t survive without it. As such, it is an important part of our editorial process at Channel View Publications / Multilingual Matters. The concept of publishing something “scientific” that hasn’t been through some form of blinded approval process is quite frightening – we could be publishing completely unfounded results with wide-reaching and very serious implications (the meningitis vaccine controversy being a perfect example).

"Wikipedian Protester", by Randall Munroe (CC-BY-NC 2.5)
“Wikipedian Protester”, by Randall Munroe (CC-BY-NC 2.5)

Having said that, the nature of peer review can and does vary according to subject, format, and potentially publisher and editor too. For example, most journals will use double-blind peer review, regardless of whether social science and humanities or STM (scientific, technical and medical). Books will often use a single- or editor-review, often because the reputation of the author will have a bearing on book sales, so the publisher needs an insight into that reputation as well as reassurance that the work is academically sound. Some more applied or practitioner journals will use editor review only (see the glossary for more explanation of these terms). In all cases reviewers are expected to highlight any potential or actual problems and make a recommendation to the editor. They are not expected to replicate results, make direct changes to the manuscript or prove any suspected misconduct, but they are expected to comment on factors such as originality, coverage, relevance and structure.

Source: https://www.flickr.com/photos/ajc1/6735929719/(CC-BY-SA 2.0)
Source: Flickr 

Peer review is under a lot of scrutiny at the moment – and there’s good reason. It isn’t a perfect system, and most people in the industry can tell a tale of peer review gone wrong (see http://retractionwatch.com/ for some recent examples). With the number of working-paper repositories in existence now, a determined reviewer can probably find out an author’s identity. Reviewing isn’t typically taught, so early-career researchers tend to hone their reviewing skills through trial and error. Editors say finding good reviewers is getting harder due to increasing pressures on researcher’s time. Reviewing is an often unpaid, often unacknowledged part of academic life, but it’s expected of every researcher. As pressures on academics grow, it stands to reason that some will simply stop doing what they’re not rewarded for. But out of discontent comes innovation, and there is certainly innovation and experimentation going on in peer review at the moment. Some publishers are publishing the reviews along with the finished article (including the reviewer’s names). Some allow comments and post-publication review on their platforms. Some publishers obligate their authors to do a minimum number of reviews. Companies are building systems to track and prove peer review has occurred (see http://pre-val.org/).

At Channel View Publications / Multilingual Matters we agree that peer review is of huge importance, and we won’t publish anything that hasn’t been through a minimum of single-blind peer review – and some manuscripts are rejected following review. The books that go on to be published can change dramatically after peer review – many of our authors comment how useful they found the process and how much their manuscript improved following review. We pay our reviewers in cash or books – we acknowledge this is an important and very valuable job that takes time, care and expertise, and we think they should be rewarded for that. Reviewers are also always sent a free copy of the book they reviewed when published so they can see the results of their hard work. We really appreciate our reviewers – so to those of you that have read and commented on a proposal or a draft manuscript, whether this year or ten years ago, thank you so much! We couldn’t do it without you.

Glossary:

  • Desk reject: a submission is rejected before any review has taken place (usually this is when a submission is completely irrelevant or inappropriate for the publication)
  • Double-blind: The author doesn’t know who the reviewer is, and the reviewer doesn’t know who the author is.
  • Editor Review: the editor reviews the submission without the assistance of another reader
  • Open Review: Neither the reviewer nor the author are anonymous
  • Pre-publication review: peer review done before publication
  • Post-publication review: peer review done after publication, usually in an open online forum
  • The terms reader, referee and reviewer are interchangeable
  • Single-blind: The author doesn’t know who the reviewer is, but the reviewer is aware of the author’s identity.

Kim