Peer Review and the Research Excellence Framework (REF)

In this blog post, one of the editors of our Aspects of Tourism series, Chris Cooper, discusses peer review, writing books and chapters and research assessment exercises.

I am embarrassed to say that this is my first ever blog post, and that is only because I was persuaded by Sarah at Channel View to write on peer review over a very nice lunch at the Trout Inn by the river in Oxford! This followed a discussion on the fact that career academics are often dissuaded from writing books or book chapters because they are not seen as being peer reviewed and therefore do not count in any research assessment exercise such as the UK REF (Research Excellence Framework).

This is a simple fact of working in higher education in the 20th century; governments are looking for value for money from the investment they make in higher education and they do this by assessing an institution’s research – and funding then flows from that assessment. Logically then for a Dean or Head of Department their research funding depends upon the quality and productivity of published research from their academics and so they persuade their researchers to publish in top, peer-reviewed journals because they generate the most cash for the department. Which brings us to the conundrum: what is the best approach for an academic? Quantity of publication or quality of publications? As a former dean and head of department the answer is simple – quality – and lots of it!

So why is scholarly peer review so important when assessing research? It submits a publication to the scrutiny of other experts in the field, often part of a community of practice of say tourism, hospitality or event management. Following the review (which is advisory) editors then make the decision to publish, reject or ask for changes. The process is normally anonymous and can be done by one, two or three persons, but not usually more than that.

Scholarly peer review has become the gold standard for assessing research outputs and is most commonly used in journal publishing – but it is not without its critics. They say that the process can suffer from unconscious bias and where reviewers are chosen from a community of practice, the use of the peer review process strengthens the status quo and suppresses new ideas, innovation and creativity. And of course, like any process, it is open to abuse. Finally, with the advent of technology new approaches to scholarly peer review are emerging, including the use of social media to crowd source or have open peer reviewing.

So scholarly peer review is important, but it is less overt in book publishing than in journals, hence the in-built bias of research exercise assessments against books and for journals. For example, in the 2014 UK REF the business panel received 353 books/chapters to assess set against 11,660 journal papers, whilst the Sport, Exercise Science and Tourism panel received only 76 books/chapters and 2,685 journal papers to assess.

A number of commentators on the 2014 REF have called for a more sympathetic consideration of books and chapters. I believe that if publishers follow – and overtly publicise – a scholarly peer review approach, then books and chapters will be taken seriously in research assessment exercises and we will begin to change the views of academic managers of their value. In Channel View’s Aspects of Tourism series for example, the commissioning editors always use peer review of manuscripts and also scrutinise initial proposals carefully to preempt reviewers’ comments where possible. The peer review process is rigorous and many books in the series have gone back for revision following reviewers’ comments. So, use of the scholarly review process by academic book publishers could enhance the perceived academic value of books and chapters, so making them more acceptable to academic managers and boosting the funding to departments.

Chris Cooper, Oxford, June 2018

We are currently in the process of developing a peer review certification – watch this space! If you found this interesting, you might also enjoy our blog post Peer Review Guidelines.

2 thoughts on “Peer Review and the Research Excellence Framework (REF)

  1. These are important reflections. The big picture in how academia is being shaped in the neoliberal world is insightfully assessed in Stefan Collini’s book ‘Speaking of universities’ (Verso). Pretty disturbing reading, with a whole series of analyses leading to the conclusion that the UK iss a guinea pig in Europe for the forces that have already shaped universities in the USA, harmfully for academic freedom and university autonomy.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s