SAGE debates value of Research Excellence Framework

The Research Excellence Framework, the process which assesses the quality of university research and drives funding,  is an "enormously costly" system which does not pay its way and is "a key driver of inequality in institutions", a debate at the Institute of Contemporary Arts heard yesterday (8th December).

The debate, organised by SAGE to mark the launch of a new series of polemical shorts, SAGE Swifts, was held this week before the latest set of REF outcomes are published (Thursday 18th December). It heard Lancaster cultural studies professor Derek Sayer and Warwick University English professor Thomas Doherty attack the REF system, which sees a lengthy and complex process of panel assessments, while David Sweeney, research director at the Higher Education Funding Council for England (HECFE) and former university minister David Willetts defended it.

Sayer, the author of Rank Hypocrisies: The Insult of the REF, a SAGE Swift to be published in December, said the REF system had grown into an enormously expensive and laborious exercise, without producing value for money in its results. The exercise costs nearly £60m: £47m of that within universities and £12m in HEFCE administration costs.

But REF panels put in place to assess the quality of research "lack expertise to the the job for which they are trusted by the British taxpayer," claimed Sayer, commenting that panel members are "eminent" but often lack detailed specialist knowledge, while they also lack the time to do the job they are supposed to do. "Around 1,000 REF panellists grade nearly 200,000 outputs in less than a year. They say they skim-read it – this is for the rigorous assessment we pay a lot of money for," he added. The adoption of metrics instead would deliver "comparable outcomes to REF for a fraction of the financial and other costs," he claimed.

Meanwhile Doherty, the author of SAGE Swift volume Universities at War, claimed that REF was set up "to legitimise the cutting of funding for research", calling it "a key driver of inequality in institutions and society at large", adding that REF was "part of a project of driving us to conformity" when "research is, if about anything, about breaking conformity" and exploring ideas and projects that may or may not work.

However Willetts said there needed to be a defensible system of research assessment that did not simply involve an elite making judgements over dinner at the Athenaeum club, and only used metrics for assessment if they were legitimate, trusted and credible. "Ultimately its a question of cost benefit analysis," he said. "If we can find a mechanism that is trusted by the academy but less work, my successor [as universities minister] will go for it. The growth of REF has been a result of the academy itself wanting to have that."

Sweeney said the decisions on allocating resources must depend on assessing and rewarding performance. "You may reject this, but I find it difficult to see how it can be other," he said. "We have a system we [the academy] have designed ourselves within guidelines set by the government, and that is a pearl not to be given up."

SAGE Swifts will offer polemical arguments published in digital form and in print for libraries, of 30-50,000 words, with a 12-week turnaround from submission to publication.