RAE madness

As every university teacher and administrator in the UK knows, the results of the Research Assessment Exercise were announced last Thursday. The reaction since then has been fairly sick-making, especially from those who came well out of this dreadful process.

Me included I’m afraid. According to most calculations Cambridge came out “top” overall (unless you use a different method of calculation, which lands the LSE at the top of the table). And in the Classics race alone, Cambridge came top, “beating” Oxford by a whisker. And yes, I’ll ‘fess up, I have shared a self-satisfied smile or two about that with colleagues here since I got back.

But hang on. Have we all forgotten what a dreadful process this is? And isn’t it the responsibility of those who have done well out of it to speak out loudest against it? For them, at least , it doesn’t look like sour grapes.

Now I don’t mean to insult all those hardworking academics (over a thousand of them) who gave up weeks of their time to assess the submitted work (and make the process as fair as it could be). And I don’t mean to insult all my colleagues in Cambridge who spent years trying to make sure that our submission was a good as possible. In fact I'm really grateful to them. Besides I was on my own Faculty’s RAE committee and spent many hours strategizing.

But lets remember that the real point of this exercise is to divide up inadequate research resources under the cover of “objectivity”. But how objective can it ever be – when the main element of the process involves grading each of up to four “outputs” written by every academic “lucky” enough to be entered into one of four/five grades:
4* -- world leading in terms of originality, significance and rigour;
3* -- internationally excellent in terms of originality, significance and rigour, but which nonetheless falls short of the highest standards of excellence;
2* -- recognised internationally in terms of originality, significance and rigour.;
1* -- recognised nationally in terms of originality, significance and rigour;

and “unclassified” -- falling below the standard of nationally recognised work, or outside the guidelines in some way.

OK, there must be some open and shut cases, but an awful lot must be much closer calls between 4*, 3* or even 2*. And if we knew the individual gradings, I bet there would be some we would take issue with.

And this is not to mention the vast amounts of university time and resources that went into trying to figure out how to work the system best (would it be financially better in the end to submit more academics, even if the average “score” went down?), assembling all the information, writing the department’s statement about itself, and chasing up all the submissions which had to be sent off to the RAE HQ somewhere.

And it’s not to mention either the nasty culture of competition that resulted, as strategic appointments were made – and anxious deans put pressure on anxious heads of department, who then put pressure on their overworked staff to come up with another book….It’s no secret that all this chased some people out of the profession, and made others terminally miserable.

Less discussed are the bigger changes of academic culture that have resulted. In particular, there is now a tremendous pressure and incentive to publish – and no longer much sense that it might be possible to have a good idea without getting it print. There is too much publishing going on in British academic life, not too little – and that has been encouraged by the RAE.

Just to make matters worse, most newspaper accounts have failed to get to the bottom of the complicated grading system in use. The results came for each department (I mean "Unit of Assessment") under 5 heads, with the percentage of ‘outputs’ submitted by each department awarded each of the 5 grades (with some adjustment for various other factors also taken into account and which are far too complicated to explain). My own Faculty scored like this:

4* - 45%; 3* - 25%; 2* - 30%; 1* 0%

That does NOT mean what many newspapers thought it meant – that 45% of Cambridge classicists were world leading (though that might also be true!). It means that 45% of the individual submissions  (up to 4 per academic) were judged world leading. Most of us will have made submissions that were given different ratings – our “big book’ maybe getting a 4*, a more popular book something ‘lower’ etc, etc..

In my case, as I remember, I submitted my Triumph book, my Parthenon book, an article on Cicero’s letters and an article on William Ridgway. I would (now I’ve seen the figures) be a bit surprised if the Triumph didn’t get a 4*, but the Parthenon was a book with a lot of research behind it aimed at a more general audience – which should not have got a 4* (but was nevertheless an important contribution – I think – for an academic to make).

Oh well this is last RAE. The trouble is that its replacement (which almost certainly will be more mechanistic) will almost certainly be worse.