The Sun attempts to get a new song going on the terraces. Still not going to be selling newspapers in Liverpool though

Disclaimer: I support Liverpool FC, so it is entirely possible that the following may be a slightly skewed account of recent football history and its implications.

Last week Manchester United sacked their manager after 14 months. Since Alex Ferguson left in the summer of 2013, United have now had 10 different managers (it was announced this week that Michael Carrick was coming back for a second bash at caretaker manager).

This is despite the fact that it is very difficult to establish whether changing managers helps, with most studies citing how many different factors are at play. One masters dissertation concludes that, although it may produce a short term boost, it is generally not enough to save a club from relegation from the Premier League. So why do it?

Surprisingly this leads us into something called Ritual Scapegoating Theory, first cited at least 60 years ago as a possible explanation for managerial succession in baseball. So changing manager may or may not make much difference to performance, but it does make everyone feel better.

And it may be that the catharsis of sacking a Special One every season or so is making Manchester United supporters feel better than the two FA Cups, two League Cups and a Europa League win (that they have managed in the 13 years since Ferguson left) have. In that time Liverpool, with just three managers, have won two Premier League titles and a Champions League, UEFA Super Cup and Club World Cup as well as an FA Cup and two League Cups. Manchester City, with just two managers in that time, have won seven Premier League titles, a Champions League, UEFA Super Cup and Club World Cup as well as two FA Cups and six League Cups.

In the previous 27 years when Manchester United had just one manager, they won 13 League titles, two Champions Leagues, five FA Cups, four League Cups, and the UEFA Europa League, Super Cup and Cup Winners’ Cup, and the FIFA Club World Cup and the Intercontinental Cup. The lesson Manchester United have learned from this apparently is not that they need to minimise manager turnover.

Chelsea are even worse. They have had 13 different managers over the same period. They sacked the current England manager Thomas Tuchel after just 20 months despite him winning them the UEFA Champions League, Super Cup and Club World Cup. They sacked Enzo Maresca after just 18 months despite him winning them the UEFA Conference League and the Club World Cup. That appears to be catharsis on steroids.

Now this doesn’t matter very much when it is just about baseball or football. Despite what Bill Shankly said, life and death and our politics more generally are much more important than football. My fear is that the view that we need to sack someone whenever we don’t get the result we want immediately is leaching out of football into everything else.

Nigel Farage’s favourite phrase appears to be that “Heads must roll”. He has deployed it about the head of children’s services in Rotherham over three children being removed from their foster families in 2012, the whole of the NatWest board when he was refused a Coutts account in 2023, unspecified individuals from Essex Police managing the demonstrations outside an Epping hotel last year, and also last year again feeling that some head rolling was in order in response to the Government’s national inquiry into grooming gangs.

Meanwhile Kemi Badenoch said in 2024 of civil servants “There is about 5 per cent to 10 per cent of them who are very, very bad. You know, ‘should be in prison’ bad”. Last week she called for the Chief Constable of West Midlands Police to go over the decision to ban fans of Maccabi Tel Aviv from the Aston Villa ground for their Champions League match in November (which, hilariously, now appears to have been, at least partly, influenced by how badly Maccabi Tel Aviv supporters behaved at an entirely fictional game against West Ham United). Also in November, she called for the people involved in making the Panorama documentary about the Capitol Riots, to be sacked (another “heads should roll” headline). Also in November, she decided other BBC “heads must roll” over the decision to uphold complaints and reprimand a newsreader for altering her script from “pregnant people” to “pregnant women” on camera in a piece about groups most at risk during UK heatwaves.

It feels like the footballisation of public life. Farage and Badenoch are routinely reported as furious about something or other, in addition to the frequent demands for sackings, nearly always of public servants. They can clearly see there are votes in it, but the effects of this continuous vilification of public services and the people running them go way beyond that.

The immediate effect is often to make someone’s job untenable. So both the NatWest and Coutts CEOs left after the Farage complaint. The BBC’s Director General and BBC News Chief Executive both left soon after the Badenoch complaint. Now these are all comfortably off individuals who will no doubt be fine, but the impact on an organisation of the idea that any of its leaders could be hounded out at any moment, not through any regulatory process or indeed any process at all, but by someone in politics with a big mouth can be seen increasingly in our public institutions.

One example would be in the NHS. The HSJ recently published an article on the distribution of tenure of trust chief executives (it required four screenshots to show you just how long a tail can be!). Notice how it is not until half way up the 3rd screenshot (counting up from the bottom) that you find a CEO who has been in post for 50 months, and four of them are about to leave.

Source: https://www.hsj.co.uk/leadership/trust-ceos-time-in-post/7040645.article?storyCode=7040645

As Roy Lilley says:

Public reporting over the last eighteen months points to exits by:

  • NHS England CEO, plus
  • a cluster of senior national directors exiting.
  • At least 10 ICB chief executives stepping down or being replaced.
  • Trust CEO turnover approaching a quarter of posts;
  • implying 50 or more trust-level departures, including permanent and unplanned exits.

Taken together, a guesstimate is between 60 and 80 chief executives and equivalent senior leaders have left their roles across the NHS in the last eighteen months.

As he goes on to say:

High turnover at the top tells the rest of the system that leadership is temporary. Risk is unrewarded and survival matters more than stewardship.

Authority is put at question.

That encourages short-term-ism, compliance and caution. Precisely the opposite of what complex service reform needs.

For those fans of Manchester United’s way of running things who seem to be running things at the moment, that is a hell of a lot of catharsis. Evidence suggests that it is unlikely to achieve much else.

The catharsis never lasts, as at some point the music has to stop, and someone needs to actually be sitting in actual chairs, actually managing the thing. Just ask Chelsea and Manchester United.

Source: https://xkcd.com/2415/ licence at: https://creativecommons.org/licenses/by-nc/2.5/

Happy new year all! New year, new banner, courtesy of my brilliant daughter who presented me with a plausible 3-D model of my very primitive cartoon of a reverse-centaur over Christmas. And I thought I would kick off with a relatively uncontentious subject: examinations!

“Back to normal!” That was the cry throughout education when the pandemic had finally ended enough for us to start cramming students into rooms again. The universities had all leveraged themselves to the maximum, and perhaps beyond, to add to the built estate, so as to entice students in both the overseas and the uncapped domestic market to their campuses, and one by-product of this was they had plenty of potential examination halls. So let’s get away from all of that electronic remote nonsense and get everyone in a room together where you can keep an eye on them and stop them cheating. This united the purists who yearned for the days of 10% of the cohort turning up for elite education via chalk and talk rather than the 50% we have today, senior management needing to justify the size of the built estate and politicians who kept referring to traditional exams in an exam hall as the “gold standard”.

So, in a time when students have access to information, tools, how to videos of everything imaginable, the entire output of the greatest minds of thousands of years of human history, as well as many of the less than great minds, in short anything which has ever caught anyone’s attention and been committed to some form of media: in this of all times, we want to sort the students into categories for the existing job market based on how they answer academic questions about what they can remember unaided about the content of their lecture courses and reading lists with a biro on a pad of paper perched precariously on a tiny wooden table surrounded by hundreds of other similar scribblers, for a set period of time as minders wander the floors like Victorian factory owners.

And for institutions that thought the technology we fast-tracked for education delivery and assessment in the pandemic would surely be part of education’s future? Or perhaps they just can’t afford to borrow half a billion or have the the land available to construct more cathedrals of glass and brick to house more examination halls? Simple! We just create the conditions for that gold standard examination right there in the student’s own bedroom or the company they work for!

There are 54 pages to the Institute and Faculty of Actuaries’ (IFoA’s) guidance for remotely invigilated candidates. It covers everything from the minimum specification of equipment you need, including the video camera to watch your every movement and the microphone to pick up every sound you make, to the proprietary spying software (called “Guardian Browser”) you will need to download onto your own computer, how to prove who you are to the system, what you are allowed to have in your bedroom with you and even how you need to sit for the duration of the exam (with a maximum of two 5 minute breaks) to ensure the system has sufficient visibility of you at all times:

These closed book remote arrangements replaced the previous open book online exams which most institutions operated during the pandemic. The reason given was that the exam results shot up so much that widespread cheating was suspected and the integrity of the qualifications was at risk. The IFoA’s latest assessment regulations can be found here.

The belief in examinations is very widespread. A couple of months ago I was discussing the teacher assessments which replaced them briefly during the pandemic with a secondary business studies teacher. He took great pride in the fact that he based his assessments solely on mock results, ie an assessment carried out before all of the syllabus had been covered and when students were unaware it would be the final assessment. But still in his mind more “objective” than any opinion he might have of his own students.

If a large language model can perform enormously better in an examination than your students can without it, what it actually demonstrates is that the traditional examination is woefully unprepared for the future. As Carlo Iacono puts it:

The machines learned from us.

They learned what we actually valued and it turned out to be different from what we said we valued.

We said we valued originality. We rewarded conformity to genre. We said we valued depth. We measured surface features. We said we valued critical thinking. We gave higher marks to confident assertion than to honest uncertainty.

So now the machines produce what the world trained them to produce: fluent, confident, passable output that fits the shapes we reward.

And we’re horrified. Not because they stole something from us. Because they showed us what the systems were selecting for all along.

The scandal isn’t that a model can imitate student writing. The scandal is that we built an educational and professional culture where imitation passes as competence, and then acted shocked when a machine learned to imitate faster.

We trained the incentives. We trained the rubrics. We trained the career ladders.

The pattern recognition which gets you through most formal examinations is just too cheap and easy to automate now. It is no longer a useful skill, even by proxy. It might as well be Hogwarts’ sorting hat for all the use it is in a post scarcity education world. If the machines have worked out how to unlock the elaborate captcha system we have placed around our gold standard assessments, an arms race of security measures protecting a range of tests which look increasingly narrow compared to the capabilities which matter does not seem like the way to go.

What instead we are doing is identifying which students are prepared to put themselves through literally anything to get the qualification. Companies like students like that. They will make ideal reverse-centaurs. The description of life as a reverse-centaur even sounds like the experience of a proctored exam:

Like an Amazon delivery driver, who sits in a cabin surrounded by AI cameras, that monitor the driver’s eyes and take points off if the driver looks in a proscribed direction, and monitors the driver’s mouth because singing isn’t allowed on the job, and rats the driver out to the boss if they don’t make quota.

The driver is in that van because the van can’t drive itself and can’t get a parcel from the curb to your porch. The driver is a peripheral for a van, and the van drives the driver, at superhuman speed, demanding superhuman endurance. But the driver is human, so the van doesn’t just use the driver. The van uses the driver up.

Source: Cory Doctorow, Enshittification

And, even if you are OK with all of that, all of these privacy intrusions don’t even work to prevent cheating! The ACCA, the world’s largest accounting professional body, has just announced it is stopping all remote exams after giving up the arms race against the cheats, facilitated in some cases seemingly by their Big Four employers lying about what had gone on.

Actuarial exams started in 1850, only 2 years after the Institute of Actuaries was established (Dermot Grenham wrote about them recently here). This pre-dated the establishment of the first examination boards by a few years (1856 Society of Arts, the Society for the encouragement of Arts, Manufactures and Commerce, later the Royal Society for the encouragement of Arts, Manufactures and Commerce (Royal Society of Arts); 1857: University of Oxford Delegacy of Local Examinations (founded by the University of Oxford); and 1858: University of Cambridge Local Examinations Syndicate (UCLES, founded by the University of Cambridge)), so keen were actuaries to institute examinations. However it was the massive expansion of the middle classes as the Industrial Revolution disrupted society in so many ways that led to the need for a new sorting hat beyond the capacity of the oral examinations that had previously been the norm.

Now people seem to be lining up to drag everyone back into the examination hall. Any suggestion of a retreat from traditional exams is met by howls of outrage from people like Sam Leith at The Spectator about lack of “rigour”. However, in my view, they are wrong.

Yes of course you can isolate students from every intellectual aid they would normally use, as a centaur, to augment their performance, limit the sources they can access, force them to rely on their own memories entirely, and put them under significant time pressure. You will definitely reduce marks by doing that. So that has made it harder and therefore more rigorous and more objective, right?

Well according to the Merriam-Webster dictionary, rigorous is a synonym of rigid, strict or stringent. However, while all these words mean extremely severe or stern, rigorous implies the imposition of hardship and difficulty. So promoting exams above all as an exercise in rigour reveals their true nature as a kind of punishment beating in written form, for which the prize for undergoing it is whatever it qualifies you for. Suddenly the sorting hat looks relatively less arbitrary.

The problems of traditional exams are well known, but the most important ones in my view are that they measure a limited range of abilities and therefore are unlikely to show what students can really achieve. Harder does not mean more objective. It is like deciding who can act by throwing students out, one at a time, in front of a baying mob of, let’s say for argument, readers of The Spectator. Sure, some of the students might be able to calm the crowd, some may even be able to redirect their anger towards a different target. But are the people who can play Mark Antony for real necessarily the best all-round actors? And has someone who can only stand frozen on the spot under those circumstances really proved that they could never act well?

It also means that education ends a month or more before the exams, to allow the appropriate cramming, followed by engaging all of the teaching staff in the extended exercise of marking, checking and moderating what has been written in answer to academic questions about what the students can remember unaided about the content of their lecture courses and reading lists with a biro on a pad of paper perched precariously on a tiny wooden table surrounded by hundreds of other similar scribblers, for a set period of time as minders wander the floors like Victorian factory owners. But what if instead the assessment was part of the teaching process? What if students felt that their assessment had been a meaningful part of their educational experience? What if, instead of arguing the toss over whether they scored 68% or 70% on an assessment, students could see for themselves whether they had demonstrated mastery of their subject.

One model of assessment which is getting a lot of attention at the moment, one I am a big fan of having used it at the University of Leicester on some modules, is something called interactive oral assessment, where students meet with a lecturer or tutor, individually or in a small group, and answer questions about work they have already submitted. It is a highly demanding form of assessment, for both the students and the assessors, but it means the final assessment is done with the student present and, with careful probing from the assessors, who will obviously need to have done a close reading of the project work beforehand, you can be highly confident of the degree to which the student understands the work they have submitted. It also allows the student to submit a piece of work of more complexity and ambition than can be accommodated by a traditional exam. And it needn’t take any more time if the interviews are carried out online when set against the exam marking time of the traditional exam. Something which all the technology we developed through the pandemic allows us to do, without the need for spyware.

There are other models which also assess the technological centaurs we wish our students to become rather than the reverse-centaurs we are currently dooming too many to become. It is looking like it may be time to start telling students to stop writing and to put down their pens on the traditional exam. And perhaps the actuarial profession, who led us into the era of professional written examinations so enthusiastically 175 years ago, might now want to take the lead in navigating our way out of them?