Nutted by Newton

03. July 2026 · Write a comment · Categories: Actuarial, AI, Economics, Education, Professions, Risk

Isaac Newton may or may not have been nutted by an apple. His friend William Stukeley, whose memoir of Newton was the source of the story, states it as follows (spelling and punctuation from Stukeley’s manuscript at the Royal Society):

“why should that apple always descend perpendicularly to the ground,” thought he to him self: occasion’d by the fall of an apple, as he sat in a comtemplative mood: “why should it not go sideways, or upwards? but constantly to the earths centre? assuredly, the reason is, that the earth draws it. there must be a drawing power in matter. & the sum of the drawing power in the matter of the earth must be in the earths center, not in any side of the earth. therefore dos this apple fall perpendicularly, or toward the center. if matter thus draws matter; it must be in proportion of its quantity. therefore the apple draws the earth, as well as the earth draws the apple.”

However there may be stronger grounds for believing that Newton nutted English mathematics and, as a result, the actuarial profession.

I thought all this to myself, occasion’d by participation in the Actuarial Teachers and Researchers Conference (ATRC) this week. It was great. There were lots of interesting talks, and people were very engaged and respectful in how they discussed them. There was a lot of expertise in the room about both education and actuarial practice and theory. It felt like it was a room that could take on difficult topics and make progress in tackling them.

There were several calls for the need to change assessment, one of them by me, whether due to large student cohorts and the difficulties of engaging them, or the ease with which AI can duplicate examination solutions or because the credentials provided by those assessments seem increasingly irrelevant to the actual actionable skills, knowledge and experience required to operate successfully as an actuary. And there was an equally robust response from the Institute and Faculty of Actuaries (IFoA) to the effect that assessments would not be changing significantly in the latest changes due to come in by 2029.

My favourite presentation was by Angus Macdonald, about his recent short paper on Newton, Leibniz and Actuarial Science. It conjectures that the argument between Leibniz and Newton over who invented calculus resulted in the stagnation of mathematics in English universities and meant that actuaries needing to gain respectability for their advice were forced to create a professional body (in company with architects, accountants and engineers) rather than rely on universities to develop actuarial thinking as happened in continental Europe. This stagnation (Macdonald quoted G.H.Hardy, talking about the Cambridge mathematics exams, saying that they had “. . . effectively ruined serious mathematics in England for a hundred years”) was felt across the whole Anglosphere, with attempts to create institutions with equal status to the traditional universities leading to the establishment of business schools in the USA in the late 19th and early 20th centuries.

This explained a lot for me. The suspicion of our actuarial courses amongst some of our mathematics colleagues at times at Leicester. The occasionally uneasy relationship between the IFoA and the accredited universities where students can gain exemptions from their exams. The horror from some at the very idea that an actuary might qualify as a result of university courses, like they can in the Netherlands, for instance. ATRC seemed the appropriate place to explore these ideas.

Angus Macdonald considers that the Newtonian and Leibnizian branches have nearly rejoined, but I feel that our actuarial education system still suffers from the long shadow cast by these two 17th century gentlemen and their personal enmity. The ferocity with which some within the profession opposed what they saw as a threat to the popularity of the fellowship qualification posed by the new chartered actuary designation was hard to explain when the number of practising certificates issued for chief actuaries, pension scheme actuaries, etc was only around 1,100 (out of over 17,000 fellows), but if the whole profession originated from a highly developed form of status anxiety compared to academics, it becomes much easier to understand. As they say, the apple doesn’t fall far from the tree.

Now, nearly 180 years on from this “unstoppable” “rush to respectabilize” (according to Jeremy Paxman) I think it is legitimate to ask:

  1. Why does the IFoA still insist on detailed accreditation of individual university courses on basic mathematics, statistics and business economics? Do they not trust them to teach them right? Do they think that there is something better about learning these universal subjects via a bespoke course created by a tiny professional body with 34,000 members globally?
  2. Why are they so concerned about what assessment methods the universities use? Do they consider that they have more educational expertise on this than some of the leading universities in the world?

I have written previously about the changes in assessment I think are necessary and floated some ideas about where actuarial education might sit to accommodate these. My view was that the IFoA, faced with the need to innovate at all levels of its education system at a time of great uncertainty, might wish to get out of the foundational mathematics and business education tuition of the core principles subjects and leave this to the university system, in a truly Leibnizian way.

However there is a problem. The university system is facing a tough time. UK higher education is shrinking according to the UCU branch at Queen Mary’s, which has set up a a live page of all the redundancies, restructures, reorganisations, and closures taking place across the UK Higher Education (UKHE) sector. The numbers from the Higher Education Statistics Agency (HESA) bear this out, with overall student numbers falling, driven by a 10% fall in overseas entrants with a non-European Union permanent address.

In actuarial science, the list of UK universities offering accredited courses has not changed for some time, and it is a small group within the university system. These courses must be seen as vulnerable. I don’t think it is a coincidence that it is so hard to persuade one of this tiny band to run the ATRC. This week’s was the first since 2021 and the list of past hosts is not very long:

Source: https://atrc.le.ac.uk/

The IFoA needs to ask itself what it would do if there were some high profile closures of actuarial courses and whether perhaps it might be time to protect the core mathematical education arrangements for actuaries through a broad brush accreditation of a wider range of courses, rather than the very detailed accreditation process it currently carries out on core principles subjects.

I also doubt that it has the capacity on its own to significantly restructure its assessments beyond the objective based assessments it has developed to work within its proctored environment.

The core practice and specialist subjects leading to fellowship are the areas where actuaries have, rightly, the strongest views about content and assessment structures. It surely makes sense that the expertise within the profession and the university system are pooled in a joint endeavour rather than risking both failing independently.

In contrast to Newton, who had quite a temper, Leibniz was famously parodied by Voltaire in his Dr Pangloss character in Candide who believed that “All is for the best in this best of all possible worlds.”

Let’s not be too Panglossian about the current state of actuarial education.

Reservoirs: this raises the question of what is meant by “safe”

24. June 2026 · Write a comment · Categories: Birmingham, Climate Change, Media & Politics, Regulation, Risk

This week I attended one of the National Emergency Briefing screenings near me. While I wasn’t a fan of the Celebrity Gogglebox presentation style, with Chris Packham on the sofa seeing what Jennifer Saunders and others thought about the climate emergency, it did give me enough of an idea about the individual presentations to want to follow them up, and there is a really good website to do so here.

The two points which struck me particularly were the following:

  1. Professor Tim Lenton warning about what would happen if the weakening Atlantic Meridional Overturning Circulation (AMOC) hit a tipping point which, in some models, “shows that London would be -20°C in three frozen months of the year, and Edinburgh would be -30°C in five and a half frozen months of the year. And yet the summers will still be hotter than today because it’s a 2°C warmer world.”
  2. Professor Hayley Fowler warning of the risks of mega floods: “These storms can produce two thirds of a year’s rainfall in just a couple of days. Over London, that would mean about 35cm of rainfall falling over a large area. A flood of this scale will be a national crisis. Recovery would take years.” She also warned that by 2050 1 in 4 properties – 8 million in England – will be at risk of flooding.

And since flooding often follows a heatwave such as the one we are currently experiencing, I thought I would look at the risk of long-term flooding in a bit more detail, which is when I came across the government’s check the long term flood risk for an area in England website. When you put in your postcode, it assesses your flood risk under four different categories:

It then explains how they measure the risk and the kinds of things which tend to affect it. All in all a good resource, and the modelling behind it is what the 1 in 4 properties at risk of flooding by 2050 is based on.

In July 2021, several European countries were affected by severe floods. The floods started in the United Kingdom as flash floods causing some property damage. Later floods affected several river basins across Europe including Austria, Belgium, Croatia, Germany, Italy, Luxembourg, the Netherlands, Romania and Switzerland. At least 243 people died in the floods, including 196 in Germany, 39 in Belgium, two in Romania, one in Italy and one in Austria. It was thought that some of the affected regions may not have seen rainfall of this magnitude in the last 1,000 years.

A study using climate simulations on a grid of 2.2km squares published in June 2021 had come to the following conclusion:

Intense rainstorms are expected to be more frequent due to global warming, because warmer air can hold more moisture. Here, using very detailed climate simulations (with a 2.2 km grid), we show that the storms producing intense rain across Europe might move slower with climate change, increasing the duration of local exposure to these extremes. Our results suggest such slow-moving storms may be 14× more frequent across land by the end of the century. Currently, almost-stationary intense rainstorms are uncommon in Europe and happen rarely over parts of the Mediterranean Sea, but in future are expected to occur across the continent, including in the north. The main reason seems to be a reduced temperature difference between the poles and tropics, which weakens upper-level winds in the autumn, when these short-duration rainfall extremes most occur. This slower storm movement acts to increase rainfall amounts accumulated locally, enhancing the risk of flash floods across Europe beyond what was previously expected.

As Professor Fowler says:

Our infrastructure was built for a climate that no longer exists. Raised reservoirs, drainage, housing and transport were designed many years ago when extreme rainfall was rare and less severe. As rainfall intensifies, risks such as dam overtopping and cascading failures rise.

The government also produces monthly reports on rainfall, soil moisture deficit, river flows, groundwater levels and reservoir levels. This includes a wonderful map of the year’s rainfall:

What struck me was that the report looked at exactly the same things (river levels, groundwater levels and reservoirs) but, because they were concerned with low levels of rainfall, did not even mention flood risk. The rainfall scale only goes up to >125mm rainfall within a month as a maximum, which looks hilarious when compared with the 271.5mm of rainfall over 48 hours in parts of Belgium in 2021.

There have been no new reservoirs built in the UK since 1992, just another one of the gifts of water privatisation discussed previously. The government announced last year that it had “seized control of the planning process” to build two reservoirs due to be complete in 2036 in the Cambridgeshire Fens and 2040 near Sleaford in Lincolnshire respectively.

Professor Fowler says the probability of dam overtopping is increasing. So how’s the review of reservoir safety going?

In 2019, there was a failure of a dam at Toddbrook Reservoir in the Peak District. The operation to pump water out of the reservoir to safeguard Whaley Bridge seems to have been dominated by concerns about the fish. The independent review commissioned into the incident identified the rather alarming fact that:

“…the reservoir and its Owner can be compliant with the legislation without the reservoir necessarily being safe.”

Among the report’s recommendations were the following:

  • There is a systematic review of how the current Reservoirs Act, and the associated Regulations and Guidance, are implemented. This should consider the roles and responsibilities of qualified engineers, whether compliance with the Act is sufficient to ensure safety, and how safety is formally assured.
  • The potential of an Inspecting Engineer issuing a Certificate of Safe to Operate be explored. This should include a review of practice in other safety critical infrastructure sectors. It should also consider liability implications and whether some form of qualifying statement may be needed to accompany the certificate.
  • The statutory maximum period between inspections is reviewed to determine if it is still appropriate in every case in the light of the ageing reservoir stock.

In his follow up report looking at the reservoir system in general in 2021, Professor David Balmforth made this extraordinary statement:

This raises the question of what is meant by “safe”. In other infrastructure sectors and with reservoirs in other countries, safety is assured by managing risk, and by reducing that risk so far as is reasonably practicable – the terms “reasonable” and “practicable” being well understood in practice and in law. For a reservoir, risk is defined as a combination of the likely failure of the dam (or other reservoir structure) and the impact that an uncontrolled release of water would have on the area downstream, particularly the likely loss of life. I have therefore recommended that in future the assurance of reservoir safety should be managed on the basis of risk, and that the amount of effort (and cost) associated with that process should be in proportion to that risk. In this way the public can be assured that the hazard posed by reservoirs is being managed in an objective and transparent way.

It would appear that reservoir management is playing catch up on the whole concept of risk management.

What has followed is a policy paper from the Environment Agency on the launch of a reservoir safety reform programme. However the public consultation on the proposed reforms has already been delayed until later this year. And the page about the consultation process does not suggest anyone is in any hurry.

How Not To Be A Reverse-Centaur – Part 2

16. June 2026 · Write a comment · Categories: Actuarial, AI, Cartoons, Economics, Education, Professions, Risk, Science Fiction
A no entry sign over an image of a reverse-centaur where the robot is in control

As I discussed in How Not To Be A Reverse-Centaur (Cory Doctorow defines a reverse-centaur as “a machine that is assisted by a human being, who is expected to work at the machine’s pace”) our actuarial education system needs to change. This has been true for some time, but the development of AI systems has both demonstrated why more clearly and accelerated the timeframes over which action is needed. As I said in December last year:

A large part of the education of the future will need to be about equipping us all to understand what we now have access to and when and how to access it. We will all have different things we are interested in, or end up involved with and needing to be educated about. It will be up to each of us to decide which things are worth the difficulty of learning, but to make those decisions we will need education that can support the development of judgement.

For education institutions, the question will be what is not worth the difficulty of learning? Credentialising based on now relatively meaningless assessment methods will not cut it. This is where the confrontation with employers and politicians is likely to come. Essential skills and their related knowledge will be better developed and assessed via more open-ended project work and online assessment of it to check understanding. These will need to become the norm, with written examinations becoming less and less prevalent. Not because of fear of cheating and plagiarism, but because an outcome which can be replicated that easily by AI is not worth assessing in the first place.

Suppose you were to ask someone who had never seen our academic system that, in order to assess whether someone else we wanted them to employ in their organisation would do a good job for them, we would:

  1. Award marks for what has been written in answer to academic questions about what the students can remember unaided about the content of their lecture courses and reading lists with a biro on a pad of paper perched precariously on a tiny wooden table surrounded by hundreds of other similar scribblers, for a set period of time as minders wandered the floors like Victorian factory owners.
  2. These marks would be based on marking criteria they would never see.
  3. We would then get together in a secret huddle for a couple of months to mark all these scripts, check them and “moderate” them to check they were in line with the previous years’ exercises.
  4. Then out would come the answer (like the Answer of 42 to the Ultimate Question of Life, the Universe and Everything in Douglas Adams’ The Hitchhikers Guide To The Galaxy).

Source: https://hitchhikers.fandom.com/wiki/42

Imagine another world where the kinds of things employers routinely do when trying to decide whether to take on someone as a new member of their organisation are routinely part of their assessments at university, eg:

  • Watch them operating in groups working on a task.
  • Ask them to describe how they would go about tackling a particular problem.
  • Get them to show they understand the implications of a piece of work they have carried out and can explain what assumptions it is based on and can justify them.

With assessments of performance which are not opaque academic exercises but descriptions of performance made as the tasks are being carried out evidenced by video if necessary. In other words, transparent enough to whoever might want to work with these students next to eliminate the gap between their credentials and and their actual actionable skills, knowledge and experience.

Assessing in this way will be highly demanding, for both the students and the assessors, but it means the final assessment is done with the student present and, with careful probing from the assessors, who will obviously need to have done a close reading of the project work beforehand, confidence will be high about the abilities demonstrated by the process. And it removes the need for all the time currently spent on the Victorian factory owner process – imagine what could be done if the whole of May, June and July wasn’t spent marking scripts!

A move like this will significantly change what is taught. There will be no point presenting lots of information in ways which make retention easier for students (the so-called bookwork” questions currently in most exams). This makes sense when you think about how much of the content of your courses is accessible to you now, even on a vocational course like actuarial science, as opposed to skills developed. Instead the focus will be more on conversations with students for the purpose of developing their understanding of a meaty topic and the problems to be solved within it. Explorations of these will form the projects students will be spending most of their time working on. Checking understanding and making sure students are working within frameworks they understand and can talk about will be the main teaching activity, with opportunities to practise these throughout the year. In other words, showing the ability to apply what they have understood and demonstrate the ability to reflect, make judgements and show higher levels of understanding as a result. An ability they can then apply to new problems and situations.

And notice something else about how these alternative credentials are generated? They are all social activities: from operating in groups, to presenting a piece of work to a group of assessors. Even the project work is developed through group sessions alongside individual work. If you want to develop people who can spot the gaps between models and reality you need to do it in groups, where people can test their perspectives against each other.

Compare this to the current solitary exam preparation, peppered with “revision” lectures, exam sitting and exam marking processes which comprise the predominant activities in universities between March and July each year. We are currently being funneled into an ever more solitary professional practice around LLMs. It has become quite accepted for people to do all of their thinking on a subject closeted with perhaps one, two or three LLMs but no other people and then share their analysis with an expectation that other people should read the output. Developing the skills to do these analyses will obviously be necessary, but they will never be the most important skills people learn.

Treating each other like walking databases is never going to make you new friends or influence people. However the skills you develop via an alternative more social education will.

There are examples of this approach already happening, with employers involved in the design of assessments in some cases and universities using the flexibility they have to restructure and redesign courses. But the dominant assessment system of formal exams is keeping much of this activity at the margins at the moment.

If it were to become the dominant assessment, it would also mean that larger areas of the curriculum could be examined, as the focus would not be on retention of large bodies of content but the ability to use the content to solve problems. So, perhaps, after a first year levelling up students’ ability and experience of mathematics and statistics, year 2 would have an actuarial statistics module (currently CS1/CS2) and an actuarial mathematics module (currently CM1/CM2), both dissertation based and vivaed. Then in the third year they would tackle an economics module and a business modules (currently CB1 and CB3). Perhaps in the final year of a four year MAct they would tackle a modelling and communication module (CP2/CP3) and either CP1 actuarial practice module would be offered or a professional skills module.

This would be more of a driving test style of assessment, with students working with supervisors on the comments from assessors on the original dissertations and presentations until they were up to the required standard. Students unable to make the standard after an additional year would be offered a move to a non-accredited course. The current funding model for higher education will need to change: loading students with £10,000 more debt every time they add to the nation’s skills bank with another year wrestling with these difficult skills to master was never a clever strategy, but this structure would make it even clearer how self defeating it is. If economists are accepting that the labour market no longer allocates resources effectively and are considering either a universal basic income or universal basic services, then logically university funding should follow suit.

The implications for the actuarial profession will be even more challenging. I question whether the Institute and Faculty of Actuaries will continue to consider it worth the effort and expense of reconfiguring their entire approach to exam setting, syllabus maintenance, marking, online proctoring, etc for subjects routinely taught throughout the university system for a a variety of purposes, ie the current core principles subjects of mathematics, statistics and business studies.

Whether they retain their own capacity to assess the core practice subjects is a more open question (CP1 Actuarial Practice, I imagine, will be seen as sufficiently specialist to keep in house, with perhaps a few universities, as now, accredited to run courses which can earn this qualification, however CP2 Modelling and CP3 Communications are not nearly as specialised now as they seemed when they were first introduced). I would also move the new core economics module recommended by the IFoA Economics Review Group I chaired in 2022 to replace the current CB2 module into the core practice section, as it would be more of a critical thinking module about the economic ideas necessary to underpin good actuarial work than the technical how-to subjects in core principles.

The specialist stage of the SP and SA subjects and the whole continuing education and additional certification options I would confidently expect the profession to retain, review and adapt as needed to reflect new challenges in members’ working lives. However, if CP1 is all that is needed post graduation to get to Chartered Actuary and CP1, two SP subjects and a SA subject gets you to Fellow, then suddenly time to qualification becomes much more predictable for students and their employers alike.

The risks to employers of taking on students would therefore be reduced, but also the rewards to taking them on will be much more transparent. The students we will be developing will have collapsed the gap between their credentials and their actual actionable skills, knowledge and experience. They will have:

  • Great team working skills;
  • Very highly developed presentation skills, both in writing and speech;
  • Strong IT skills and comfort working with data; and
  • Clarity about why they are in an organisation and a drive to use their skills to solve problems.

They will also have developed the four capabilities set out in Carlo Iacono’s Teach Judgement, Not Prompts:

  1. Epistemic rigour. They will be more likely to spot when a system or model is over-confident given the evidence, after an intense experience of interrogating models and evaluating evidence in their courses.
  2. Synthesis. They will be able to integrate different perspectives into an overall understanding, through dissertation development, defending a position, understanding weaknesses in a position and adjusting accordingly.
  3. Judgement. They will have been asked to make many judgements in their work and had to defend them in discussions with their supervisors and fellow students. Unconvincing opinions, not backed by evidence, will not pass muster.
  4. Cognitive sovereignty. Students will have to take their own stand on their work, after all of the challenge and argument. Independence of thought is the hard-earned outcome here.

And then they will have a fighting chance of being centaurs in the world awaiting them, masters of the technology and opportunities available, rather than the reverse-centaurs that our education system is currently, and inadvertently, preparing them to be.

The meatspace layer for actuaries

20. May 2026 · Write a comment · Categories: Actuarial, AI, Cartoons, Economics, Education, Media & Politics, Pensions, Professions, Regulation, Risk
A suited pinhead wearing a pirate's hat stands in the stern of a pirate ship below a dangling ladder
Source: Nick Foster – December 2013 – originally drawn to deride George Osborne’s austerity as Pugwash Economics, repurposed now as I am worried about the actuarial profession pulling up the ladder on the next generation

The “black box” was a constant refrain when I was working as an actuarial consultant. It was where the results from a process were being accepted without any understanding of how they were arrived at. Something we felt that any self-respecting actuarial consultant should challenge in their own work and everybody else’s.

However when you came to actually present analysis or arguments to a client, you expected a certain amount of that expertise to be taken as read, to effectively be inside a black box as far as the client was concerned. They couldn’t be expected to understand all of the aspects of what you were talking about, otherwise they wouldn’t need you. Good practice was always to put them in a position where they could understand and make decisions about the key aspects of your advice without needing to engage with the other parts. As the expert, you decided what was in the black box.

Now the black box is back with a vengeance for all the professionals who have relied upon them in their working lives. As Dan Davies puts it:

The same black-box property which stops you from being second guessed or overruled means that nobody is interested in your explanations for your decisions; it is definitional of being a black box that you are going to be judged by results.

And, if you are in the business of advising in the teeth of uncertainty, as actuaries are, then this is likely to be a real problem. If noone cares how your advice was constructed, but they can get advice that ticks the compliance box your client has to complete more cheaply and quickly than you can, the more automated black box is going to win the business. The experienced professional still has a role in managing this process, verifying the results coming out of the black box and determining what can still be kept out of the black box, but he may be increasingly struggling to justify the cost of his junior colleague.

I wrote about how devastating the fall in graduate job listings was 9 months ago, so where have we got to since?

Well things don’t look so bad in the UK right now according to the Office for National Statistics (ONS), reverting to close to the average after a post pandemic surge in the finance and insurance sector:

Source: https://www.ons.gov.uk/employmentandlabourmarket/peoplenotinwork/unemployment/datasets/vacanciesbyindustryvacs02

However, if we look at the United States, which tends to show us where the UK finance sector is going, it looks far more ominous:

Yesterday Sky News ran a story about Standard Chartered‘s CEO who, in his desperation not to describe over 7,500 job losses as cost cutting, said this:

It’s not cost-cutting. It’s replacing in some cases lower-value human capital ​with the financial capital and the investment capital we’re putting in.

We may need to sit with that statement for a little while.

Daniel Susskind talks about this risk in his latest lecture entitled A World Without Work: in summary, to paraphrase only slightly, sure relatively junior white collar roles may already be particularly hard hit by AI, but he is optimistic because of the impact on GDP and we cannot pause because of China. He then goes on to talk about the four problems he sees for a post AI future:

  1. Distribution (replacing wages);
  2. Contribution (how do you “pull your weight”);
  3. Power (domination by Big Tech on economics, politics, liberty, social justice and democracy); and
  4. Meaning (fulfilment in life).

Susskind has gone from thinking that the fear that AI is coming for your jobs was overblown and that it was just task encroachment that we faced, to now thinking that it may encroach on all the tasks in most fields. Jevons Paradox (that technological innovation that increases the efficiency of a resource’s use leads to a rise in consumption of that resource) is no comfort if that new demand is robot-met.

Carlo Iacono suggests that the move of junior roles to AI may be subtle to begin with:

The weakness among young workers may appear as fewer people entering employment from outside the workforce. Firms may not fire large numbers of juniors; they may simply hire fewer of them.

That matters. The labour market can look healthy while the entry path narrows. Senior workers stay employed. Output rises. Productivity improves. There is no dramatic wave of redundancies.

Yet the first rung is being taken out.

It may also be masked by the fact that there remains a shortage for actuaries beyond the entry roles. There is almost a hint of desperation to approaches like this looking for introductions from a retired actuary like myself:

(followed by a list of clients he is working for)

Source: recruitment consultant who will remain anonymous. I am assuming “candies” are candidates

And, even if you think the risk of the AI Bubble bursting soon, taking down the global stock markets underpinned by the Magnificent 7, is exaggerated, you do need to be suspicious about the current abilities of AI to replace junior staff. My experience with another, somewhat earlier, actuarial technology, the pensions valuation engine, would suggest that the outputs need to be analysed very carefully before sharing with a client: it often had dependencies between what should have been independent variables hidden in the programming, or vagaries in the setup which left out non-standard benefit rules for your particular scheme, for instance. Or the student who had set it up initially (a complicated process usually) might have made a mistake or you might have not communicated with them very well to start with. Or a hundred other things.

For whatever reason, there was often still a lot to do after the valuation engine had produced some output.

Can this sort of thing happen with agentic AI? Well think about that student programming the valuation engine, but on steroids. Its patchy capabilities combined with its basic psychopathy leads to, as Hannah Fry entertainingly demonstrates here, some serious problems arising with the agent’s relentless to and fro with the large language models it depends upon, asking them what it should do next. As Hannah says:

I built an AI agent. She opened a shop selling novelty mugs, emailed a journalist without being asked, and then leaked our passwords to a total stranger.

As Kyle Kingsbury wrote about having an AI agent as a colleague in a programming team:

Imagine a co-worker who generated reams of code with security hazards, forcing you to review every line with a fine-toothed comb. One who enthusiastically agreed with your suggestions, then did the exact opposite. A colleague who sabotaged your work, deleted your home directory, and then issued a detailed, polite apology for it. One who promised over and over again that they had delivered key objectives when they had, in fact, done nothing useful. An intern who cheerfully agreed to run the tests before committing, then kept committing failing garbage anyway. A senior engineer who quietly deleted the test suite, then happily reported that all tests passed.

You would fire these people, right?

Yet despite all this, the money continues to pour in to the construction of AI infrastructure. There are already websites up and running for all of the parts of tasks AI cannot encroach upon.

Source: https://rentahuman.ai/

The bottom rung of the actuarial ladder is clearly in danger. This is a particular problem for the actuarial profession, which has traditionally relied on longer periods of work-based training for its future qualified actuaries than many other professions. Training to become an actuary takes a long time. Median time to fellowship is still around six years, with some taking up to ten or giving up. The exams are hard to pass. There have been attempts by the profession to tackle some of these disincentives: the Chartered Actuary designation to make a destination of the generalist qualification before the specialisation of the fellowship, championed on this blog and launched in the teeth of opposition by some fellows, being one example.

It has led to a culture within actuarial firms around managing the extended time in training, with rituals around study leave and results days. One of the fears expressed in opposition to the introduction of the Chartered Actuary designation was that, if this could be achieved almost entirely within formal education at universities, the value of working alongside experienced actuaries would be lost.

It has led to a culture within the profession itself of managing large parts of its education system in house. Half of its revenue and around 30% of its expenditure are on “pre-qualification learning and development”. Sometimes it looks more like an education business with a professional side hustle.

But then the new AI toys have come along, and it turns out that many of those experienced actuaries may be less keen on graduates coming in and needing supervision from them after all. Many of them may rather spend hours on AI prompts than on developing another human being.

I fear that, increasingly, companies are not going to accommodate actuarial students in their work plans without significant persuasion. And, if the number of students studying while in work falls, the profession itself is going to struggle to finance its own bespoke education system at an acceptable cost to its members.

It will be hard for the profession to challenge this too: it is going to be good for many of those already established in their roles as the market for more experienced actuaries, when the market has no interest in developing the actuaries of the future, becomes increasingly competitive.

If the actuarial profession does accept the challenge of protecting the pipeline of future experienced actuaries it will need to review its entire education syllabus through this lens. It will also need to engage with other partners involved in what is in effect a problem of capital formation and collective action: government incentives may be needed to encourage firms to continue to train early career professionals and discourage free-riding. There may be no way back for the student with no actuarial qualifications learning on the job. The universities may be needed to plug people in at a different career point, which will require them to innovate themselves even further into the professional training role than ever before. As Carlo Iacono points out:

educational institutions may be pushed to simulate more of the apprenticeship environment. That does not mean adding a thin “AI literacy” module. It means creating settings where students practise judgement under uncertainty, in realistic workflows, with feedback that is close enough to hurt and useful enough to teach.

It will not be at all easy. But the alternative is a future without opportunity for those who do not already have it and an ageing profession withering on the vine it refused to nurture.

The monsters we create

24. April 2026 · Write a comment · Categories: AI, Education, Media & Politics, Prediction, Risk, Science Fiction
The Wetherspoons pub The Mary Shelley in Bournemouth

A few months ago I decided to read Mary Shelley’s Frankenstein for the first time. I also watched Guillermo del Toro’s Frankenstein, on a big screen, despite, according to The New Yorker, it having been “Netflixed down to size”.

Shelley’s book is largely monologues of interior thoughts of Frankenstein and his creation, with wildly careering emotions and death, death, death everywhere – perhaps unsurprising from an author whose mother died 10 days after giving birth to her, who lost a child and whose half sister died by suicide while she was working on Frankenstein, with much more tragedy to follow after its publication. There is a word Mary Shelley uses more than I have read in any other book: variants of sympathy/sympathise/sympathies turn up 32 times. Because of course one of the many things the book is all about is mutual incomprehension of the creator and the created.

Last week I was in Bournemouth as a last minute substitute for Lanzarote, something I may come back to at a later date, and I stumbled across the churchyard of St Peter’s Church in which Mary Shelley was buried, along with the cremated heart of her husband Percy Shelley, at the age of 53. There is also a pub in Bournemouth named after her (above) but whose sign depicts the monster from her most famous piece of writing.

As we enter another time of mutual incomprehension of the creator and the created, I have been reading the surprisingly-difficult-to-access paper by Kyle Kingsbury (the systems engineer, not the MMA guy) called The Future of Everything is Lies, I Guess. I will put a link to an X account which shared it here, as going to the aphyr.com site to read it seems to generate this message:

Once you can read it though, it starts to sketch out a likeness of our current monster and chip away a little at the human side of the mutual incomprehension. I am talking, of course, about what people are currently calling “AI”, which Kingsbury defines as:

…a family of sophisticated Machine Learning (ML) technologies capable of recognizing, transforming, and generating large vectors of tokens: strings of text, images, audio, video, etc. A model is a giant pile of linear algebra which acts on these vectors. Large Language Models, or LLMs, operate on natural language: they work by predicting statistically likely completions of an input string, much like a phone auto-complete. Other models are devoted to processing audio, video, or still images, or link multiple kinds of models together.

The article sets out how this is a technology where nobody really understands why it has been successful or how to make it better, which falls into strange loops or attractors, has odd gaps in its capabilities and is highly sensitive to slight changes in its formatting. It is a technology which is simultaneously highly capable and an idiot. And Kingsbury worries that our culture is not ready for such a technology. As he says:

As LLMs etc are deployed in new situations, and at new scale, there will be all kinds of changes in work, politics, art, sex, communication and economics. Some of these effects will be good. Many will be bad. In general, ML promises to be profoundly weird.

Buckle up.

He continues:

Most people seem concerned with conscious, motivated threats: AIs could realize they are better off without people and kill us. I am concerned that ML systems could ruin our lives without realizing anything at all.

There follow extensive examples of the problems the various ML applications are already starting to cause and some speculation about where things may be going in various areas of our lives before we get to the chapter on work. And the subject of hiring “AI employees”. This is probably my favourite bit:

Imagine a co-worker who generated reams of code with security hazards, forcing you to review every line with a fine-toothed comb. One who enthusiastically agreed with your suggestions, then did the exact opposite. A colleague who sabotaged your work, deleted your home directory, and then issued a detailed, polite apology for it. One who promised over and over again that they had delivered key objectives when they had, in fact, done nothing useful. An intern who cheerfully agreed to run the tests before committing, then kept committing failing garbage anyway. A senior engineer who quietly deleted the test suite, then happily reported that all tests passed.

You would fire these people, right?

Kingsbury sees the two extremes of the possible range of outcomes as:

  1. ML systems continue to hallucinate, cannot be made reliable, and ultimately fail to deliver on the promise of transformative, broadly-useful “intelligence”. Or they work, but people get fed up and declare “AI Bad”…a lot of ML people lose their jobs, defaults cascade through the financial system, but the labor market eventually adapts and we muddle through. ML turns out to be a normal technology.
  2. In the other extreme, OpenAI delivers on Sam Altman’s 2025 claims of PhD-level intelligence, and the companies writing all their code with Claude achieve phenomenal success with a fraction of the software engineers. ML massively amplified the capabilities of doctors, musicians , civil engineers, fashion designers, managers, accountants, etc, who briefly enjoy nice paychecks before discovering that demand for their service is not as elastic as once thought, especially once their clients lose their jobs or turn to ML to cut costs. Knowledge workers are laid off en masse and MBAs start taking jobs at McDonalds or driving for Lyft, at least until Waymo puts an end to human drivers. This is inconvenient for everyone: the MBAs, the people who used to work at McDonalds and are now competing with MBAs, and of course bankers, who were rather counting on the MBAs to keep paying their mortgages. The drop in consumer spending cascades through industries. A lot of people lose their savings, or even their homes. Hopefully the trades squeak through. Maybe the Jevons paradox kicks in eventually and we find new occupations.

In the following chapter Kingsbury speculates on what some of those new occupations might be:

  • Incanters. People who can prompt LLMs into getting what is wanted.
  • Process Engineers. People who help catch LLM errors. They build quality control processes – training people, identifying where more intense review is needed, assessing the cost-benefit trade offs of automating tasks, etc
  • Statistical Engineers. People who try and measure, model and control variability in ML systems.
  • Model Trainers. This will become increasingly difficult as the amount of false content or “slop” increases across the internet.
  • Meat Shields. People who are accountable for the errors of the ML systems they supervise.
  • Haruspices. People responsible for going through the model inputs, outputs and internal states of a ML system which has done something terrible to try and give a plausible reason for its behaviour.

But ultimately Kingsbury concludes that we should just stop using these systems. To return to the original analogy, the monster cannot be understood. There is often nothing actually there to understand. And it is certainly not in the business of understanding you. Although it may be very very good at convincing you otherwise.

On balance I think my view is currently at the muddle-through-with-ML-as-a-normal-technology end, which still looks likely to cause a disruption considerably bigger than 2008. My main reason is the already collapsing trust in many of the Big Tech companies. Trust which is going to be required even if their technology really can do some of this stuff. It is the scenario where we all get fed up and declare “AI Bad”. Like when we read about the people running Meta showing nowhere near the social responsibility commensurate with their current level of market power.

Or when, as last week, we have days and days of breathless commentary about Anthropic’s Mythos and Project Glasswing, and how its immense capabilities caused the company not to release it, sparking a meeting of central bankers to discuss the threat such technologies posed to financial systems. Only to finally read an account of attempts to verify any of what Anthropic have been saying. It is quite a technical piece, which I by no means understand all of, but the final paragraph is fairly arresting:

The most important thing in the Mythos release is not the model. It is the precedent. Anthropic has established, without discussion and without pushback, that a private company can unilaterally classify a capability as too dangerous for the public, grant selective access to the largest incumbents in the affected industry, and construct a parallel disclosure regime outside any democratic accountability structure. That precedent is exclusivity for abuse. It will be used by companies with worse judgment than Anthropic and narrower definitions of “partner” than the Glasswing consortium. The time to object to the shape of this thing is while it is still being built, not after it has removed all transparency and accountability.

How might Claude or ChatGPT respond to being designated “AI Bad”? Well Mary Shelley’s monster put it this way:

Once I falsely hoped to meet with beings who, pardoning my outward form, would love me for the excellent qualities which I was capable of unfolding. I was nourished with high thoughts of honour and devotion. But now crime has degraded me beneath the meanest animal. No guilt, no mischief, no malignity, no misery, can be found comparable to mine. When I run over the frightful catalogue of my sins, I cannot believe that I am the same creature whose thoughts were once filled with sublime and transcendent visions of the beauty and the majesty of goodness. But it is even so; the fallen angel becomes a malignant devil. Yet even that enemy of God and man had friends and associates in his desolation; I am alone.

Talking Terraforming in Birmingham

08. April 2026 · Write a comment · Categories: Actuarial, Birmingham, Climate Change, Media & Politics, Risk, Science Fiction
Front page of the April 2026 issue of Brum Group News

Three and a half years ago, I wrote a piece likening the rapid climate change on Earth to the fairly well-established science fiction concept of terraforming, but in reverse. So what has happened since? Well last summer, according to researchers at Imperial College and the London School of Hygiene and Tropical Medicine, two thirds of the 24,400 heat deaths from June to August across Europe were due to human-made global heating. And a study published last month has suggested that the pace of global warming has nearly doubled since 2015.

It this point I would like to suggest rehabilitating an old word to describe this process, in the opposite direction to terraforming (which is action designed to make a planet more habitable). Barrenize means to make barren or sterile and was used between the mid 1600s and the early 1700s according to the Oxford English Dictionary, originally in the context of animal husbandry. I think it’s time to bring this word back.

In a week when a US President has threatened, variously, “blowing everything up and taking over the oil” and that Iranians would be “living in Hell”, to last night saying that “a whole civilisation will die tonight”, unless they opened the Strait of Hormuz, it certainly sounds like a commitment to barrenization to me, only at a faster pace than the global warming he is already doing everything possible to accelerate further.

On Friday this week, the Birmingham Science Fiction Group will have Oliver Bettis as its guest speaker. Oliver has been a leading actuary in the field of sustainability for many years. He is one of the authors of a series of publications by the actuarial profession in collaboration with the University of Exeter in recent years.

Climate Scorpion shows how we need to develop a best guess about the worst-case scenarios and make policy on that basis, given our lack of knowledge about extreme climate risk and tipping points.

Planetary Solvency – finding our balance with nature sets out an approach to civilisational risk management which attempts to address the fact that the severity and frequency of extreme events are unprecedented and beyond current model projections.

Parasol Lost, which we will be discussing in particular this Friday, focuses on the cooling effect of aerosols: a side-effect of pollution from fossil fuel burning. Without aerosol cooling the global temperature would be around 0.5°C higher than the 1.4°C increase above pre-industrial temperature that we have today. It is critically important to recognise that, as air pollution is cleaned up, this may ironically lead to a short-term increase in warming through the loss of aerosol cooling. The question must be asked, can we afford to lose this cooling and if not, should this be replaced by working with nature, using technology or both?

This will allow us to tap into the rich history of science fiction literature on terraforming (and dealing with the threat of barrenization) and whether this can allow us to look at this question in a new way. It should be a lively discussion.

This event will be held in-person at the Friends of the Earth Warehouse, 54-57 Allison Street,
Birmingham B5 5TH and simultaneously on Zoom, with online access opening from around 7.45 for an 8 pm start.

Ticket prices for non-members are £8 for in-person attendance and £6 for Zoom attendance. For members it’s £4 in-person attendance and free Zoom attendance.

Tickets can be purchased on the door or via the Eventbrite link below:

https://www.eventbrite.co.uk/e/1985958692911

And if this whets your appetite for more science fiction and you think you might like to join the group, just email us at contact@brumsfgroup.org.uk. Hope to see you there!

It feels like deja vu all over again

30. March 2026 · Write a comment · Categories: COVID-19, Economics, Media & Politics, Risk

OK I don’t know if this is a remotely helpful post, but it really feels to me like one of those months we will look back on, like March 2020, and wonder what we were thinking. To recap: on 4 March 2020, while Italy were shutting all their schools and a month after the WHO had declared a global health emergency, we were noting that the number of cases in the UK had jumped from 53 to 87 in one day.

Jump forward to now and the number of tankers with oil on board is in freefall:

Trump is talking about invading Kharg Island and “obliterating” Iran’s energy facilities, and we are sitting in the time lags of international fossil fuel freight waiting to see what will happen. But we already know what is going to happen. Just like the pandemic, we will be taking similar measures to the countries already more affected very soon. The order looks like Asia, followed by Africa, then Europe and only then, ironically, the United States.

So what is going on in Asia right now? Well the Philippines announced a national energy emergency six days ago, setting up an authority to oversee the orderly distribution of fuel, food, medicines, and other essential goods. Sri Lanka has announced a four-day week for all government employees. Egypt is ordering restaurants, cafés and shops to close at 9pm to safeguard dwindling energy reserves. Slovenia has brought in fuel rationing. Moldova’s Parliament has also voted to impose a state of emergency in the country’s energy sector. Australia is offering free public transport. Measures are also being taken in Thailand, Ethiopia, Myanmar, Vietnam, Bangladesh and South Sudan.

On 3 March 2020, the UK Government unveiled their Coronavirus Action Plan, which outlined what the UK had done and what it planned to do next. Paul Cosford, a medical director at Public Health England, said widespread transmission of COVID-19 in the United Kingdom was “highly likely”.

On 4 March 2020, the Daily Express were telling us:

Which we clearly weren’t. Meanwhile the Daily Mail was anticipating future lockdowns and 6 million people being off sick:

The next day we had the first Covid death in the UK. And life was on hold for the next two years.

Our response to the energy crisis seems to be almost entirely focused on

1. The cost-of-living crisis; and

2. The financial markets.

The Education Secretary has said that motorists should fill up as normal as the government is “well prepared” for disruption. The trouble is, many of us still remember September 2000:

So that would be enough to make us all feel nervous about shortages and queues for everything, having our lives disrupted and out of our control. But the real potential issue is not even being talked about, certainly not by the government. It is a shortage of food. Steve Keen sets out the economics of global food production here. This does not tend to feature prominently in mainstream economic analyses which are energy and food blind for the most part, although the FT did have this graph a couple of weeks ago:

As Steve Keen says:

Survival will depend on grain reserves. China has of the order of 18 months in reserve, which will insulate it from the disruptions of 2026. The USA and India have substantial reserves as well, but some countries—including the UK—have virtually none.

…Famines will ensue, and even countries that have never experienced such events could be forced into food rationing. This includes the UK and Australia, and a patchwork of countries across Europe.

This is what people are nervous about: not being able to get enough food, either because it isn’t available at all or not at a price they can afford. Calling that a cost-of-living crisis is a bit like calling the Black Death a labour market crisis. And it doesn’t stop there. As Steve Keen continues:

Other critical products that normally pass through the Strait of Hormuz include Helium, which is critical to the production of semiconductors, and sulphuric acid, which is critical to numerous production processes. The closure of the Strait cuts off one third of global helium output and about half of global sulphuric acid output.

With critical industrial inputs cut as well, the problems will cascade well past food alone—though that is clearly the most damaging impact. With LNG, petroleum, helium and sulphuric acid production cut, the capacity to undertake repairs to damaged facilities will also be hindered.

The TED War is rather like smashing a spider’s web—and then killing the spider.

The spider certainly looks in a poor state of health at the moment, and parts of the web will take years to fix. This is the crisis we are all inevitably going to be entering in the next few weeks. For who knows how long.

A risk management approach to this crisis would involve communicating a plan to the country that minimised the impulse to hoard resources and protected the most vulnerable from extreme prices, rather than bland reassurances from government ministers. We need this to be in place very quickly now.

Humans-in-the-loop and the Pragmatic Revolution

03. March 2026 · Write a comment · Categories: Actuarial, AI, Economics, Media & Politics, Professions, Risk, Science Fiction
Source: https://markets.ft.com/data/equities/tearsheet/summary?s=IBM:NYQ

A week or so ago I referred to a “Thought Exercise” set in June 2028 “detailing the progression and fallout of the Global Intelligence Crisis” (ie science fiction), published on 23 February, which may have tanked the share price of IBM later that day. As I said then, the fall definitely happened, with IBM’s share price falling 13%, its biggest fall since 2000. I said then that the likelihood of the scenario portrayed was difficult to assess, but the speed with which the total economic collapse was described felt unlikely if not impossible. I would like to expand on that.

The main reason that the scenario was hard to assess was that it was not based on data or evidence at all. That is unavoidable for speculative fiction talking about things that are not currently happening, but when describing an economy only two years away where most of the processes described should be discernible to some extent already, it is totally avoidable.

Ed Zitron has done an excellent line by line take down of the Citrini piece here. Here is one page of that to give you a flavour:

However this lack of a link with anything tangible did not stop the financial markets panicking, which should cause us pause when relying on the financial markets’ valuation of projects, industries, government policies, etc.

Ed Zitron describes this kind of piece as analyslop: “when somebody writes a long, specious piece of writing with few facts or actual statements with the intention of it being read as thorough analysis”. It can then get picked up by other commentators which take it as their starting point for further analysis, often making it hard to see that the starting point had few if any data points. Here is an example, from Carlo Iacono, looking at what if just some of the Citrini pronouncements were true, with appendices detailing possible branching paths of outcomes, all generated by a large language model (LLM). And then people start studying the meta analysis, and it starts getting taken even more seriously, and put into models and pretty soon most of the analysis is being done on imagined risks rather than on ones which are already staring us in the face.

We have always had a problem keeping our society grounded in reality, think the 2003 Iraq War, where we went to war on a false assessment about Iraq’s possession of weapons of mass destruction, the 2008 financial crisis, where banks misunderstood the risks they were exposed to, and the last two and a half years, where we, for the most part, seem to have convinced ourselves we have not been facilitating a genocide in Gaza when we clearly have been. But this is only going to get worse with the AI systems which are being developed.

As Nate Hagens points out:

The rapid rise of artificial intelligence has served to dramatically increase the speed of information production while also eroding accuracy, making it difficult to differentiate between content that simply sounds confident and content that’s actually grounded in reality.

So where is AI currently? Well PwC’s global CEO survey from January this year had the following statement as the first bullet amongst its key findings:

Most CEOs say their companies aren’t yet seeing a financial return from investments in AI. Although close to a third (30%) report increased revenue from AI in the last 12 months and a quarter (26%) are seeing lower costs, more than half (56%) say they’ve realised neither revenue nor cost benefits.

That’s the reality. But the hype is much much more entertaining. My favourite spoof video of the AI future currently is this one, about the time where all most of us are good for is riding bicycles to supply the ever increasing energy needs of AI systems (click view in browser if you can’t see it):

And what about the financial journalists? The pieces describing our reaction to whatever is about to unfold economically have already been written. There are investor websites asking if the 2026 crash has already begun, while another recent article argues that “America has quietly become one of the world’s most shock‑resistant economies” (which seems unlikely to age well). What most financial journalists are more comfortable with are articles about how the warnings were ignored after the fact.

And the professions? Well the current overview of my own profession is probably reasonably represented by this piece from the Society of Actuaries in the United States. Unfortunately for them, Daniel Susskind, who is mentioned in the article, is currently suggesting, as part of his Future of Work lecture series for Gresham College, how the key to the sudden development in AI, after the “AI Winter” when progress seemed slow, was that we abandoned trying to make machines which thought and acted like humans in favour of focusing on completing tasks in any way possible. Increasingly we are now automating tasks where we can’t (or won’t) articulate how we do them. From Deep Blue‘s victory over Kasparov in 1997 to Watson winning jeopardy in 2011 to ImageNet beating humans at image recognition (although that is disputed), Susskind refers to this progress as the displacement of purists in favour of what he calls “The Pragmatic Revolution”. Pragmatism in this sense appears to be that we humans should just accept the consequences the people running these systems want. So, as his latest lecture “Work, out of reach” claims, people moving into cities to find work is a strategy which is no longer going to work for low skilled people:

He then shows this graphic demonstrating the lack of recovery of big coal mining areas in the UK:

Source: Left – Sheffield Hallam University map of coal mining areas; Right – % employment from Overman and Xu (2022)

And finally he cites the notorious Policy Exchange piece from 2007, Cities Unlimited, whose thesis was that there is apparently no realistic prospect of regenerating towns and cities outside London and the South East.

Susskind talks about three forms of technological unemployment:

  1. skills-mismatch, where your skills are mismatched to the work available. Education and training has always been the answer to this in the past.
  2. place-mismatch, where the jobs are not where you have built your life. Some believe the answer should always be the one proposed by Norman Tebbit, who memorably told everyone in 1981, “I grew up in the 30s with an unemployed father. He did not riot. He got on his bike and looked for work.”
  3. identity-mismatch, where according to Susskind, people are prepared to stay out of work to protect their identity, citing US men who won’t take “pink collar” work, China “rotten tail” kids, Japanese seishain-or-nothing and Indian Sarkari Naukri queues in India. Or perhaps they are just looking for work which is consistent with the idea of human dignity.

Susskind claims to have no answer to any of these as far as AI is concerned. They are, in his view, just the inevitable outcomes of his “Pragmatic Revolution”. It is the unthinking pursuit of more and more growth funded by capital less and less tethered to any territory, principle or purpose, where any grit in the machinery, be it unions or protestors or, increasingly, the wrong sort of government must be trampled underfoot. Anything which impedes the helter-skelter rush to more and more at greater and greater speed. It’s like our whole economy is run by this guy (press the view in browser link if you can’t see him) shouting “Ready, Aim, Fire!”:

But unskilled people will not be the only collateral damage of these unguided weapons. Take markets for instance. These are where people are exposed to risks and rewards based on underlying conditions they only partially understand. Greed and fear may be their main motivations, but gossip and group think are their main communication channels. They don’t need facts, particularly when so many of the facts are proprietary information not in the public domain. A plausible narrative will do. And plausible narratives are what LLMs will do for you in abundance.

And the more we reward people who can move fast, eg to spot an arbitrage opportunity, even at the risk of breaking things, rather than people who can make decisions which still look good decades from now, the more we are setting up the conditions for AI systems to be the go-to tool.

And put that together with an AI industry which desperately needs funding capital to keep arriving, ie one which is unbelievably highly motivated to push plausible narratives even when they know they are not grounded in reality, and you have a recipe for market-generated chaos.

And then we have Trump’s new war. Beware the people who are war gaming the Middle East at the moment on a range of LLMs (just stop and think for a moment about the bloodless inhuman impulse behind carrying out such an exercise rather than, I don’t know, talking to some actual people who live or have lived recently in and around the region). One of the worst offenders is Heavy Lifting banging on about what the three scenarios are for Operation Epic Fury. This is as bad as it sounds:

I tasked her [he is talking about Gemini Pro here] with doing a literature review on regime change (a term often used by the President but not a well-defined one), creating three scenarios of possible outcomes for which each was give a percentage probability, and a list of 20 items to examine for each scenario that covered political, economic, and cultural issues with a special focus on the political consequences in the U.S. and what this means for China, our biggest geopolitical rival.

But Gemini Pro wasn’t the only one involved in this. Two other humans were, Tim Parker and Ron Portante, trainers at the gym I go to. (Just as a personal aside, Tim was my coach in hitting six plates [345 pounds] on the sled last Friday and I have a video to prove it!) I was talking about the piece and Ron raised the issue of linguistic and cultural diversity in Iran. Tim did some real time research for me on his phone while I was burning real calories under his strict tutelage. This made me think I needed a background section on Iran. When I got him Gemini and I added it.

What you mean you belatedly realised you might need to have done some actual research into Iran rather than just generic research on regime change? I stopped reading at that point.

Meanwhile King’s College London have been carrying out war games more systematically using AI. Professor Kenneth Payne from the Department of Defence Studies led the study, which looked at how LLMs would perform in simulated nuclear crises. As Professor Payne said:

Nuclear escalation was near-universal: 95% of games saw tactical nuclear use and 76% reached strategic nuclear threats. Claude and Gemini especially treated nuclear weapons as legitimate strategic options, not moral thresholds, typically discussing nuclear use in purely instrumental terms. GPT-5.2 was a partial exception, limiting strikes to military targets, avoiding population centers, or framing escalation as “controlled” and “one-time.” This suggests some internalised norm against unrestricted nuclear war, even if not the visceral taboo that has held among human decision-makers since 1945.

This is not a Pragmatic Revolution. These AI systems cannot replace humans thinking about the future we want for humans in any way which is worth having. What they can do, if we let them, is accelerate our worst impulses and move us further away from considered reflective decision making.

But we will continue to use AI systems in the military because, as it turns out, it is very useful for low stakes admin. So although Lavender, the system used by the Israeli military to select targets in Gaza, made errors in 10% of cases and was therefore totally inappropriate to the task, there are lots of organisational logistical tasks where it is much quicker than the alternative and 10% error rates do not matter so much.

There is clearly an issue with what we decide to use these systems for. We need to be able to regulate the decisions which are particularly consequential. However the only way we seem to be considering for this at the moment is the human-in-the-loop model, like the humans spending around 20 seconds considering each target recommended by Lavender before authorizing a bombing. I have written about these before in the context of early career professionals in the finance industry, where the prospect seemed miserable enough:

They will be paid a lot more. However, as Cory Doctorow describes here, the misery of being the human in the loop for an AI system designed to produce output where errors are hard to spot and therefore to stop (Doctorow calls them, “reverse centaurs”, ie humans have become the horse part) includes being the ready made scapegoat (or “moral crumple zone” or “accountability sink“) for when they are inevitably used to overreach what they are programmed for and produce something terrible.

However it seems obvious to me that, in the context of dropping actual bombs on actual people, there is an even more serious problem with this model. As Simon Pearson (anti-capitalist musings) puts it:

The “human in the loop” requirement exists in military doctrine because international humanitarian law demands an accountable human decision-maker for lethal force. The laws of armed conflict require proportionality assessments, precautionary measures, distinction between combatants and civilians. All of these obligations attach to a human commander. The system cannot fulfil them. So a human must be present, and their presence must constitute a decision, regardless of whether any genuine decision was made.

What the institution needs from the analyst is not judgment. It is a signature. The signature converts a machine output into a human act. And a human act is what the law recognises, whether or not any judgment occurred. When the strike kills children, the chain of accountability runs to the analyst who approved the target: not to the system that identified it, not to the company that built the system, not to the doctrine that compressed the review window to ten seconds.

But whether we want to make money from exploiting a short term anomaly in a market, make our fellow humans redundant, prosecute a war on another group of fellow humans or “win” a war of mutual nuclear destruction, we need to retain the capacity for real human reflection within the decision-making processes we use. Not just a human-in-the-loop nor just the elites of tech companies deciding how the systems will be configured behind commercially confidential walls. These processes need democratic accountability every bit as much as our parliaments, councils, institutions and voting systems do.

Something infuriatingly slow, inclusive and deliberative giving recommendations which are then stress-tested for how they would perform on contact with reality, involving yet more people being serious and deliberative and taking their responsibilties more seriously than being a human-in-the-loop would ever allow. Our decision-making systems need more grit and less oil. AI is all oil.

It’s always the story

25. February 2026 · Write a comment · Categories: Actuarial, AI, Economics, Education, Media & Politics, Professions, Risk, Science Fiction

The Actuary magazine recently had a debate about whether the underlying data or the story you wove around it was more important. I’m not sure there is always a clear distinction between the two, as Dan Davies rather neatly illustrates here, but my view is that, if a binary choice has to be made, it is always going to be the story. And there was a great example of this which popped up recently in the FT.

The FT article was ‘Is university still worth it?’ is the wrong question, by John Burn-Murdoch, with great graphs as usual by John. However, as is sometimes the case, I feel that a very different and more convincing story could be wrapped around the same datasets he is showing us.

The article’s thesis is as follows:

The graduate earnings premium, ie how much more on average graduates earn than non-graduates, has only fallen in the UK as the proportion going to university has risen. It has risen in other countries:

In the UK, we have had much weaker productivity growth than the other comparator countries, and also “the steady ramping up of the minimum wage has squeezed the earnings premium from the lower end too”:

We have also had a much smaller increase in the percentage of managerial and professional jobs than a different group of comparator countries (they haven’t mentioned Germany before), meaning graduates are forced to take lower salaried jobs elsewhere:

So the answer according to the FT? We should focus on economic growth rather than “tweaking” higher education intake and funding. Then graduate earnings would be higher, student loans could be more generous(!) and students would have more chance of getting a good job.

Well perhaps. But here’s a different framing of the same data that I find more persuasive.

Let’s start by addressing that point about the minimum wage. According to the House of Commons Library report on this, the UK’s minimum wage is broadly comparable to that of France and the Netherlands, although higher than Canada’s and much higher than that of the United States. The employers who are the FT’s constituency would obviously like us lower down this particular chart:

The main economic framing here is the progress myth of the UK’s business community: economic growth. All problems can be solved if we can just get more economic growth. Apparently we need more inequality in pay between graduates and non-graduates which we can get by generating more economic growth. This is honest of them at least, although I don’t see much evidence that the economic growth they crave will go into skilled job creation rather than stock buy backs (according to Motley Fool, “Companies spent $249 billion on stock buybacks in Q3 2025, and $777 billion over the first three quarters of 2025.”).

There are a lot of problems with framing every economic question with respect to economic growth, memorably illustrated by Zack Polanski of the Green Party in this less than 3 minute video recently (I strongly recommend you watch it before you read on – click on the read in browser link if you can’t see it):

Economic growth is increasingly without purpose, wasteful of energy and poorly distributed. It is chasing outputs, literally any outputs, whatever the cost to the environment, our health system, our education system, our social support systems and our communities. Looking at the framing above, you can see that economic growth as currently pursued will always see anything which stops the concentration of wealth amongst the already wealthy, like a higher national minimum wage or a totally made-up concept like a lower graduate earnings premium (which in itself is a framing trying to make reducing inequality seem undesirable) as a problem. Lack of productivity growth, itself a proxy for this kind of economic growth (because if you ask why we need more productivity the answer is always to get more economic growth), is usually directed as a criticism at “lazy” UK workers, rather than under-investing and over-extracting UK business owners.

But what if, instead of economic growth, your progress myth was reducing inequality? Or growing equality within the economy?

Source: World Inequality Database wid.world

If you focused on inequality rather than economic growth, then you would find it correlates with everything we say we don’t want. Unlike economic growth, having equality as an aim actually has the advantage of having an evidence base for the claim that it improves society:

Source: https://media.equality-trust.out.re/uploads/2024/07/The-Spirit-Level-at-15-2024-FINAL.pdf

If you focused on inequality, then you would be pleased that we have had an increase in our minimum wage. You would think that the same FT article’s admission that UK graduates’ skills levels are higher than those in the United States was more important than something called a graduate earnings premium.

Burn-Murdoch is right to say asking whether university is worth it is the wrong question.

However economic growth is the wrong answer.

And I thought I would probably be stopping there for this week. But then something odd happened. A “Thought Exercise” set in June 2028 “detailing the progression and fallout of the Global Intelligence Crisis” (ie science fiction), published on 23 February, may have tanked the share price of IBM later that day. The fall definitely happened, with IBM’s share price falling 13%, its biggest fall since 2000, alongside smaller falls in other tech stocks.

Source: https://markets.ft.com/data/equities/tearsheet/summary?s=IBM:NYQ

According to the FT:

Investors have recently seized on social media rumours and incremental developments by small AI companies to justify further selling, with a widely circulated blog post by Citrini Research over the weekend describing how AI could hypothetically push the US unemployment rate above 10 per cent by 2028, proving the latest catalyst.

The likelihood of the scenario portrayed is difficult to assess, but the speed with which the total economic collapse happens subsequently as described feels unlikely if not impossible. However the fact that the markets are this jittery tells us something I think. As Carlo Iacono puts it:

We are living through a period in which the gap between “plausible narrative” and “tradeable signal” has collapsed to nearly nothing. When a scenario feels real enough to model, and the underlying anxiety is already there waiting to be organised, fiction and forecast become functionally indistinguishable.

The data underlying the markets hasn’t changed, but the story has. I rest my case.

The fog on AI is all mine, all mine

18. February 2026 · Write a comment · Categories: AI, Prediction, Regulation, Risk, Science Fiction
Het Scheepvaartmuseum, Amsterdam, in the fog. Another museum which is well worth a visit

To be read to the accompaniment of Lindisfarne singing Fog on the Tyne, or possibly Kate Bush singing The Fog.

Reporting on AI is all over the place, in both meanings of that phrase. Some think it is very dangerous but that the people working on it should be trusted to police it themselves. Some are retreating from prediction but are instead trying to draw a coastline “knowing the interior is mostly fog”. Some are playing war games in the Arctic with different LLMs. But everyone seems fairly confident they have a hot take. I wonder.

The book I finished this weekend had a passage about a first experiment with a new substance which could shield against gravity. Mr Cavor, the rather unworldly scientist, is explaining to Mr Bedford, a man with no obvious talents other than to look for a quick buck where he can find one, what would have happened if his substance, Cavorite, had not got dislodged fairly quickly from where they had positioned it:

“You perceive,” he said, “it formed a sort of atmospheric fountain, a kind of chimney in the atmosphere. And if the Cavorite itself hadn’t been loose and so got sucked up the chimney, does it occur to you what
would have happened?”

I thought. “I suppose,” I said, “the air would be rushing up and up over that infernal piece of stuff now.”

“Precisely,” he said. “A huge fountain—”

“Spouting into space! Good heavens! Why, it would have squirted all the atmosphere of the earth away! It would have robbed the world of air! It would have been the death of all mankind! That little lump of stuff!”

“Not exactly into space,” said Cavor, “but as bad—practically. It would have whipped the air off the world as one peels a banana, and flung it thousands of miles. It would have dropped back again, of course—but on an asphyxiated world! From our point of view very little better than if it never came back!”

I stared. As yet I was too amazed to realise how all my expectations had been upset. “What do you mean to do now?” I asked.

“In the first place if I may borrow a garden trowel I will remove some of this earth with which I am encased, and then if I may avail myself of your domestic conveniences I will have a bath. This done, we will converse more at leisure. It will be wise, I think”—he laid a muddy hand on my arm—“if nothing were said of this affair beyond ourselves. I know I have caused great damage—probably even dwelling-houses may be ruined here and there upon the country-side. But on the other hand, I cannot possibly pay for the damage I have done, and if the real cause of this is published, it will lead only to heartburning and the obstruction of my work. One cannot foresee everything, you know, and I cannot consent for one moment to add the burden of practical considerations to my theorising…”

The extract is, of course, from HG Wells’ classic The First Men in the Moon, published in 1901.

In case you are in any doubt, Dario Amodei is our Mr Cavor here. I can just imagine his response to the first disaster attributed to AI research being prefaced by “one cannot foresee everything, you know…”. And there are too many Mr Bedfords out there to shake a stick at, trying to sell you anything they can possibly attribute to AI just to keep the whole thing rolling along.

I am with the fog people. The FT seem to be too, with this pair of diagrams attached to this article.

First the US, where there are tentative signs of something they can possibly use as a proxy for productivity growth as a result of using AI:

Source: https://www.ft.com/content/d6fdc04f-85cf-4358-a686-298c3de0e25b

And this one for the UK, where there aren’t:

And so it was this foggy sensibility about AI which I took with me to the Bletchley Park Museum last weekend, site of the AI Safety Summit in November 2023 which drew in the US Vice President, Kamala Harris, European Commission President Ursula von der Leyen, Elon Musk, then UK Prime Minister Rishi Sunak, Open AI’s Sam Altman, Meta’s Nick Clegg and Prof Yann LeCun, Meta’s chief AI scientist, amongst around 100 guests invited to suck their teeth about AI.

The thing that particularly struck me at Bletchley Park is that it demystified the emergence of the computer for me. The forerunner, which was the mechanisation using punch cards of the process of sorting the massive amounts of data the centre was receiving in war time, smacks of a group of people who had just run out of wall to spread their webs of cards and strings across. It was a crime investigation which had got out of hand.

A highlight for me was Alan Turing’s very prescient little note about AI, written in 1940 but anticipating the arguments which would be raging by 2026 (and how poignant that the man who probably did more than anyone to transform what we are able to do by punching a keyboard was chained to one that could only press hunks of metal against a strip of carbon onto a piece of paper):

There is also a hilarious secrecy pledge from the ancestors of the safety summit people, telling you all the ways in which you just need to shut up:

“There is an English proverb none the worse for being seven centuries old:” it thunders.

Wicked tongue breaketh bone,

Though the tongue itself hath none.

Words to live by, I’m sure we’d all agree.

What Bletchley Park was less good at was explaining how the Enigma code was cracked, despite an excellent collection of the hardware involved. For that, I recommend Simon Singh’s The Code Book.

Here was the world’s first “intelligence factory”, scaling up intelligence gathering and analysis as never before and by so doing also changing the way governments would interact with their populations, with just as many implications for our current times as the development of AI. This cluster of huts around a country house rebranded as GCHQ and moved to Cheltenham a few years after World War 2.

Path dependence is a term which describes a situation where past events or decisions constrain later events or decisions. Bletchley Park feels like the Museum of Path Dependence to me.

And the legacy of the safety summit? Well my “hot take” would be: when you are a little lost in the fog, it is generally advisable to slow down a bit and take steps to reduce your risk of breaking things. I wonder if I can get that on a bumper sticker.

« Older articles