Getting Serious About Miracles

17. January 2021 · Write a comment · Categories: Prediction, Regulation, Risk

There are many papers about model risk, and the dangers of blindly relying on algorithms or metrics without allowing for human judgement at any point in any subsequent analysis (in effect “baking in” whatever analysis was done at the time the computer model or algorithm was constructed as the final word), but these can often descend into the same level of technical impenetrability as the programmes they are attempting to critique.

I watched the film Sully: Miracle on the Hudson for the first time this week, on the anniversary of the landing on the Hudson. In the final scenes there is a hearing (spoiler alert!), where the evidence presented up until that point based on computer simulations, with and without pilots involved, was leading to the unanimous conclusion that Sully and Skiles could have turned back to La Guardia or Teterboro airports rather than landing on the Hudson River in January. However Sully had appealed to have the video recordings of the pilot simulations shown to the hearing, and these revealed the pilots responding to the catastrophic bird strikes which had taken out both engines (again something later confirmed when the actual engines were recovered, but which the simulations themselves did not accept because of the instrument readings on one of the engines from the aircraft) by calmly immediately setting course for La Guardia or Teterboro with no decision or response or recovery time needed at all. When a 35 second allowance for this was inserted into the simulations, the results were fatal crashes in both cases.

What struck me was how invisible this deficiency in the programming of the simulation would have been without a cockpit recording of the simulations. In many of the programmes we use to automate judgement-heavy processes, such as recruitment, many of the capital allocation decisions in financial institutions or even A-level grades, we do not have anything equivalent to a cockpit recording available to us. Perhaps we wait until either events prove us wrong (bad) or those on the receiving end of our automated decisions start to complain in sufficient numbers for us to reconsider (worse). What if quite a large proportion of the cost savings from automating these processes is in fact illusory as a result of our not putting enough time and attention into the original programming and/or not setting aside enough budget for maintaining it and challenging its decisions with parallel processes which do allow for human judgement? How much bigger is this problem going to become in the era of machine learning, where the programmes we are running are themselves several steps of abstraction away from those originally written by humans?

Our ability to programme machines to carry out billions of calculations in seconds would have been regarded as miraculous only a few decades ago and is still pretty astonishing to us now. We need to start thinking a lot more about how we can live alongside these ever more capable machines amicably over the long term. And it can’t be only programmers who get to see what the machines are doing – whatever the technical problems of allowing the equivalent of a cockpit recording to be made which can be understood by any of us, they need to be solved with as much urgency as the process automation itself. All of our decision-making processes need to be understandable and challengeable by the society in whose name they are carried out. It’s time to get serious now about our miracles.

A-levels, algorithms and pulling up the ladder

06. January 2021 · Write a comment · Categories: Actuarial, Professions

I have written about school qualifications once before here in 2014, when I was criticising the move to adding an A* grade at GCSE and the consequent narrowing of the grade boundaries to mimic the A-level ones. We have of course since moved to a numerical grade system for GCSEs which is even narrower. However, if the exam grade system was a bad way to assess students, the algorithm which replaced it in the summer (explained here and critiqued here) was clearly worse still.

So, against a background of steadily less reliable grade information at both GCSE and A-level, it was interesting to look at the Institute and Faculty of Actuaries’ (IFoA’s) employer directory and note that, of the 25 separate adverts for graduate roles, 11 of them have an A-level or UCAS points requirement in addition to the university degree requirement. My question is why?

I understand that employers, particularly this year, are likely to have very large numbers of applicants and need some way of reducing the number they need to review in detail, but there are many much better sieves than A-levels these days. Psychometric tests can assess how rusty students’ numeracy is. Application forms can be digital and given a computerised first pass on any number of criteria and, if the questions are constructed thoughtfully, will give companies a smaller set of applicants much more closely aligned to their goals than the grade given at mathematics A-level.

Even if you accept the grades as representative, there are clearly issues around social mobility and widening participation from relying on them to exclude a large number of candidates initially, which was highlighted when an algorithm attempted to reproduce the results based on subject studied and school attended. The news today that they will not be trying this again this summer is encouraging, but even if mark allocations are fairer, many problems with A-levels remain.

I have felt that this has been a growing issue for some time – it has always seemed to me ridiculous that a student on my programme (the BSc Mathematics and Actuarial Science at the University of Leicester – a qualification accredited by the IFoA), doing well and on track for all 6 of the core principles exemptions available as a result, still feels the need to retake an A level taken before they had discovered the motivation for actuarial work that they now have, in order to have a chance with many of the top employers. Are those employers so lacking in confidence in the integrity of their own profession’s qualification system that they need the security backstop of an A-level pass?

It is likely to be a tough environment for young people attempting to start their careers this year, whatever their skill set. I hope employers will review their current approach to recruitment and check they are not inadvertently pulling up the ladder before seeing all of the talent available.

Fiscal space

05. January 2021 · Write a comment · Categories: COVID-19, Demographics, Economics, Risk

NASA, ESA, and the Hubble Heritage Team (STScI/AURA), Public domain, via Wikimedia Commons

Fiscal space is defined as the difference between a nation’s sovereign debt-to-GDP ratio and the limit beyond which the nation will default unless policymakers take fiscal steps that are outside of anything they have done historically. That limit is sometimes referred to as the fiscal cliff, just to ram home the imagery of fixed physical limits beyond which disaster beckons.

How much fiscal space does the UK have? Moody’s have an answer, which depends most heavily on when you ask the question. In September 2019 it was as follows:

This shows the UK with a fiscal space (the “dynamic” means they assume interest rates increase as borrowing does, due to “crowding out” arguments – ie government borrowing pushing up the price of borrowing for everyone – so beloved of most economists) of around 175% of GDP, with this then projected to fall over the following 5 years as rates “normalized”. While the cost of borrowing seems to be dynamic, the actual borrowing itself is not allowed to be in these calculations – it is assumed that they just add to debt without increasing the revenue components of the primary balance.

Well of course then we had 2020, at which point (June 2020) Moody’s appear to have stopped talking about fiscal space and instead are now focusing on something called “debt affordability”. What happened to dynamism and crowding out? Not explained:

However despite this triumph of debt affordability, they then produce another graph to indicate that governments still need to be bearing down on debt to GDP ratios:

As they say in the document “rating implications will depend on governments’ ability to reverse debt trajectories ahead of potential future shocks”. Remember this was in June 2020. Let’s also remind ourselves of another graph:

Requiring governments to reverse debt trajectories in this environment is insane and likely to result in more deaths if not ignored. However as recently as last month in their issuer comment for the UK they said:

However, compared to the government’s March budget (that was quickly overtaken by events), there are some initial signs that fiscal policy outside of investment is likely to be less expansive than previously announced. What remains unclear is whether this ambition will be able to withstand the political pressures that seem to be inevitable given the government’s previous commitments. Even before the Spending Review, longer-term spending commitments for health, education, and defence had already been announced. Together, these three areas account for around 60% of total expenditure.

I have been hard on Moody’s in this piece, they are most certainly not alone. But this attempt to divorce sovereign debt levels from what is actually going on in countries needs to stop as does the constant discounting of the value of any government spending at all. Political pressures to spend more on health and education are not always things that governments need to “withstand” in order to look good in a Moody’s graph. There are far more important things at stake.

 

Change and Innovation-Speak

03. January 2021 · Write a comment · Categories: COVID-19, Economics

Blaise Pascal, mathematician and philosopher, once said:

All of humanity’s problems stem from man’s inability to sit quietly in a room alone.

This seems to have a particular relevance at the moment, when many of us are being asked to do precisely that. I also agree that this is definitely a problem we have. However I prefer to think of it as just one consequence of our inability to think about change in any rational way. We fear change, which is why we yearn so much to go back to “normal” at the moment, even if normal life was pretty unsatisfactory for many of us before the pandemic struck. We fight against change if we think what we have is threatened from outside the room we might otherwise sit quietly in, whether that is the loss of our income or that of our influence in the world or our “sovereignty”.

The only way in which we can contemplate change is in the context of some utopian ideal of improved productivity making one aspect of our lives much better while not requiring us to change any other part of them. Hence so much resistance to any idea of redistributing what we already have in favour of “Pareto improvements” to the economy, ie those which benefit some people without making anyone else worse off, and the obsession amongst economists with the “productivity puzzle” in the UK in particular:

So we look for ways to achieve this miraculous productivity improvement while leaving everything else essentially unchanged and the magic word which promises this more than anything else is innovation. Innovation will enable us to do more with less (or, more usually, make us do a lot more much cheaper, therefore encouraging us to use even more in the process). Innovation will have spin offs in lots of other areas we have not even imagined yet, but they will all be good ones! Innovation will solve the productivity puzzle.

In The Innovation Delusion, Lee Vinsel and Andrew Russell challenge this. As we have become more and more desperate for all change to look like innovation, we have made actual innovation harder to achieve, while saddling ourselves with higher and higher maintenance costs of new “innovative” infrastructure which is increasingly unsustainable to finance, rather than maintaining what we already have better.

I therefore prefer the quote that they use, from Kurt Vonnegut:

Another flaw in the human character is that everybody wants to build and nobody wants to do maintenance.

Innovation-speak, as they call it, is not innovation at all, but presenting ideas as innovative when they are not. As they say:

It plays on our worry that we will be left behind: our nation will not be able to compete in the global economy; our businesses will be disrupted; our children will fail to find good jobs because they don’t know how to code…Innovation-speak is a dialect of perpetual worry.

No wonder we are unable to sit quietly in a room alone.

And in the coming years when we will need to make substantial changes that work well enough for all of us to be able to continue living on this planet together, this approach will not work. We need for our thinking not to be magical, but grounded in realism. We need to make new things that we can afford to maintain sustainably. Innovation-speak will not get us there.

NEW versus England and Wales – an update

02. January 2021 · Write a comment · Categories: COVID-19, Demographics

I previously wrote a blog in 2013 based around the Office for National Statistics (ONS) statistical bulletin entitled Estimates of the Very Old (including Centenarians), 2002-2011, England and Wales, which summarises how the proportions living to 90 years old and above have changed since 1981. It showed us a population living within a population: Nonagenarian (ie the over 90s) England and Wales (NEW) within the full population of England and Wales. I thought it might be time for an update, based on the latest ONS bulletin from September 2020, which now covers the period 2002-2019.

There have been quite a few changes. There are still more women than men in NEW, although the overall ratio has reduced from 2.7:1 in 2011 to around 2:1 in 2019 (see below). The NEW population, which was somewhere between the sizes of Malta’s and Cape Verde’s full population in 2011, has now just passed that of Western Sahara and has its sights firmly set on passing Luxembourg’s population next.

The population of NEW is still growing far more quickly than that of England and Wales, or indeed the UK, with a 25% increase between 2011 and 2019. However, with the NEW population you need to look beyond just improvements in public health and medical advances to the time at which they were being born. For instance, the number of people alive at almost every age from 90 years and above was higher in 2019 than in 2018, but with by far the largest increase at age 99 years (62.2%). This was caused by a big increase in births from the second half of 1919, compared to the previous year, as a result of the end of World War 1!

The bulletin ends with a sombre reminder that, although we would normally expect the large increase in those aged 99 years in 2019 to translate into a record number of centenarians in 2020, other factors, particularly the COVID-19 pandemic, are likely to have had a significant impact. COVID-19 deaths are highest for the 85 years and over age group. Public Health England have calculated excess deaths in the over 85 population at 11,656 between 21 March and 18 December 2020 (with 13,844 categorised as COVID deaths, suggesting a drop in excess deaths from other causes). This compares with the 2019 NEW population of 605,181, an increase of 21,157 on 2018.