Earthquakes and Equities

10. August 2013 · Write a comment · Categories: Economics, Prediction

Plotting the frequency of earthquakes higher than a given magnitude on a logarithmic scale gives a straightish line that suggests we might expect a 9.2 earthquake every 100 years or so somewhere in the world and a 9.3 or 9.4 every 200 years or so (the Tohoku earthquake which led to the Fukushima disaster was 9.0). Such a distribution is known as a power-law distribution, which gives more room for action at the extreme ends than the more familiar bell-shaped normal distribution, which gives much lower probabilities for extreme events.

earthquakes

Similarly, plotting the annual frequency of one day falls in the FTSE All Share index higher than a given percentage on a logarithmic scale also (as you can see below) gives a straightish line, indicating that equity movements may also follow a power-law distribution, rather than the normal distribution (or log normal, where the logarithms are assumed to have a normal distribution) they are often modelled with.
However the similarity ends there, because of course earthquakes normally do most of their damage in one place and on the one day, rather than in the subsequent aftershocks (although there have been exceptions to this: in The Signal and the Noise, Nate Silver cites a series of earthquakes on the Missouri-Tennessee border between December 1811 and February 1812 of magnitude 8.2, 8.2, 8.1 and 8.3 respectively). On the other hand, large equity market falls often form part of a sustained trend (eg the FTSE All Share lost 49% of its value between 11 June 2007 and 2 March 2009) with regional if not global impacts, which is why insurers and other financial institutions which regularly carry out stress testing on their financial positions tend to concern themselves with longer term falls in markets, often focusing on annual movements.

equities

How you measure it obviously depends on the data you have. My dataset on earthquakes spans nearly 50 years, whereas my dataset for one day equity falls only starts on 31 December 1984, which was the earliest date from which I could easily get daily closing prices. However, as the Institute of Actuaries’ Benchmarking Stochastic Models Working Party report on Modelling Extreme Market Events pointed out in 2008, the worst one-year stock market loss in UK recorded history was from the end of November 1973 to the end of November 1974, when the UK market (measured on a total return basis) fell by 54%. So, if you were using 50 years of one year falls rather than 28.5 years of one day falls, a fall of 54% then became a 1 in 50 year event, but it would become a 1 in 1,000 year event if you had the whole millennium of data.

On the other hand, if your dataset is 38 years or less (like mine) it doesn’t include a 54% annual fall at all. Does this mean that you should try and get the largest dataset you can when deciding on where your risks are? After all, Big Data is what you need. The more data you base your assumptions on the better, right?

Well not necessarily. As we can already see from the November 1973 example, a lot of data where nothing very much happens may swamp the data from the important moments in a dataset. For instance, if I exclude the 12 biggest one day movements (positive and negative) from my 28.5 year dataset, I get a FTSE All Share closing price on the 18 July 2013 of 4,494 rather than 3,513, ie 28% higher.

Also, using more data only makes sense if that data is all describing the same thing. But what if the market has fundamentally changed in the last 5 years? What if the market is changing all the time and no two time periods are really comparable? If you believe this you should probably only use the most recent data, because the annual frequency of one day falls of all percentages appears to be on the rise. For one day falls of at least 2%, the annual frequency from the last 5 years is over twice that for the whole 28.5 year dataset (see graph above). For one day falls of at least 5%, the last 5 years have three times the annual frequency of the whole dataset. The number of instances of one day falls over 5.3% drop off sharply so it becomes more difficult to draw comparisons at the extreme end, but the slope of the 5 year data does appear to be significantly less steep than for the other datasets, ie expected frequencies of one day falls at the higher levels would also be considerably higher based on the most recent data.

Do the last 5 years represent a permanent change to markets or are they an anomaly? There are continual changes to the ways markets operate which might suggest that the markets we have now may be different in some fundamental way. One such change is the growth of the use of models that take an average return figure and an assumption about volatility and from there construct a whole probability distribution (disturbingly frequently the normal or log normal distribution) of returns to guide decisions. Use of these models has led to much more confidence in predictions than in past times (after all, the print outs from these models don’t look like the fingers in the air they actually are) and much riskier behaviour as a result (particularly, as Pablo Triana shows in his book Lecturing Birds on Flying, when traders are not using the models institutional investors assume they are in determining asset prices). Riskier behaviour with respect to how much capital to set aside and how much can be safely borrowed for instance, all due to too much confidence in our models and the Big Data they work off.

Because that is what has really changed. Ultimately markets are just places where we human beings buy and sell things, and we probably haven’t evolved all that much since the first piece of flint or obsidian was traded in the stone age. But our misplaced confidence in our ability to model and predict the behaviour of markets is very much a modern phenomenon.

Just turning the handle on your Big Data will not tell you how big the risks you know about are. And of course it will tell you nothing at all about the risks you don’t yet know about. So venture carefully in the financial landscape. A lot of that map you have in front of you is make-believe.

Needles and Pins 31 July 2013

31. July 2013 · Write a comment · Categories: Actuarial, Cartoons, Demographics, Economics, Pensions, Prediction

spikes colour

Uncertainty and its discontents

20. June 2013 · Write a comment · Categories: Economics, Prediction, Regulation

There’s certainly a great deal of uncertainty about.

In Nate Silver’s book, The Signal and the Noise, there is a chapter on climate change (which has come in for some criticism – see Michael Mann’s blog on this) which contains a diagram on uncertainty supposedly sketched for him on a cocktail napkin by Gavin Schmidt. It occurred to me that this pattern of uncertainty at different timescales was more generally applicable (it describes very well, for instance, the different types of uncertainty in any projections of future mortality rates). In particular, I think it provides a good framework for considering the current arguments about economic growth, debt and austerity. Some of these arguments look to be at cross-purposes because they are focused on different timeframes.

uncertaintyIn the short term, looking less than 10 years ahead, initial condition uncertainty dominates. This means that in the short term we do not really understand what is currently going on (all our knowledge is to some extent historical) and trends which might seem obvious in a few years are anything but now. Politics operates in this sphere (long term thinking tends to look two parliaments ahead at most, ie 10 years). However, the market traders who by their activities move the markets and market indices on which we tend to base our forecasts and our economic policies are also working in the short term, the very short term (ie less than 3 months to close off a position and be able to compare your performance with your peers), even if they are trading in long term investments.

So both the politics and economics is very short term in its focus, and this is therefore where the debate about growth and austerity tends to be waged. The Austerians (which include the UK Government) claim to believe that debt deters growth, and that cutting spending in real terms is the only possible Plan A policy option. The Keynesians believe that, in a recession, and when interest rates cannot go any lower, demand can only be revived by Government spending. This argument is now well rehearsed, and is in my view shifting towards the Keynesians, but in the meantime austerian policies (with all the economic destruction they inevitably cause) continue in the UK.

However, there are other groups seemingly supportive of the UK Government’s position in this argument for altogether different reasons. Nassim Nicholas Taleb argues that high levels of debt increase an economy’s fragility to the inevitable large devastating economic events which will happen in the future and which we cannot predict in advance. He therefore dismisses the Keynesians as fragilistas, ie people who transfer more fragility onto the rest of us by their influence on policy. These concerns are focused on the structural uncertainty which is always with us and is difficult to reduce significantly. It is therefore important to reduce (or, if possible, reverse) your fragility to it.

At the longer term end are the groups who believe that we need to restrict future economic growth voluntarily before it is done for us, catastrophically rapidly, by a planet whose limits in many areas may now be very close to being reached. They are therefore implacably opposed to any policy which aims to promote economic growth. These concerns are focused where there are many possible future scenarios (ie scenario uncertainty), some of which involve apocalyptic levels of resource depletion and land degradation.

These different groups are tackling different problems. I do not believe that those concerned with the structural fragility of the economy really believe that the people paying for the restructure should be the disabled or those on minimum wage. Similarly, there is a big difference between encouraging people to consume less and move to more sustainable lifestyles and recalibrating down what is meant by a subsistence level of existence for those already there.

We do need to worry about too big to fail. Our economy houses too many institutions which appear to be too large to regulate effectively. We do need to reduce levels of debt when economic activity has returned to more normal levels. We will need to restructure our economy entirely for it to make long-term sense in a world where long term limits to growth seem inevitable. But none of these are our immediate concern. We need to save the economy first.

Skin in the right game

11. June 2013 · Write a comment · Categories: Actuarial, Economics, Prediction, Regulation

skin in the gameIn my previous post, I looked at some of the reasons why we are so useless at making economic predictions, and some of the ideas for what might be done about it. One of the key problems, raised by Nassim Nicholas Taleb most recently in his book Antifragile, is the absence of skin in the game, ie forecasters having something to lose if their forecasts are wildly off.

But what if all forecasters had to have something to lose before they were allowed to make forecasts? What if every IMF or OBR forecast came with a bill if it was seriously adrift? What if you knew whenever you read a forecast in a newspaper or on a television screen that the person making that forecast had invested something in their belief in their own forecast?

Betting on events dates back at least to the 16th century, but prediction markets (also known as predictive markets, information markets, decision markets, idea futures, event derivatives, or virtual markets) have developed most strongly over the last 25 years or so (the University of Iowa launched its first electronic market as an experiment in 1988). Intrade, which describes itself as the world’s leading prediction market, is now a little smaller than it was following the news just before Christmas last year that it would no longer let Americans trade on its site. It had been sued by the US regulator of the commodities derivative markets for breaking a commitment not to allow trading on the constituents of those markets.

A paper on prediction markets earlier this year called Prediction Markets for Economic Forecasting by Snowberg, Wolfers and Zitzewitz suggests there are 3 main types:

·         Winner takes all. If the event you have bet on happens you win. If not, you lose your stake. Intrade is this type of prediction market: you pay a proportion of £10 a share based on the average probability of the event happening (according to the market participants) and get £10 back if it happens, nothing if it doesn’t. The Iowa Electronic Market current offerings, on congressional elections and US federal monetary policy, are also winner takes all. The price of the bet at any time should reflect the market’s view of the probability of the event happening.

·         Index. The amount paid out is unknown but tied to the variable you are betting on, eg the number of seats won by a party in a particular parliamentary election, or the value of an index at close of business on a particular date. The price of the bet at any time should reflect the market’s view of the expected value of the outcome.

·         Spread betting. Most commonly found on sports betting sites, the amount bet and the amount paid out are fixed, but the event that leads to a pay out (eg number of goals scored in a match more than x) changes until the numbers of buys and sells match (a “buy” in this example is betting the number of goals will be above x, a “sell” is betting the number of goals will be below y, a number less than x. The spread, on which the betting site makes its money, is the difference between x and y. This could equally be applied to the values of an index at a particular date (eg Spreadex offer just such bets on several major share indices as well as currency exchange rates). Depending on the relationship between the pay out and the bet, the value of the spread points at any time should reflect the market’s view of a particular point in the probability distribution of the event, eg if the pay out is twice the bet, this would be the median (ie a 50% chance of the outcome being higher or lower).

As we saw in my previous post, currently economic predictions are largely blown off course by either:

  • Over-confidence in the information used to make them; or
  • The difficulty in standing out against a market which is making everyone a lot of money (buying has limited downside, selling limited upside); or (another possibility I haven’t mentioned before)
  • Bias in individual “expert” judgements, eg those with reputations at stake may want to keep their assumptions somewhere in the middle of the pack rather than standing out most of the time as this is less risky (hence the obsession with “benchmarking” assumptions in the actuarial world for instance).

Prediction markets can help with all of these problems:

  • Having to bet on your opinion should cause you to weigh the evidence backing it more carefully. Also, once a market is established and attracting a lot of bets, the range of evidence on which opinions are being based should expand. Prediction markets also appear to be quite difficult to manipulate or arbitrage.
  • Prediction markets can respond very rapidly to changes in the information available. As an example, within 25 minutes of Donald Rumsfeld’s former chief of staff tweeting about the death of Osama Bin Laden, the market view of the probability of this event on a prediction market rose from 7% to 99%.
  • Betting can take place anonymously. So, although the betting site knows who you are, no one else does, and the data from the voting therefore gets out into the public domain without any individual being accused of talking down a market or risking their reputation.

For these reasons, amongst others, forecast accuracy for established prediction markets might be expected to outperform that of professional forecasters. The paper of Wolfers et al suggests that this is the case.

There are still problems. The markets need to be popular to be much use for predictions, so the questions need to be interesting enough for mass participation. Secondly, a market could theoretically be undermined (although not necessarily to the detriment of its predictive ability) by traders with inside information. However, there are quite a few safeguards in place against this type of activity. Intrade, for instance, requires a photo ID and two proofs of address before it registers anyone to trade on their site. And Spreadex are regulated by the Financial Conduct Authority. A third problem is referred to as the “longshot bias”, which is observed on all types of betting markets. People tend to over-bet on events with long odds against them and under-bet on events which are fairly likely (which explains the narrowing of the odds as the starting gun approached on that horse you bet on just because of its name in the Grand National a couple of months ago). This is a problem of the winner takes all type of market, seemingly related to behavioural factors around the difference between how we view winning and losing, and it is difficult to see how it could be avoided completely. Care may need to be taken therefore when interpreting prediction markets on events which are seen as having fairly low probabilities.

But overall, prediction markets would seem to offer a way of significantly improving economic predictions, so why not make them compulsory for people who want to make such forecasts? By putting a cost on such predictions (a minimum bet could be set based on the size of the organisation making it), it would remove the casual forecasting we currently see too much of, and encourage people to review their beliefs rather more carefully. It would also ensure that the markets were popular enough to be effective. It may be that economic forecasting will always be far from perfect, but this seems a good place to start if we want, in Nate Silver’s words, to be “less and less and less wrong”.

A tax on economists? Not at all. But it might mean that we all have skin in the right game.

Skin in the wrong game

10. June 2013 · Write a comment · Categories: Economics, Prediction

The main reason I set up this website was to draw more attention to the fact that we are fairly useless at making economic predictions. The graph I use as a logo is just one of many comparisons of economic projections with reality which could be shown to demonstrate just how useless. The problem is that there is then a bit of a void when we are looking for ways to support economic decision making. Nassim Nicholas Taleb and Nate Silver are, in different ways, exploring ways to fill this void.

Taleb, in his book Antifragile, links the fragility of the economic system, amongst other things, to an absence of “skin in the game”, ie something to lose if things do not go to plan. Something is fragile if it is vulnerable to volatility, robust if it is relatively unaffected by it, and antifragile if it profits from volatility. If decision makers gain an upside when things go right, but are effectively bailed out from the downside when they don’t by others who then suffer the downside instead, they will have no real incentive to be sufficiently careful about the decisions they take. This makes the chance of a downside for everyone else consequently greater. Taleb refers to people in this position, championing risky strategies (or, more often, strategies where the risks are not really known) without facing the risks personally, as “fragilistas”. He suggests instead a system of “nonpredictive decision making under uncertainty” on the basis that “it is far easier to figure out if something is fragile than to predict the occurrence of an event that may harm it”.

Silver, in his book The Signal and the Noise, suggests the main reason why economic predictions are routinely so awful, even when the forecasters are trying to be accurate rather than making a splash, is that the events forecasters are considering are often out of sample, ie the historical data they are considering to make their forecasts do not include the circumstances currently being faced, but that the forecasters are confident enough to make predictions despite this. This explains, for instance, the failure in the US to predict the housing crash (there had never been such a boom before), the failure of the ratings agencies to understand how risky mortgage-backed securities were (they were in new more complex forms) and the failure to predict that the housing market would take the rest of the economy down with it (much more trading betting on house price rises that did not materialise than had ever been seen before).

Silver cites the economist Terence Odean of the University of California at Berkeley, whose paper Do Investors Trade Too Much shows that equity traders trade excessively in the sense that their returns are, on average, reduced through trading (due to transaction costs), and suggests that this is partly due to overconfidence in the constant stream of information in the media drawing their attention from one possible model to another. This effect can be modelled to show that markets behave irrationally in the presence of overconfident decision making, even when decision-makers are otherwise completely rational.

However, when we are looking at the calls made by traders there are other forces at work that make things even worse. Silver’s book advances the theory that it is not so much an absence of skin in the game that leads traders to continue betting on rising markets when bubbles have started to develop, but skin in the wrong game (at least from the point of view of the rest of us fragile people). He gives the example of a trader looking a year ahead to make a call on whether the market will crash or not. If he buys and the market rises everyone is happy. If he buys and the market crashes, he will be in the same boat as everyone else and will probably keep his job if not his bonus. If he sells and the market crashes he will be seen as a genius, but the tangible rewards of this beyond his current position might not be life changing. However, if he sells and the market rises he may never work in the industry again.

For a trader who wants to keep his job and remain popular with his colleagues, selling therefore has limited upside and unlimited downside, to analyse it in a Talebian way. Buying on the other hand has potentially unlimited upside while the music is playing and limited downside when it stops. So these traders do have skin in the game, which keeps them buying even when they can see the iceberg approaching, and does not particularly reward accuracy in forecasting. It’s just a different game to the one the rest of us are playing.

For these amongst many other reasons, economic forecasting (Silver differentiates forecasting – meaning planning under conditions of uncertainty – from predicting – meaning calling the future correctly) is unlikely ever to be very accurate. But whereas Taleb believes that planning for the future must be nonpredictive to avoid making suckers of us all, Silver believes there may be some scope for improvement. This brings us to the idea of prediction markets and their ability to introduce some of the right skin in the right game, which I will discuss in my next post.

DA = CB

23. May 2013 · Write a comment · Categories: Actuarial, Defined Ambition, Economics, Pensions, Regulation

Steve Webb, the pensions minister, thinks we only have 12 months to save DB but that, in its current form, it might be like trying to apply electrodes to a corpse. Unfortunately his prescription – Defined Ambition (DA) – is still very much undefined and therefore, as yet, unambitious.

Pension active membership

Number of members of private sector occupational pension schemes: by membership type and benefit structure, 2004-11

Source: Office of National Statistics

The graph above shows how dramatic the decline of DB active membership (ie members still accruing benefits in defined benefit schemes which provide a pension defined in advance, where the balance of funding is committed to by the employer in nearly all cases) has been in recent years. It also shows, contrary to some reports, that there has been no advance in DC active membership (ie defined contribution schemes where only the contributions are defined in advance and final benefits are at the mercy of financial markets and annuity rates). It just hasn’t fallen much. In fact, if all of the DC active members had instead been offered DB active membership, the number of DB active members would still have fallen.

So it is a crisis and it appears to be those who are opting for no pension scheme at all who are really growing in number. The auto-enrolment programme starting to be rolled out across the country will have an impact, after all if you keep asking the question and don’t take no for an answer you will attract customers – just ask the banks who were selling PPI cover.

But I wonder if the crowd avoiding pensions of any sort up until now might perhaps have more wisdom than those trying to pile them into schemes whether they want to or not. Because DC has to date been a very poor offer for most, with very low levels of contributions. The latest survey by the ONS of households between 2008 and 2010 where the primary earners are between 50 and 64 revealed that median pension savings in DB schemes were equivalent to around six times those in DC schemes. And the minimum contributions under auto-enrolment of 8% of qualifying earnings from all sources with all risks staying with the member is unlikely to change this massive inequality quickly if at all.

If you have very little money, and the pension option means that your pension contributions are likely to be bounced around by the markets for a few decades before dribbling out in whatever exchange the insurance companies are prepared to give you, is it irrational to think that you might want to keep some access to your savings along the way? The following graph suggests most people don’t think so.

Decile savings

Breakdown of aggregate saving, where household head is aged 50 to 64: by deciles and components, 2008/10

Source: Office of National Statistics

This graph suggests that people do save for a pension where they can, but if there is not much to go round, they also want some more liquid savings. The problem is not that they are not saving for a pension, it is that they have no assets at all.

So what is to be done? Clearly campaigning for a living wage needs to continue and be intensified, and reductions to benefits are going to make the problem worse. But fiddling around with marginally different forms of DC arrangements for decades will also be disastrous. Think not just a few naked pensioners on the beach as we had before the Pension Protection Fund (PPF) came in for DB members. Think armies of them with a genuine grievance against a society that did this to them. And what will have been done to them is to suggest that by paying 4% of their salary into a pension scheme, they have somehow safeguarded their future. Good employers are not going to want to be associated with scenes (or schemes) like this.

DC contributions need to be much higher while they remain so risky, which is why DB schemes target asset levels much higher than their best estimate of the cost in most cases, but clearly DB levels are too high for nearly all employers. There is not much time, as Steve Webb says, so let’s stop messing around and pick an alternative.

I vote for cash balance (CB). There are many different sorts but the feature they all have in common is a defined cash sum available at retirement which members can then take in a combination of lump sum, annuity and drawdown (ie keeping the sum in the scheme and drawing income from it as needed). It means that the bumping around by the markets is taken on the chin by your employer not you, but only until retirement (the type of risk employers are used to managing in their businesses anyway), and the risk of you living longer (reflected in lower annuity rates) when you get to retirement is your problem. It seems reasonable to me. Whoever thought that an employer should be concerned with how long you are going to live (unless they were the mafia)? Good employers could also offer a broking service for annuity purchase to avoid the problem of pensioners not shopping around adequately.

There are a few of these in existence already, although only 8,000 members in total benefit from them so far. In the case of Morrisons, the guarantee is 16% of salary a year, uprated in line with CPI. This is one of the current minimum levels to be accepted as an auto-enrolment plan. Alternatively you could drop to 8% a year, but uprate it by CPI plus 3.5% pa. Either would be a huge improvement for someone with limited means to relying on what 8% of earnings pa might amount to in 40 years’ time, and unable to take the risk that the answer is not much.

But the first step is to establish CB as what is meant by DA and that will need Government support to work. I propose:

  • CB to be promoted as one of the main options for an auto-enrolment scheme, equivalent to the 8% minimum but without total risk transfer to the employee.
  •  Develop a colour coding scheme for a combination of benefit level and risk transfer, with DC at minimum auto-enrolment at the red end, minimum CB at amber running through green to the equivalent of a public sector DB scheme or better as (NHS) blue.
  • Sort out the PPF position on CB. They currently treat them as full DB schemes. Scale down PPF levies to reflect the lower level of risk that they present to the PPF.
  • Simplify the pensions legislation around CB to reflect the fact that the scheme’s responsibility for managing risk ends at retirement.

And we really need to start now!

Getting excited about economics

16. May 2013 · Write a comment · Categories: Economics

The people in power have no real belief that Plan A will work but refuse to even consider there might be a Plan B. They occupy themselves and the surrounding elite class bubble in which they operate with trivial concerns played out as if they were life and death ones, and the rest of the population are pacified with horse tranquilisers muscle relaxant.

Sound familiar? There is also inevitably a banker involved and someone who cannot stop herself from making dire predictions which her fellow travellers cannot prevent themselves from taking seriously.

Pedro Almodovar has come in for a fair amount of criticism for the perceived shallowness of his latest film Los Amantes Pasajeros (presented as “I’m So Excited” in English due to the prominence of the Pointer Sisters’ song in the film, but more literally translated as the on board or passing lovers). However what came to mind most strongly for me when watching it was Bismarck’s famous comparison between the making of laws and sausages (ie not a pretty sight). And indeed quite a few sausages are “made” during this film, under the influence of a “Valencia cocktail” with added mescaline, as all efforts are concentrated on sleepwalking to the planned emergency landing at an as yet unavailable airport in as pleasantly mindless a way as possible.

An economic policy in which only a tiny minority have any idea what is going on and only a tiny minority of that tiny minority feel able to influence what is going on in the cockpit is a shallow one. Perhaps we all need to get a bit more excited.

The Clocks and the Wind

12. May 2013 · Write a comment · Categories: Birmingham, Economics, Prediction

There has been much discussion over the past few months over whether high levels of debt cause low growth (the “austerian” camp, eg Britain, Canada and Germany within the G7) or whether instead low growth causes high levels of debt to accumulate (the “Keynesian” camp, to which Japan appears to be providing leadership currently). There has been relatively little discussion about the possibility that neither is the case.

We are compulsive pattern spotters. That explains to a large extent our dominance as a species, and completely explains the dominant position that mathematics and its applications holds in our culture.

I was reminded most stirringly of this a few years ago, on a lunch break. The Ikon Gallery in Birmingham was hosting an exhibition by Japanese sound artist Yukio Fujimoto called The Tower of Time. However, instead of siting it at their gallery space in Brindley Place, it had instead been staged at Perrott’s Folly, just around the corner from my office at the time.

Yukio Fujimoto. The Tower of Time
Installation view – Perrott’s Folly, Birmingham, UK 2009  Photo: Stuart Whipps

Perrott’s Folly was built in 1758 by John Perrott. It is a building 94 feet high, with one room on each of its six octagonal floors, and no obvious purpose (hence “folly”). It may have been somewhere to spy on his wife from, while she was alive or dead, or it may have been a gambling den for him and his mates. Or it may have been something else entirely. I think we are unlikely to ever know for sure.

After a brief introduction on the ground floor, I climbed the stairs to the first floor to find one little black square alarm clock with a red second hand ticking in the middle of the wooden floor. The next floor had ten such clocks, in a row. The next 100, in a square, the fifth floor had 1,000.

A curious thing happened to me as I moved up the tower. The clocks’ mechanisms appeared to alter with altitude. I put it that way as an example of an obviously false causality, ie that the height above sea level in some way affected how the clocks worked (and before I get complaints, I mean effects that could be detected within a matter of a few tens of feet and with no measuring equipment other than my eyes and ears). Because what I saw did change. I looked at one clock and I could see that the battery was powering the gear mechanism that kept the second hand, minute hand and hour hand in their required relative motion. I looked at ten clocks in a row and I could see the same, although I also noticed the second hands were not all at the same point along the row and that there was an order in which each piece of red plastic reached the top before beginning the next circuit. I found myself having to watch the clocks for several minutes to see the pattern confirmed. But was this “pattern” anything which had any meaning, or was it just a way for my brain to store the images it was collecting in an easily fileable format?

When I moved to 100 clocks, the relevance of the gear mechanism became secondary. I could “see” lines of second hands moving together in the way that lines of plants in a cornfield move with the breeze. This, combined with the swooshing of 100 clocks (as the ticking of each individual clock combined to make a different noise – this change in sound was I believe the artist’s main reason for constructing the installation in the first place), made me need to check several times that one of the strange pointed windows in the tower had not been opened and let in a stray breeze. At 1,000 clocks it was just pure cornfield, the individual clocks now as hard to imagine as it had been to imagine anything else four floors below.

I can “see” that the “wind” is blowing a pattern through the second hands of the clocks and yet I “know” that this is not happening. Now transfer that wind I can see to a situation where I do not readily have a theory for what is happening to individual elements within a system. Suddenly what anyone with eyes can see becomes so much more powerful than what we might know. Returning to the austerity debate for instance, perhaps the individual growth clocks have no relationship with the patterns of debt I can see being blown through them. Perhaps if I just arranged the clocks differently I would see the wind blowing from a different direction. Perhaps the clocks and the wind have nothing to do with each other outside my head, despite the “evidence” of my eyes.

Why does it matter? Because if we cannot prevent ourselves from seeing patterns and then extending them via models where we have to make some things depend on other things, even in the face of weak and conflicting evidence, then we need to know this about ourselves. Because if giving a person the wrong map is worse than not giving him one at all, our natural instinct to construct these maps is likely to keep getting us into trouble.

The Threat of a Good Example

09. May 2013 · Write a comment · Categories: Actuarial, Economics, Pensions, Regulation

The interests of the UK’s private sector defined benefit (DB) pension scheme members, and the security of their vested benefits (ie the ones they are entitled to keep), were weakened this week. The Pensions Regulator, slow to act in many cases, bureaucratic and inconsistent in others, did at least have a coherent set of objectives which allowed it to focus on reducing the fragility of the pensions system overall. However this is not an example of how the Government wants its regulators to behave it seems. The announcement in the Budget in March that the Regulator is to get an additional statutory objective to encourage “sustainable growth” amongst scheme sponsors, following sustained lobbying from the National Association of Pension Funds and the Confederation of British Industry, led to a swift consultation on, and acceptance of, the proposals. It also appears to have led to an equally swift exit for the Regulator’s chief executive Bill Galvin (he leaves next month) who had had dared to reject calls for such an objective, pointing out reasonably that the existing arrangements required the Regulator and trustees to balance the interests of business, the pension scheme and the Pension Protection Fund.

So here it is, the Pensions Regulator’s first statement on DB pension schemes since the new objective was announced. The Regulator looks to have been very mindful of the not-yet-quite-existing objective in framing this statement and, although the precise wording of the objective is not expected until later in the year, has obviously already decided which way the wind is blowing. The key word that jumps out at you on a first skim is “flexibility”, which seems to be the new code for weakening regulation now that “light touch” has been discredited. This contrasts with last year’s statement, when the use of the word was accompanied by a warning that “we will consider whether the flexibility in the funding framework has been used appropriately”, ie emphasising the limits of flexibility rather than its possibilities.

There are also a number of areas where the position taken by the Regulator on funding appears to have noticeably weakened since 12 months ago. Here, in my view, are some of the main ones (italics are mine):

Section

Pension scheme funding in the current environment – April 2012

Section

Defined benefit annual funding statement – May 2013

17

In the regulator’s view, investment outperformance should be measured relative to the kind of near-risk free return that would be assumed were the scheme to adopt a substantially hedged investment strategy.

7

Trustees can use the flexibility available in setting the discount rates for technical provisions…to adopt an approach that best suits the individual characteristics of their scheme and employer.

19, 14

The regulator views any increase in the asset outperformance assumed in the discount rate to reflect perceived market conditions as an increase in the reliance on the employer’s covenant. Therefore, we will expect trustees to have examined the additional risk implications for members and be convinced that the employer could realistically support any higher level of contributions required if the actual investment return falls short of that assumed.

…Where appropriate the use of actual post valuation experience is acceptable.

 

8

The assumptions made for the relative returns of different asset classes may rise or fall from preceding valuations reflecting changes in market conditions and the outlook for future returns. Trustees should ensure that they document their reasons for change and have due consideration to any increase in risk this might bring.

2

As a starting point, we expect the current level of de­ficit repair contributions to be maintained in real terms, unless there is a demonstrable change in the employer’s ability to meet them.

 

12

Where there are significant affordability issues trustees may need to consider whether it is appropriate to agree lower contributions and this may also include a longer recovery plan. Trustees should ensure that they document the reasons for any change and indicated that they have had due consideration of the risks.

Finally, under the heading what you can expect from us, the Regulator also mentions that it has discarded any triggers it had for subjecting schemes to further scrutiny “on individual items such as technical provisions”.

Unfortunately the combined impact of the changes in emphasis, specific wording and the ditching of the triggers would appear to directly conflict with two of the Pensions Regulator’s definitely-still-existing objectives, namely:

  • to protect the benefits under occupational pension schemes of, or in respect of, members of such schemes; and
  • to reduce the risk of situations arising which may lead to compensation being payable from the Pension Protection Fund.

The House of Lords Select Committee on Regulators in 2007 concluded that:

  • Independent regulators’ statutory remits should be comprised of limited, clearly set out duties and that the statutes should give a clear steer to the regulators on how those duties should be prioritised.
  • Government should be careful not to offload political policy issues onto unelected regulators.

We will have to wait and see exactly where this new objective is to be pitched, but, on the evidence of this funding statement from the Regulator, there must now be considerable doubt that either of the select committee principles will be met.

Set any organisation conflicting objectives and no clear way of prioritising between them and the chances are they won’t achieve any of them. The Pensions Regulator has already started to run this risk.

Spreadsheets and the Under Toad

29. April 2013 · Write a comment · Categories: Actuarial, Economics

In The World According to Garp, John Irving describes how Garp’s son mishears the word “undertow” as a source of danger at the seaside as a child, and spends the rest of his life in fear of the “Under Toad”.  This word now appears in dictionaries as referring to a general fear and anxiety about the unknown and mortality. It sounds like a word almost designed for actuaries, and never more so than when dealing with spreadsheets.

Spreadsheets are of course currently in the news because Thomas Herndon, a graduate student at the University of Massachusetts, was set an exercise to choose an economics paper and replicate its results. He chose Growth in a Time of Debt, a paper by Professors Reinhart and Rogoff, which had been cited by George Osborne more than any other in defence of his policy of austerity.

He couldn’t replicate any of it, and when the professors sent him the spreadsheet they had used, the reasons why became apparent. Only 15 of the 20 countries with high public debt in the analysis had been included in the calculation of average GDP growth. The As to Ds had been missed off. The paper had not been peer reviewed.

This particular error, when combined with other criticisms Herndon and his professors had of the methodology used in the paper, provided considerable challenge to the original conclusions of the analysis and was therefore widely reported due to its implications for UK economic policy in particular. However errors of this kind in Excel spreadsheets are very common.

The European Spreadsheet Risks Interest Group, or EuSpRIG (“yewsprig”) for short, is an organisation sponsored by a group of heavy spreadsheet users which runs conferences and forums designed to pool users’ experiences and suggest best practice in spreadsheet use. EuSpRIG are therefore connoisseurs of the spreadsheet error. They helpfully include a list of spreadsheet horror stories on their website, including the GDP growth one.

Perhaps the most significant spreadsheet foul up on their list is described in the Report of JP Morgan’s Management Task Force regarding billions of losses in 2012 in its chief investment office, which cited a number of spreadsheet errors. However, my personal favourite is the one involving the London 2012 organising committee (Locog) confirming in January 2012 that an error in its ticketing process had led to four synchronised swimming sessions being oversold by 10,000 tickets. Locog said the error occurred when a member of staff made a single keystroke mistake and entered “20,000” into a spreadsheet rather than the correct figure of 10,000 remaining tickets.

It is tempting to think that our technological advancement and exponentially increasing computer power have made some kind of computational HD within our grasp, with every wart and blemish of the object of investigation now detectable by our ever more sophisticated tools. But EuSpRIG estimate that over 90% of spreadsheets contain errors. Most of these will never be found, but lurk beneath the surface threatening the accuracy of any calculations carried out by the spreadsheets concerned. In other words, the Under Toad.

Carveth Read once said (although only famously when it got attributed to Keynes): “It is better to be vaguely right than exactly wrong.” However, when most spreadsheets contain Under Toads, it is clear that a lot of the supposed precision with which information is provided to us is illusory. That exponential increase in computer power has made even the very measurement of precision in the more complicated spreadsheets virtually unknowable. We may never be more than vaguely right, but often have no real idea how wrong we are.

So we check, to ensure that the numbers coming out of the spreadsheets and other models we use are within a tolerable distance of what we would expect. Some of us use pen and paper. The GDP growth Under Toad might for instance happen when a formula which adds up a column in one worksheet is copied to another worksheet where the columns have a different number of rows in them, and the formula is not adjusted. I have certainly done that before now, and only found it when I checked it against a number of other sources. For this reason, I am always a bit nervous about model results being checked by spreadsheet. It doesn’t seem sufficiently unlikely to me that the two could be acceptably close to each other but, perhaps due to entirely different errors, many miles from the truth.

There is a generation of actuaries, of which I am one, that experience almost physical pain when we see students carrying out even the simplest calculations using spreadsheets, knowing that each new one on the block is almost certainly adding to the unknown unknowns of the Under Toad. I know there are just as many mistakes in my biro scrawls, but I also know it will be a lot easier to find them later.

A GDP increase of 0.3% on Thursday was greeted with relief at a triple dip averted, when a fall of just 0.1% would have been met with anguish. Tiny movements in the FTSE 100 are described as “up” and “down”, as if the direction were more important than the amount and when “broadly unchanged” would be a more informative description.

We are obsessed with tiny movements which contain no information and which, thanks to the Under Toad, we cannot meaningfully calculate. This obsession distracts us from seeing the bigger picture, the fuzzy connections that only become apparent when we look up from our HD sharp tiny piece of detail. And our spreadsheets won’t help us with that.

« Older articles
Newer articles »