Prediction – Page 3 – We Know Zero

The difficulties of looking forward

25. January 2014 · Write a comment · Categories: Economics, Prediction, Regulation

We are only six months into the Bank of England’s new regime of giving forward guidance about what circumstances might lead them to adjust the Base Rate and they are already in a bit of a mess with it. Whether forward guidance is abandoned or not is still in the balance, amid much confusion. However, much of this confusion seems to be due to the challenge that events have provided to the assumption that the Bank of England could make reasonably accurate economic predictions.

It turns out that not only did the Bank not know how fast unemployment would fall (not a surprise: the Monetary Policy Committee (MPC) minutes from August make clear that they suspected this might be the case), but neither did they know, when it did fall, what a 7% unemployed economy would look like. The Bank has been very surprised by how fragile it still is.

Back in August 2013, when unemployment was still at 7.7%, the MPC voted to embrace the forward guidance which has now fallen on its face. This said that: In particular, the MPC intends not to raise Bank Rate from its current level of 0.5% at least until the Labour Force Survey headline measure of the unemployment rate has fallen to a threshold of 7%, subject to the conditions below.

The “conditions below” were that all bets would be off if any of three “knockouts” were breached:

1. that it would be more likely than not that CPI 18 to 24 months ahead would be at 2.5% or above (in fact it has just fallen to 2%);

2. medium-term inflation expectations no longer remained “sufficiently well anchored” (the gently sloping graph below would suggest it hasn’t slipped that anchor yet); or

3. the Financial Policy Committee (FPC) judged monetary policy posed “a significant threat to financial stability”. More difficult to give an opinion on that one but, looking beyond the incipient housing market bubble, it is difficult to see that monetary policy is causing any other instability currently. Certainly not compared to the instability which would be caused by jacking up interest rates and sending mortgage defaults through the roof.

Source: Bank of England implied spot inflation curve

So it seems that there has been no clear knock out on any of these three counts, but that the “threshold” (it was never a target after all) of 7% is no longer seen as significant a sign of economic recovery as it had been believed it would only last August.

Fun as it is to watch the illusion of mastery of the economy by the very serious people flounder yet again, as what is an intrinsically good piece of economic news is turned into a fiasco of indecision, I think the Bank is right to believe that it is far too early to raise interest rates. I say so because of two further graphs from the Office of National Statistics (ONS) latest labour market statistics, which were not included in their infographic on the left.

The first is the graph of regional unemployment, which shows very clearly that large areas of the UK are still nowhere near the magic 7% threshold: the variations are so wide and, in austerian times, the resources to address them are so limited that it makes sense not to be overly dazzled by the overall UK number.

The second is the graph of those not looking or not available for work in the 16-64 age group since the 1970s. As you can see, it has recently shown a very different pattern to that of the unemployment graph. In the past (and borne out by the data from 1973 to around 1993) the number not available to work has tended to mirror the unemployment rate as people who could manage without work withdrew from the job market when times got tough and came back in when things picked up. However in the early 90s something new started to happen: people withdrawing from the job market even when unemployment was falling. There has been a steady increase in their number until it finally started to fall only last year. So what is happening?

One of the factors has been a big increase in the number of people registered as self employed, rising from 4.2 million in 1999 to 5.1 million in 2011. However, many of these people are earning very little and I suspect that at least some of them would have been categorised as unemployed in previous decades. There must therefore be some doubt about whether 7% unemployed means what it used to mean.

The Bank of England have shown with their difficulties over forward guidance that it is very hard to look forward with any degree of precision. It should be applauded for admitting that it doesn’t know enough at the moment to start pushing up interest rates.

£1,000 a year rise? Probably not

07. January 2014 · Write a comment · Categories: Economics, Pensions, Prediction, Regulation

There has been the usual flurry of misleading headlines around the Prime Minister’s pledge to maintain the so-called triple lock in place for the 2015-20 Parliament. The Daily Mail described it as a “bumper £1,000 a year rise”. Section 150A of the Social Security Administration Act 1992, as amended in 2010, already requires the Secretary of State to uprate the amount of the Basic State Pension (and the Standard Minimum Guarantee in Pension Credit) at least in line with the increase in the general level of earnings every year, so the “bumper” rise would only be as a result of earnings growth continuing to grind along at its current negative real rate.

However, the Office for Budget Responsibility (OBR) is currently predicting the various elements of the triple lock to develop up until 2018 as follows:

The OBR have of course not got a great track record on predicting such things, but all the same I was curious about where the Daily Mail’s number could have come from.

The Pensions Policy Institute’s (PPI’s) report on the impact of abandoning the triple lock in favour of just a link to earnings growth estimates the difference in pension in today’s money could be £20 per week, which might be the source of the Daily Mail figure, but not until 2065! I think if we maintain a consistent State Pensions policy for over 50 years into the future a rise of £20 per week in its level will be the least remarkable thing about it.

The PPI’s assumption is that the triple lock, as opposed to what is statutorily required, would make a difference to the State Pension increase of 0.26% a year on average. It is a measure of how small our politics has become that this should be headline news for several days.

Carry on Clio

04. January 2014 · Write a comment · Categories: Actuarial, Demographics, Economics, Prediction

It’s a relatively new science, and one which binds together many different academic disciplines: mathematical modelling, economics, sociology and history. In economic terms, it is to what economists in financial institutions spend most of their time focusing on – the short to medium term – as climate science is to weather forecasting. Cliodynamics (from Clio, the Ancient Greek muse or goddess of history (or, sometimes, lyre playing) and dynamics, the study of processes of change with time) looks at the functioning and dynamics of historical societies, ie societies for which the historical data exists to allow analysis. And that includes our own.

Peter Turchin, professor of ecology and mathematics at the University of Connecticut and Editor-in-Chief of Cliodynamics: The Journal of Theoretical and Mathematical History, wrote a book with Sergey Nefedev in 2009 called Secular Cycles. In it they took the ratio of the net wealth of the median US household to the largest fortune in the US (the Phillips Curve) to get a rough estimate of wealth inequality in the US from 1800 to the present. The graph of this analysis shows that the level of inequality in the US measured in this way peaked in World War 1 before falling steadily until 1980 when Reagan became US President, after which it has been rising equally steadily. By 2000,inequality was at levels last seen in the mid 50s, and it has continued to increase markedly since then.

The other side of Turchin’s and Nefedev’s analysis combines four measures of wellbeing: economic (the fraction of economic growth that is paid to workers as wages), health (life expectancy and the average height of native-born population) and social optimism (average age of first marriage). This seems to me to be a slightly flaky way of measuring this, particularly if using this measure to draw conclusions about recent history: the link between average heights in the US and other health indicators are not fully understood, and there are a lot of possible explanations for later marriages (eg greater economic opportunities for women) which would not support it as a measure of reduced optimism. However, it does give a curve which looks remarkably like a mirror image of the Phillips Curve.

The Office of National Statistics (ONS) are currently developing their own measure of national well-being for the UK, which has dropped both height and late marriage as indicators, but unfortunately has expanded to cover 40 indicators organised into 10 areas. The interactive graphic is embedded below.

Graphic by Office for National Statistics (ONS)

I don’t think many would argue with many of these constituents except that any model should only be as complicated as it needs to be. The weightings will be very important.

Putting all of this together, Turchin argues that societies can only tolerate a certain level of inequality before they start finding more cooperative ways of governing and cites examples from the end of the Roman civil wars (first century BC) onwards. He believes the current patterns in the US point towards such a turning point around 2020, with extreme social upheaval a strong possibility.

I am unconvinced that time is that short based solely on societal inequality: in my view further aggravating factors will be required, which resource depletion in several key areas may provide later in the century. But Turchin’s analysis of 20th century change in the US is certainly coherent, with many connections I had not made before. What is clear is that social change can happen very quickly at times and an economic-political system that cannot adapt equally quickly is likely to end up in trouble.

And in the UK? Inequality is certainly increasing, by pretty much any measure. And, as Richard Murphy points out, our tax system appears to encourage this more than is often realised. Cliodynamics seems to me to be an important area for further research in the UK.

And a perfect one for actuaries to get involved in.

State Pension Age and its discontents

10. December 2013 · Write a comment · Categories: Demographics, Economics, Pensions, Prediction, Regulation

When I started writing this blog in April, one of its main purposes was to highlight how poor we are at forecasting things, and suggest that our decision-making would improve if we acknowledged this fact. The best example I could find at the time to illustrate this point were the Office of Budget Responsibility (OBR) Gross Domestic Product (GDP) growth forecasts over the previous 3 years.

Eight months on it therefore feels like we have come full circle with the publication of the December 2013 OBR forecasts in conjunction with the Chancellor’s Autumn Statement. Little appears to have changed in the interim, the coloured lines on the chart below of their various forecasts now joined by the latest one all display similar shapes steadily moving to the right, advising extreme caution in framing any decision based on what the current crop of forecasts suggest.

However, the worse the forecasts are revealed to be, the keener it seems politicians of all the three main parties are to base policy upon them. The Autumn Statement ran to 7,000 words, of which 18 were references to the OBR, with details of their forecasts taking up at least a quarter of the speech. In every area of economic policy, from economic growth to employment to government debt, it seemed that the starting point was what the OBR predicted on the subject. The Shadow Chancellor appears equally convinced that the OBR lends credibility to forecasting, pleading for Labour’s own tax and spending plans to be assessed by them in the run up to the next election.

I am a little mystified by all of this. The updated graph of the OBR’s performance since 2010 does not look any better than it did in April, the lines always go up in the future and so far they have always been wrong. If they turn out to be right (or, more likely, a bit less wrong) this time, then that does not seem to me to tell us anything much about their predictive skill. It takes great skill, as Les Dawson showed, to unerringly hit the wrong notes every time. It just takes average luck to hit them occasionally.

For another bit of crystal ball gazing in his Statement, the Chancellor abandoned the OBR to talk about state pension ages. These were going to go up to 68 by 2046. Now they are going to go up to 68 by the mid 2030s and then to 69 by the late 2040s. There will still be people alive now who were born when the state retirement age (for the “Old Age Pension” as it was then called) was 70. It looks like we are heading back in that direction again.

The State Pension Age (SPA) was introduced in 1908 as 70 years for men and women, when life expectancy at birth was below 55 for both. In 1925 it was reduced to 65, at which time life expectancy at birth had increased to 60.4 for women and 56.5 for men. In 1940, a SPA below life expectancy at birth was introduced for the first time, with women allowed to retire from age 60 despite a life expectancy of 63.5. Men, with a life expectancy of 58.2 years were still expected to continue working until they were 65. Male life expectancy at birth did not exceed SPA until 1948 (source: Human Mortality Database).

In 1995 the transition arrangements to put the SPA for women back up to 65 began, at which stage male life expectancy was 73.9 and female 79.2 years. In 2007 we all started the transition to a new SPA of 68. In 2011 this was speeded up and last week the destination was extended to 69.

Where might it go next? If the OBR had a SPA modeller anything like their GDP modeller it would probably say up, in about another 2 years (just look again at the forecasts in the first graph to see what I mean). Ministers have hit the airwaves to say that the increasing SPA is a good news story, reflecting our increasingly long lives. And the life expectancies bear this out, with the 2011 figures showing life expectancy at birth for males at 78.8 and for females at 82.7, with all pension schemes and insurers building in further big increases to those life expectancies into their assumptions over the decades ahead.

And yet. The ONS statistical bulletin in September on healthy life expectancy at birth tells a different story which is not good news at all. Healthy life expectancies for men and women (ie the maximum age at which respondents would be expected to regard themselves as in good or very good health) at birth are only 63.2 and 64.2 years respectively. If people are going to have to drag themselves to work for 5 or 6 years on average in poor health before reaching SPA under current plans, how much further do we really expect SPA to increase?

Some have questioned the one size fits all nature of SPA, suggesting regional differences be introduced. If that ever happened, would we expect to see the mobile better off becoming SPA tourists, pushing up house prices in currently unfashionable corners of the country just as they have with their second homes in Devon and Cornwall? Perhaps. I certainly find it hard to imagine any state pension system which could keep up with the constantly mutating socioeconomics of the UK’s regions.

Perhaps a better approach would be a SPA calculated by HMRC with your tax code. Or some form of ill health early retirement option might be introduced to the state pension. What seems likely to me is that the pressures on the Government to mitigate the impact of a steadily increasing SPA will become one of the key intergenerational battlegrounds in the years ahead. In the meantime, those lines on the chart are going to get harder and harder for some.

Land mines and bank bailouts

28. August 2013 · Write a comment · Categories: Cartoons, Economics, Prediction, Regulation, Risk

A man is sentenced to 7 years in prison for selling bomb detectors which had no hope of detecting bombs. The contrast with the fate of those who have continued to sell complex mathematical models to both large financial institutions and their regulators over 20 years, which have no hope of protecting them from massive losses at the precise point when they are required, is illuminating.

The devices made by Gary Bolton were simply boxes with handles and antennae. The “black boxes” used by banks and insurers to determine their worst loss in a 1 in 200 probability scenario (the Value at Risk or “VaR” approach) are instead filled with mathematical models primed with rather a lot of assumptions.

The prosecution said Gary Bolton sold his boxes for up to £10,000 each, claiming they could detect explosives. Towers Watson’s RiskAgility (the dominant model in the UK insurance market) by contrast is difficult to price, as it is “bespoke” for each client. However, according to Insurance ERM magazine in October 2011, for Igloo, their other financial modelling platform, “software solutions range from £50,000 to £500,000 but there is no upper limit as you can keep adding to your solution”.

Gary Bolton’s prosecutors claimed that “soldiers, police officers, customs officers and many others put their trust in a device which worked no better than random chance”. Similar things could be said about bankers during 2008 about a device which worked worse the further the financial variables being modelled strayed from the normal distribution.

As he passed sentence, Judge Richard Hone QC described the equipment as “useless” and “dross” and said Bolton had damaged the reputation of British trade abroad. By contrast, despite a brief consideration of alternatives to the VaR approach by the Basel Committee on Banking Supervision in 2012, it remains firmly in place as the statutory measure of solvency for both banks and insurers.

The court was told Bolton knew the devices – which were also alleged to be able to detect drugs, tobacco, ivory and cash – did not work, but continued to supply them to be sold to overseas businesses. In Value at Risk: Any Lessons from the Crash of Long-Term Capital Management (LTCM)? Mete Feridun of Loughborough University in Spring 2005 set out to analyse the failure of the Long Term Capital Management (LTCM) hedge fund in 1998 from a risk management perspective, aiming at deriving implications for the managers of financial institutions and for the regulating authorities. This study concluded that the LTCM’s failure could be attributed primarily to its VaR system, which failed to estimate the fund’s potential risk exposure correctly. Many other studies agreed.

“You were determined to bolster the illusion that the devices worked and you knew there was a spurious science to produce that end,” Judge Hone said to Bolton. This brings to mind the actions of Philippe Jorion, Professor of Finance at the Graduate School of Management at the University of California at Irvine, who, by the winter of 2009 was already proclaiming that “VaR itself was not the culprit, however. Rather it was the way this risk management tool was employed.” He also helpfully pointed out that LTCM were very profitable in 2005 and 2006. He and others have been muddying the waters ever since.

“They had a random detection rate. They were useless.” concluded Judge Hone. Whereas VaR had a protective effect only within what were regarded as “possible” market environments, ie something similar to what had been seen before during relatively calm market conditions. In fact, VaR became less helpful the more people adopted it, as everyone using it ended up with similar trading positions, which they then attempted to exit at the same time. This meant that buyers could not be found when they were needed and the positions of the hapless VaR customers tanked even further.

Gary Bolton’s jurors concluded that, if you sell people a box that tells them they are safe when they are not, it is morally reprehensible. I think I agree with them.

Earthquakes and Equities

10. August 2013 · Write a comment · Categories: Economics, Prediction

Plotting the frequency of earthquakes higher than a given magnitude on a logarithmic scale gives a straightish line that suggests we might expect a 9.2 earthquake every 100 years or so somewhere in the world and a 9.3 or 9.4 every 200 years or so (the Tohoku earthquake which led to the Fukushima disaster was 9.0). Such a distribution is known as a power-law distribution, which gives more room for action at the extreme ends than the more familiar bell-shaped normal distribution, which gives much lower probabilities for extreme events.

Similarly, plotting the annual frequency of one day falls in the FTSE All Share index higher than a given percentage on a logarithmic scale also (as you can see below) gives a straightish line, indicating that equity movements may also follow a power-law distribution, rather than the normal distribution (or log normal, where the logarithms are assumed to have a normal distribution) they are often modelled with.
However the similarity ends there, because of course earthquakes normally do most of their damage in one place and on the one day, rather than in the subsequent aftershocks (although there have been exceptions to this: in The Signal and the Noise, Nate Silver cites a series of earthquakes on the Missouri-Tennessee border between December 1811 and February 1812 of magnitude 8.2, 8.2, 8.1 and 8.3 respectively). On the other hand, large equity market falls often form part of a sustained trend (eg the FTSE All Share lost 49% of its value between 11 June 2007 and 2 March 2009) with regional if not global impacts, which is why insurers and other financial institutions which regularly carry out stress testing on their financial positions tend to concern themselves with longer term falls in markets, often focusing on annual movements.

How you measure it obviously depends on the data you have. My dataset on earthquakes spans nearly 50 years, whereas my dataset for one day equity falls only starts on 31 December 1984, which was the earliest date from which I could easily get daily closing prices. However, as the Institute of Actuaries’ Benchmarking Stochastic Models Working Party report on Modelling Extreme Market Events pointed out in 2008, the worst one-year stock market loss in UK recorded history was from the end of November 1973 to the end of November 1974, when the UK market (measured on a total return basis) fell by 54%. So, if you were using 50 years of one year falls rather than 28.5 years of one day falls, a fall of 54% then became a 1 in 50 year event, but it would become a 1 in 1,000 year event if you had the whole millennium of data.

On the other hand, if your dataset is 38 years or less (like mine) it doesn’t include a 54% annual fall at all. Does this mean that you should try and get the largest dataset you can when deciding on where your risks are? After all, Big Data is what you need. The more data you base your assumptions on the better, right?

Well not necessarily. As we can already see from the November 1973 example, a lot of data where nothing very much happens may swamp the data from the important moments in a dataset. For instance, if I exclude the 12 biggest one day movements (positive and negative) from my 28.5 year dataset, I get a FTSE All Share closing price on the 18 July 2013 of 4,494 rather than 3,513, ie 28% higher.

Also, using more data only makes sense if that data is all describing the same thing. But what if the market has fundamentally changed in the last 5 years? What if the market is changing all the time and no two time periods are really comparable? If you believe this you should probably only use the most recent data, because the annual frequency of one day falls of all percentages appears to be on the rise. For one day falls of at least 2%, the annual frequency from the last 5 years is over twice that for the whole 28.5 year dataset (see graph above). For one day falls of at least 5%, the last 5 years have three times the annual frequency of the whole dataset. The number of instances of one day falls over 5.3% drop off sharply so it becomes more difficult to draw comparisons at the extreme end, but the slope of the 5 year data does appear to be significantly less steep than for the other datasets, ie expected frequencies of one day falls at the higher levels would also be considerably higher based on the most recent data.

Do the last 5 years represent a permanent change to markets or are they an anomaly? There are continual changes to the ways markets operate which might suggest that the markets we have now may be different in some fundamental way. One such change is the growth of the use of models that take an average return figure and an assumption about volatility and from there construct a whole probability distribution (disturbingly frequently the normal or log normal distribution) of returns to guide decisions. Use of these models has led to much more confidence in predictions than in past times (after all, the print outs from these models don’t look like the fingers in the air they actually are) and much riskier behaviour as a result (particularly, as Pablo Triana shows in his book Lecturing Birds on Flying, when traders are not using the models institutional investors assume they are in determining asset prices). Riskier behaviour with respect to how much capital to set aside and how much can be safely borrowed for instance, all due to too much confidence in our models and the Big Data they work off.

Because that is what has really changed. Ultimately markets are just places where we human beings buy and sell things, and we probably haven’t evolved all that much since the first piece of flint or obsidian was traded in the stone age. But our misplaced confidence in our ability to model and predict the behaviour of markets is very much a modern phenomenon.

Just turning the handle on your Big Data will not tell you how big the risks you know about are. And of course it will tell you nothing at all about the risks you don’t yet know about. So venture carefully in the financial landscape. A lot of that map you have in front of you is make-believe.

Needles and Pins 31 July 2013

31. July 2013 · Write a comment · Categories: Actuarial, Cartoons, Demographics, Economics, Pensions, Prediction

Uncertainty and its discontents

20. June 2013 · Write a comment · Categories: Economics, Prediction, Regulation

There’s certainly a great deal of uncertainty about.

In Nate Silver’s book, The Signal and the Noise, there is a chapter on climate change (which has come in for some criticism – see Michael Mann’s blog on this) which contains a diagram on uncertainty supposedly sketched for him on a cocktail napkin by Gavin Schmidt. It occurred to me that this pattern of uncertainty at different timescales was more generally applicable (it describes very well, for instance, the different types of uncertainty in any projections of future mortality rates). In particular, I think it provides a good framework for considering the current arguments about economic growth, debt and austerity. Some of these arguments look to be at cross-purposes because they are focused on different timeframes.

In the short term, looking less than 10 years ahead, initial condition uncertainty dominates. This means that in the short term we do not really understand what is currently going on (all our knowledge is to some extent historical) and trends which might seem obvious in a few years are anything but now. Politics operates in this sphere (long term thinking tends to look two parliaments ahead at most, ie 10 years). However, the market traders who by their activities move the markets and market indices on which we tend to base our forecasts and our economic policies are also working in the short term, the very short term (ie less than 3 months to close off a position and be able to compare your performance with your peers), even if they are trading in long term investments.

So both the politics and economics is very short term in its focus, and this is therefore where the debate about growth and austerity tends to be waged. The Austerians (which include the UK Government) claim to believe that debt deters growth, and that cutting spending in real terms is the only possible Plan A policy option. The Keynesians believe that, in a recession, and when interest rates cannot go any lower, demand can only be revived by Government spending. This argument is now well rehearsed, and is in my view shifting towards the Keynesians, but in the meantime austerian policies (with all the economic destruction they inevitably cause) continue in the UK.

However, there are other groups seemingly supportive of the UK Government’s position in this argument for altogether different reasons. Nassim Nicholas Taleb argues that high levels of debt increase an economy’s fragility to the inevitable large devastating economic events which will happen in the future and which we cannot predict in advance. He therefore dismisses the Keynesians as fragilistas, ie people who transfer more fragility onto the rest of us by their influence on policy. These concerns are focused on the structural uncertainty which is always with us and is difficult to reduce significantly. It is therefore important to reduce (or, if possible, reverse) your fragility to it.

At the longer term end are the groups who believe that we need to restrict future economic growth voluntarily before it is done for us, catastrophically rapidly, by a planet whose limits in many areas may now be very close to being reached. They are therefore implacably opposed to any policy which aims to promote economic growth. These concerns are focused where there are many possible future scenarios (ie scenario uncertainty), some of which involve apocalyptic levels of resource depletion and land degradation.

These different groups are tackling different problems. I do not believe that those concerned with the structural fragility of the economy really believe that the people paying for the restructure should be the disabled or those on minimum wage. Similarly, there is a big difference between encouraging people to consume less and move to more sustainable lifestyles and recalibrating down what is meant by a subsistence level of existence for those already there.

We do need to worry about too big to fail. Our economy houses too many institutions which appear to be too large to regulate effectively. We do need to reduce levels of debt when economic activity has returned to more normal levels. We will need to restructure our economy entirely for it to make long-term sense in a world where long term limits to growth seem inevitable. But none of these are our immediate concern. We need to save the economy first.

Skin in the right game

11. June 2013 · Write a comment · Categories: Actuarial, Economics, Prediction, Regulation

In my previous post, I looked at some of the reasons why we are so useless at making economic predictions, and some of the ideas for what might be done about it. One of the key problems, raised by Nassim Nicholas Taleb most recently in his book Antifragile, is the absence of skin in the game, ie forecasters having something to lose if their forecasts are wildly off.

But what if all forecasters had to have something to lose before they were allowed to make forecasts? What if every IMF or OBR forecast came with a bill if it was seriously adrift? What if you knew whenever you read a forecast in a newspaper or on a television screen that the person making that forecast had invested something in their belief in their own forecast?

Betting on events dates back at least to the 16^th century, but prediction markets (also known as predictive markets, information markets, decision markets, idea futures, event derivatives, or virtual markets) have developed most strongly over the last 25 years or so (the University of Iowa launched its first electronic market as an experiment in 1988). Intrade, which describes itself as the world’s leading prediction market, is now a little smaller than it was following the news just before Christmas last year that it would no longer let Americans trade on its site. It had been sued by the US regulator of the commodities derivative markets for breaking a commitment not to allow trading on the constituents of those markets.

A paper on prediction markets earlier this year called Prediction Markets for Economic Forecasting by Snowberg, Wolfers and Zitzewitz suggests there are 3 main types:

· Winner takes all. If the event you have bet on happens you win. If not, you lose your stake. Intrade is this type of prediction market: you pay a proportion of £10 a share based on the average probability of the event happening (according to the market participants) and get £10 back if it happens, nothing if it doesn’t. The Iowa Electronic Market current offerings, on congressional elections and US federal monetary policy, are also winner takes all. The price of the bet at any time should reflect the market’s view of the probability of the event happening.

· Index. The amount paid out is unknown but tied to the variable you are betting on, eg the number of seats won by a party in a particular parliamentary election, or the value of an index at close of business on a particular date. The price of the bet at any time should reflect the market’s view of the expected value of the outcome.

· Spread betting. Most commonly found on sports betting sites, the amount bet and the amount paid out are fixed, but the event that leads to a pay out (eg number of goals scored in a match more than x) changes until the numbers of buys and sells match (a “buy” in this example is betting the number of goals will be above x, a “sell” is betting the number of goals will be below y, a number less than x. The spread, on which the betting site makes its money, is the difference between x and y. This could equally be applied to the values of an index at a particular date (eg Spreadex offer just such bets on several major share indices as well as currency exchange rates). Depending on the relationship between the pay out and the bet, the value of the spread points at any time should reflect the market’s view of a particular point in the probability distribution of the event, eg if the pay out is twice the bet, this would be the median (ie a 50% chance of the outcome being higher or lower).

As we saw in my previous post, currently economic predictions are largely blown off course by either:

Over-confidence in the information used to make them; or
The difficulty in standing out against a market which is making everyone a lot of money (buying has limited downside, selling limited upside); or (another possibility I haven’t mentioned before)
Bias in individual “expert” judgements, eg those with reputations at stake may want to keep their assumptions somewhere in the middle of the pack rather than standing out most of the time as this is less risky (hence the obsession with “benchmarking” assumptions in the actuarial world for instance).

Prediction markets can help with all of these problems:

Having to bet on your opinion should cause you to weigh the evidence backing it more carefully. Also, once a market is established and attracting a lot of bets, the range of evidence on which opinions are being based should expand. Prediction markets also appear to be quite difficult to manipulate or arbitrage.
Prediction markets can respond very rapidly to changes in the information available. As an example, within 25 minutes of Donald Rumsfeld’s former chief of staff tweeting about the death of Osama Bin Laden, the market view of the probability of this event on a prediction market rose from 7% to 99%.
Betting can take place anonymously. So, although the betting site knows who you are, no one else does, and the data from the voting therefore gets out into the public domain without any individual being accused of talking down a market or risking their reputation.

For these reasons, amongst others, forecast accuracy for established prediction markets might be expected to outperform that of professional forecasters. The paper of Wolfers et al suggests that this is the case.

There are still problems. The markets need to be popular to be much use for predictions, so the questions need to be interesting enough for mass participation. Secondly, a market could theoretically be undermined (although not necessarily to the detriment of its predictive ability) by traders with inside information. However, there are quite a few safeguards in place against this type of activity. Intrade, for instance, requires a photo ID and two proofs of address before it registers anyone to trade on their site. And Spreadex are regulated by the Financial Conduct Authority. A third problem is referred to as the “longshot bias”, which is observed on all types of betting markets. People tend to over-bet on events with long odds against them and under-bet on events which are fairly likely (which explains the narrowing of the odds as the starting gun approached on that horse you bet on just because of its name in the Grand National a couple of months ago). This is a problem of the winner takes all type of market, seemingly related to behavioural factors around the difference between how we view winning and losing, and it is difficult to see how it could be avoided completely. Care may need to be taken therefore when interpreting prediction markets on events which are seen as having fairly low probabilities.

But overall, prediction markets would seem to offer a way of significantly improving economic predictions, so why not make them compulsory for people who want to make such forecasts? By putting a cost on such predictions (a minimum bet could be set based on the size of the organisation making it), it would remove the casual forecasting we currently see too much of, and encourage people to review their beliefs rather more carefully. It would also ensure that the markets were popular enough to be effective. It may be that economic forecasting will always be far from perfect, but this seems a good place to start if we want, in Nate Silver’s words, to be “less and less and less wrong”.

A tax on economists? Not at all. But it might mean that we all have skin in the right game.

Skin in the wrong game

10. June 2013 · Write a comment · Categories: Economics, Prediction

The main reason I set up this website was to draw more attention to the fact that we are fairly useless at making economic predictions. The graph I use as a logo is just one of many comparisons of economic projections with reality which could be shown to demonstrate just how useless. The problem is that there is then a bit of a void when we are looking for ways to support economic decision making. Nassim Nicholas Taleb and Nate Silver are, in different ways, exploring ways to fill this void.

Taleb, in his book Antifragile, links the fragility of the economic system, amongst other things, to an absence of “skin in the game”, ie something to lose if things do not go to plan. Something is fragile if it is vulnerable to volatility, robust if it is relatively unaffected by it, and antifragile if it profits from volatility. If decision makers gain an upside when things go right, but are effectively bailed out from the downside when they don’t by others who then suffer the downside instead, they will have no real incentive to be sufficiently careful about the decisions they take. This makes the chance of a downside for everyone else consequently greater. Taleb refers to people in this position, championing risky strategies (or, more often, strategies where the risks are not really known) without facing the risks personally, as “fragilistas”. He suggests instead a system of “nonpredictive decision making under uncertainty” on the basis that “it is far easier to figure out if something is fragile than to predict the occurrence of an event that may harm it”.

Silver, in his book The Signal and the Noise, suggests the main reason why economic predictions are routinely so awful, even when the forecasters are trying to be accurate rather than making a splash, is that the events forecasters are considering are often out of sample, ie the historical data they are considering to make their forecasts do not include the circumstances currently being faced, but that the forecasters are confident enough to make predictions despite this. This explains, for instance, the failure in the US to predict the housing crash (there had never been such a boom before), the failure of the ratings agencies to understand how risky mortgage-backed securities were (they were in new more complex forms) and the failure to predict that the housing market would take the rest of the economy down with it (much more trading betting on house price rises that did not materialise than had ever been seen before).

Silver cites the economist Terence Odean of the University of California at Berkeley, whose paper Do Investors Trade Too Much shows that equity traders trade excessively in the sense that their returns are, on average, reduced through trading (due to transaction costs), and suggests that this is partly due to overconfidence in the constant stream of information in the media drawing their attention from one possible model to another. This effect can be modelled to show that markets behave irrationally in the presence of overconfident decision making, even when decision-makers are otherwise completely rational.

However, when we are looking at the calls made by traders there are other forces at work that make things even worse. Silver’s book advances the theory that it is not so much an absence of skin in the game that leads traders to continue betting on rising markets when bubbles have started to develop, but skin in the wrong game (at least from the point of view of the rest of us fragile people). He gives the example of a trader looking a year ahead to make a call on whether the market will crash or not. If he buys and the market rises everyone is happy. If he buys and the market crashes, he will be in the same boat as everyone else and will probably keep his job if not his bonus. If he sells and the market crashes he will be seen as a genius, but the tangible rewards of this beyond his current position might not be life changing. However, if he sells and the market rises he may never work in the industry again.

For a trader who wants to keep his job and remain popular with his colleagues, selling therefore has limited upside and unlimited downside, to analyse it in a Talebian way. Buying on the other hand has potentially unlimited upside while the music is playing and limited downside when it stops. So these traders do have skin in the game, which keeps them buying even when they can see the iceberg approaching, and does not particularly reward accuracy in forecasting. It’s just a different game to the one the rest of us are playing.

For these amongst many other reasons, economic forecasting (Silver differentiates forecasting – meaning planning under conditions of uncertainty – from predicting – meaning calling the future correctly) is unlikely ever to be very accurate. But whereas Taleb believes that planning for the future must be nonpredictive to avoid making suckers of us all, Silver believes there may be some scope for improvement. This brings us to the idea of prediction markets and their ability to introduce some of the right skin in the right game, which I will discuss in my next post.

We Know Zero