Trust me. I'M AN ACTUARY!

Trust me. I’M AN ACTUARY!

I commented on the Pensions Regulator’s new code of funding in a recent post. The reason I am returning to it so soon is that a good friend of mine has pointed out a rather important, but subtle, aspect of the new code which I had missed. It goes to the heart of what we should expect from a professional in any field.

Experts and the Problem of P2C2Es

In 1990, while still in hiding from would-be assassins keen to implement Ayatollah Khomeini’s fatwa, Salman Rushdie wrote a book for his son called Haroun and the Sea of Stories. This introduced the idea of P2C2Es or Processes Too Complicated To Explain. These were how awkward things, like the fact that the Earth had a second moon which held the source of all the world’s stories, were kept hidden from ordinary people. All the people who worked on P2C2Es were employed at P2C2E House in Gup City under a Grand Comptroller. When I read it to my son a few years later I enjoyed the story of very clever people conspiring against the general public as a fairy tale.

Since 2008, it has become increasingly clear that this is no fairy tale. Whether you are looking for the cheapest quote for insuring your life, house or your car; a medical opinion about your health; an investment that meets your needs: it is a P2C2E.

Malcolm Gladwell and others make the case that expert failure is what we should really fear, when important things rely on experts not making mistakes doing things that most people do not understand. The inability to challenge expert opinion has cost us all a lot of money in the last few years. We should stay clear of P2C2Es whenever we can in my view. Professionals should present evidence and the intuitions gained from their experience, but leave the decisions to people with skin in the game.

Other professionals disagree with this. There is, from time to time, a push to get rid of juries in cases where the evidence is thought too complicated (eg fraud) or too dangerous to make public in even a limited way (eg terrorism). Some of these succeed, others don’t. There are also frequent political arguments about what we should have a referendum on, from Scottish independence (got one if you’re Scottish) to membership of the EU (one is promised) to recalling your MP mid-term (so far no luck on this one).

There is a similar divergence of opinion amongst actuaries. Since the Pensions Regulator’s first code of practice for funding was launched, in 2006, the scheme actuary’s role has been clearly set out as one of adviser to the scheme trustees and not, other than in the rare cases it was cemented in the scheme rules, a decision maker. However there are actuaries who look back wistfully to the days when they effectively set the funding target for pension schemes and all parties deferred to their expertise. I am not one of them.

Because this was really no good at all if you were a trustee expected to take responsibility for a process you were never really let in on. The arrival at a contribution rate or a funding deficit for a scheme funding on a basis presented to them as a fait accompli was to many trustees a P2C2E. We risk returning to those days with the new code of funding.

What this has to do with pensions

Compare the wording of the new Code of Practice for pension scheme funding with the previous one:

2006 code

The actuary is not passing an opinion on the trustees’ choice of method and assumptions.

2014 code

Trustees should have good reasons if they decide not to follow the actuary’s advice. They should recognise that if they instruct their actuary to certify the technical provisions and/or schedule of contributions using an approach which the actuary considers would be a failure to comply with Part 3, the actuary would have to report that certification to the regulator as the regulator considers such certification to be materially significant.

Where Part 3 refers to the funding regulations for actuarial valuations. Previously actuaries who were unable to provide the required certification of the calculation of the technical provisions or of the adequacy of the schedule of contributions had to report the matter to the regulator only if a proper process had not been followed or the recovery plan didn’t add up to the deficit. It was thought that going any further would involve passing an opinion on the trustees’ choice of method and assumptions.

Will the new code make schemes better funded? In some cases perhaps, but at the cost of moving scheme trustees into a more passive role where they do not feel the same level of responsibility for the final outcome. It is the difference between roads where cars are driven by people concerned with road safety and the ones we have where drivers are primarily concerned with not setting off speed cameras. The general level of safety is reduced in both cases, with the further danger that this passivity will trickle into other areas of trustee responsibility. And the risk to the schemes of the group think of scheme actuaries (a relatively small group of professionals who tend to all cluster around the same schedule of continuing professional development (CPD) events) is massively increased.

Ha-Joon Chang famously said never trust an economist. Is it any less dangerous to trust an actuary under these circumstances?

 

 

 

Tweet about this on TwitterShare on FacebookShare on Google+Share on LinkedInPin on PinterestShare on TumblrEmail this to someone

I have written about false positives before, as have many others, but the media stories keep on coming. The latest reports, by the BBC most notably but also carried by the New Scientist and a number of other sources, display such an unfortunate ignorance of the issue of false positives that they risk raising false fears in the minds of sufferers of mild cognitive impairment and their families.

The BBC article makes a number of statements:

1. British scientists have made a “major step forward” in developing a blood test to predict the onset of Alzheimer’s disease.

2. Research in more than 1,000 people has identified a set of proteins in the blood which can predict the start of the dementia with 87% accuracy.

Neither of these statements are true. What the new test can do is guess right about patients with mild cognitive impairment (MCI) going on to develop Alzheimer’s disease (AD) within a year in 87% of cases. What it cannot do is predict whether someone with MCI will develop AD with 87% accuracy. The research article is here.

It all comes down to the likelihood of someone with MCI developing AD. It does not help that there is no general agreement of a definition for either term. The research reported on used the Petersen criterion for MCI. The likelihood of people within the population with MCI developing AD each year is again unknown, but 6 quite small studies (a total of 476 people across all the studies, with different average ages and sample sizes among the groups) carried out in North America analysed by Petersen et al in 2001 suggested that it lay between 6% and 25%, with an overall average rate of 12.5%.

So let’s assume that it is 12.5%. In a population of 1,000 people with MCI, we would expect 125 to develop AD. The test will identify 109 of them on average. Unfortunately, assuming the same accuracy rate of 87%, it will also give false positives for 13% (or 114) of the 875 not expected to develop AD in any particular year. That means that, of the 223 who test positive, less than half (49%) are actually expected to develop AD within a year. As the NHS choices website points out, in one of the few sceptical articles I could find, this is no better than tossing a coin.

If we assume the probability of moving from MCI to AD is at the higher bound of 25%, the positive predictive value (or PPV, as it is known) increases to 69%. However if we assume the lower bound of 6%, the PPV falls to 30%. In other words, if we get a positive blood test for this panel of 10 proteins, we currently do not know whether it is twice as likely to be a true positive than a false positive, or twice as likely to be false as true. Or anywhere in between.

Despite this, Dr Eric Karran, director of research at Alzheimer’s Research UK (clearly no conflict of interest there in promoting stories which promote the idea that Alzheimer’s research is highly effective) is widely reported as describing the study as a “technical tour de force”, while also acknowledging that the current accuracy levels risked telling many healthy people they were on course to develop Alzheimer’s.

In some reports it was pointed out that it was unlikely that the test would be used in isolation if it eventually made its way into clinics. A positive result could be backed up by brain scans or testing spinal fluid for signs of Alzheimer’s, they said. However if the test is no more predictive than a coin toss that is hardly encouraging.

There was more from Dr Karran: “This gives a better way to identify people who will progress to Alzheimer’s disease, people who can be entered into clinical trials earlier, I think that will increase the potential of a positive drug effect and thereby I think we will get to a therapy, which will be an absolute breakthrough if we can get there.”

This is simply untrue. Clearly it is important to support research into therapies for Alzheimer’s disease, but in raising funds for this the ends do not justify any means. Additional funding gained through false claims for any particular discovery will come at the expense of funding in other equally important areas. Like agreeing a definition of MCI and AD for instance, or better data on the transition probabilities from MCI to AD at different ages, without which a lot of the more laboratory-based research will be a waste of time as it will be unclear how best to apply it.

So let’s be careful how we report these things. People’s hopes and fears are at stake.

Tweet about this on TwitterShare on FacebookShare on Google+Share on LinkedInPin on PinterestShare on TumblrEmail this to someone

Another month, another consultation. This time it’s the Pension Protection Fund’s (PPF) turn. I last wrote about their plans five months ago. Since those dark days things seem to have moved on a bit: there is now a proposed model and a timetable for implementation.

And there is much to cheer here. One of the main criticisms consistently levelled at the current system was that it was hard for employers to understand how to improve their score, without handing over money to Dun & Bradstreet (D&B) for reports and fees to advisers to interpret them. Here, at last, is a model which is not owned by the credit agency running it, something I have long argued for. This means that the scores and data underlying them can be monitored by companies much more easily, and in more detail, by a free web-based portal.

Unfortunately the PPF are risking undermining this transparency for large companies by considering a credit rating override, where the insolvency risk would be determined by the company’s credit rating score instead. In my view this idea should be resisted.

Other successes are the moves to stop ABCs from getting too much credit for their complex structures, and the use of past data to review the treatment of Type A contingent assets (although they have chickened out of removing these altogether) and the last man standing levy reduction.

In all there were nine success criteria which were used to make the decision on the model used, but the one given the greatest weighting was “predictiveness”. According to the Oxford English Dictionary this word does not exist, but I take it to mean “degree to which insolvency risk assessed predicts the number of actual insolvencies for a given score”. Of course, it is nothing of the kind that has been assessed. They have taken the last eight years of data and compared the proportion at each score level with the percentage of insolvencies expected (they say “predicted”), and wrapped up the differences in an eye-catching diagram using the Gini coefficient (this is usually used to talk about inequality, when you are looking to minimise it, but here they are trying to allocate levies where the risk lies and therefore trying to maximise the distance from an even distribution).

PPF levy GiniAll a high Gini score means in this context is that the selected model fits well with the actual insolvencies over the last eight years. The danger is that the model has been over-fitted to eight years’ data, a rather untypical period for the economy in many ways and possibly not very indicative for what lies ahead until 2030 (when the levy is supposed to end). Fortunately they are proposing to continue monitoring how well the “predictiveness” works in future.

The other area of the consultation where I take issue is the PPF’s opposition to having a transition period. Their impact assessment shows that 10% of schemes are expected to see an increase of over £50,000 in their levy as a result of these changes, with 200 of them seeing an increase of over £200,000. It therefore seems odd that they should oppose a transition period to allow companies to better cope with the long term move to a fairer allocation of levies. The main argument they give for this is that it would be a cross subsidy. But so is the restriction on the increase in levy by moving down a band to 60%, which I can see much less justification for and which results in bands 2 and 3 underpaying for their insolvency risk and bands 5 and 6 overpaying for it.

But overall a broad welcome, as I will be telling them. Let’s see what survives the consultation (it ends at 5pm on 9 July).

My consultation responses are as follows:

Chapter 2

1. Do you agree that we should seek to maintain stability in the overall methodology for the levy, only making changes where there is evidence to support them?

Yes.

Chapter 3

2. Do you consider that the definition of the variables in the scorecards is sufficiently precise to provide for consistent treatment?

Yes.

3. Do you agree that it is appropriate to re-evaluate the model to ensure that it remains predictive?

Yes.

4. Do you have comments on the design of the “core model” developed by Experian?

Very pleased that the PPF have decided to move away from a proprietary model, where large parts of its operation are kept secret through commercial confidentiality arguments.

5. Do you agree with the success criteria set out by the Industry Steering Group and that the PPF-specific model developed by Experian is a better match with them than Commercial Delphi?

Yes.

6. Do you agree that it is appropriate to use the separate scorecard developed by Experian not-for-profit entities, even though this requires an extension of the data set used to generate the scorecard?

Yes.

7. Do you have comments on the approach to the rating and proposed identification of not-for profit entities, developed by Experian?

No.

8. Are there other public sources of data that Experian should consider extending coverage to?

No.

9. Do you agree with the proposed data hierarchy?

Yes.

Chapter 4

10. Do you favour a credit rating over-ride?

No. This would undermine the gain in transparency offered by the PPF-specific model.

Chapter 5

11. Do you agree with our proposed aims for setting levy rates?

I am concerned about the cross subsidy implicit in the 60% limit on levy differences between adjacent bands.

12. Do you agree it is appropriate to divide the entities with the best insolvency probabilities in to a number of bands, to ensure that the cliff-edges between subsequent bands are limited, or do you favour a broad top band?

Cliff edges are unavoidable with this model. I think there is a strong argument for having slightly fewer slightly bigger ones. This would remove many of the small band movements at the top end, which are relatively unproductive for risk management.

13. Do you agree with the proposed 10 levy bands and rates?

Not completely. Bands 2 and 3 appear to be underpaying for their insolvency risk, and bands 5 and 6 appear to be overpaying.

14. Do you agree that for 2015/16 levy year insolvency probabilities are averaged from 31 October 2014 to 31 March 2015?

Yes.

Chapter 7

15. Do you support transitional protection for those most affected by the move to the new methodology, recovered through the scheme-based levy?

Yes.

Chapter 9

16. Do you agree that the appropriate route to reflecting ABC’s in the levy is to value them based on the lower of the value of the underlying asset (on employer insolvency) after stressing or the net present value of future cashflows?

Yes. I do not accept that ABCs’ primary objective is to reduce risk. The changes proposed appear to ensure that they do not get overly favourable treatment in terms of levy reduction.

17. Do you agree that a credit should only be allowed where the underlying assets for the ABC is UK property? Do you have any comments on the example voluntary form/required confirmations?

Yes.

18. Do you support the proposal to make the certification of contingent assets more transparent, through requiring certification of a fixed amount which the guarantor could pay if called upon?

Yes.

19. Do you have any comments on the proposed revised wording for trustee certification for Type A contingent assets?

The revised wording seems appropriate.

20. Do you agree with our proposals to adjust guarantor scores to reflect the value of the guarantee they are potentially liable for? Do you favour the adjustment being achieved by a factor being applied to the guarantor’s Pension Protection Score or by an adjustment of the guarantor’s levy band?

This looks like a very complicated approach designed to put off sufficient schemes from using Type A contingent assets so that there will not be a very large squeal when they are removed altogether.

21. What other measures do you suggest to ensure that, where a scheme certifies information about a contingent asset to the PPF, any resulting levy reduction is proportionate to the actual reduction in risk?

I think the proposals are complicated enough.

22. Do you agree with the proposed form of confirmation when Last Man Standing scheme structure is selected on Exchange?

Yes.

23. Do you agree with the revised scheme structure factor calculation proposed for associated last man standing schemes?

Yes.

Tweet about this on TwitterShare on FacebookShare on Google+Share on LinkedInPin on PinterestShare on TumblrEmail this to someone

I have been reading Ha-Joon Chang’s excellent book Economics: The User’s Guide after listening to him summarising its thrust at this year’s Hay-on-Wye Festival of Literature and the Arts. It is very disarming to meet an economist who immediately tells you never to trust an economist, and I will probably return to his thoughts on the limitations of expert judgement in a future article.

But today I want to focus on his summary of the major schools of thought in economics, and what the implications might be for actuaries. Chang’s approach is that he does not completely subscribe to any particular school but does not reject any either. He bemoans what he sees as the total domination of all economic discussion currently (and therefore also all political discussion about running the economy) by neoclassical economists. I think actuarial discussion may suffer from a similar problem.

So what is neoclassical economics? Well it has become almost invisible to us due to its omnipresence, in the way fish don’t see the water they swim in, but its assumptions may surprise you. It assumes that all economic decisions are at an individual level, with each individual seeking to maximise what is known as their utility (ie things and experiences they value). The idea is that we self-interested individuals will collectively make decisions which, within the competitive markets we have set up, result in a socially better outcome than trying to plan everything. This approach has become a very conservative outlook (ie interested in preserving the status quo) in Chang’s view ever since it was further developed to include the Pareto principle in the early 20th century, which says that no change in economic organisation should take place unless no one is made worse off. This limits the scope for redistribution within a society, which can lead to the levels of inequality we see now in parts of the developed world which many are becoming increasingly concerned about, Thomas Piketty included.

Arguments between neoclassical economists in Chang’s view tend to be restricted to ones about how well the market actually works. The market failure argument says that there is a role to play for governments in using taxes and regulations (negative externalities) or in funding particular things like research (positive externalities) to mitigate the impacts of markets, particularly in areas where market prices do not fully reflect the social cost of particular activities (eg pollution on the environment). Another criticism made of neoclassical economics is that it does not allow properly for the fact that buyers and sellers do not have the same level of information available to them in many markets, and therefore the price struck is often not the one which would lead to the best outcome for society as a whole. So the more “left wing” neoclassicalism requires more market regulation to protect consumers and the environment they live in.

The more “right wing” neoclassical response to this is that people actually do know what they are doing, and even build in the likelihood that they are being conned due to asymmetric information in the decisions they make. The government should therefore reduce regulation and generally get out of the way of wealth-creating business. This form of neoclassicalism views the risk of government failure as much greater than that of market failure, ie even if we have market failure, the costs of government mistakes will inevitably be much greater.

And if you draw a line between those two forms of neoclassicalism, somewhere along that line you will find all of the main UK political parties and pretty much all economic discussion within the financial services industry.

And, on the whole, it tends to circumscribe the role that actuaries play in the UK.

One of the major drawbacks of neoclassical theory is that is assumes risks can be fully quantified if we only have a comprehensive enough model. Actuaries are predominantly hierarchists, who believe that they can manage the inequalities which flow from neoclassical theory via collectivist approaches, like insurance policies and pension schemes, and protect individuals and indeed whole financial systems from risk. Since Nicholas Nassim Taleb and others made so much money from realising that this was not the case in 2008, this has probably been neoclassicalism’s most obvious flaw, and the one which has given rise to the most discussion (although possibly not so much change to practice) amongst actuaries.

But there are others. Neoclassicalism assumes that individuals are selfish and rational, both of which have been persuasively called into question by the work of Kahneman and others, who have shown that we are only rational within bounds and make most of our decisions through “heuristics” or rules of thumb. Actuaries have tried to reflect these views, some of which were originally developed by Herbert Simon in the 40s and 50s, particularly in the way that information is communicated (eg the recent publication from the Defined Ambition working group), but have very much stayed at the microeconomic level (very much, according to Chang, like much of the Behaviouralist School themselves) rather than exploring the implications of this theory at a macroeconomic level.

Neoclassical theory is also much more focused on consumption than production, with its endless focus on markets of consumers. One alternative approach is that proposed by the Neo-Schumpeterian School, which rightly points out that, in many markets, technological innovation is considerably more important than price competition for economic development. The life-cycle of the iphone, from innovation to temporary market monopoly to the creation of a totally new market in android phones is a case in point. Actuaries have done relatively little work with technology firms.

Another school of economic thought which is much more focused on production is the Developmentalist Tradition, which believes governments can improve outcomes considerably by intervening in how economies operate: from promoting industries which are particularly well-linked to other industries; to the protection of industries which develop the productive capability of the economy, particularly infant industries which might get smothered at birth by the more established players in the market. This tradition clearly believes that the risk of government failure is less than the potential benefits of intervention. The failure of productivity to pick up in the UK since 2008 has been described as a “puzzle” by the Bank of England and other financial commentators. Perhaps some clues might lie outside a neoclassical viewpoint.

The Institutionalists have looked at market transaction costs themselves, pointing out that these extend way beyond the costs of production, and could theoretically encompass all the costs of running the economic system within which the transactions take place, from the courts to the police to the educational and political institutions. They have suggested that this may be why so much economic activity does not take place in markets at all, but within firms. I think actuaries have started to engage with failures in pricing mechanisms recently, particularly where these have environmental consequences such as in the case of carbon pollution and the implications for the long term valuations of fossil fuel reserves on stock markets.

The Keynesians I have written about before. They are probably the most opposed to the current austerity policies, pointing out how, if a whole economy stops spending and starts saving when in debt, as an individual would, the economy will stay in recession longer and recovery (and therefore the possibility of significant deficit reduction) will be slower. The coalition government in the UK have neatly proved this point since 2010.

I could go on, about the Classical or Marxist Schools which have been largely discredited by historical developments over the last 200 years, but which still have useful analysis of aspects of economics, or the spontaneous order of the markets believed in by the Austrian School. However my point is that I think Chang is right to highlight that there is a wider range of economic ideas out there. Actuaries need to engage with them all.

Tweet about this on TwitterShare on FacebookShare on Google+Share on LinkedInPin on PinterestShare on TumblrEmail this to someone

The Pensions Regulator has finally released its response to its consultation on regulating defined benefit pension schemes along with the simultaneous release of the final new code on funding defined benefits, its latest annual funding statement and two new documents: the defined benefit regulatory strategy and the defined benefit funding, regulatory and enforcement policy. It’s a bit of a mixed bag.

I set out a critique of the draft proposals back in January. These boiled down to two main criticisms:

  • That the new system proposed was effectively a return to the one-size-fits all approach of the Minimum Funding Requirement, which had done so much to undermine responsible scheme funding by employers; and
  • That the focus on governance, reverse stress testing, covenant advice, etc, effectively smuggled in from EIOPA’s latest IORP Directive, was likely to be a problem for small schemes.

BFO RIP

So what has the Regulator’s response been to these criticisms? Well, on the one-size-fits-all approach which was proposed as the Balance Funding Objective (BFO), the response is comical:

  • They have changed the name of their funding objective. The BFO is now called the Funding Risk Indicator (FRI). It is otherwise unchanged. This is reminiscent of the Lenny Henry sketch at the time that Windscale was renamed as Sellafield: “In future, radiation will be referred to as magic moonbeams”.
  • They are going to keep all their risk indicators secret. I have set out below their response in full on this point.

We believe that there may be potentially significant benefits to be gained in using the FRI and publishing more detail on our risk indicators in terms of providing clarity around standards, especially for small schemes, driving consistency and providing a useful framework for evaluating impact. However, after careful consideration of the risks and benefits highlighted in consultation responses, we have concluded that we should develop further our approach to risk assessment over the next year, including our risk indicators, to make sure it is sufficiently robust to support our intended uses beyond using it, alongside our other risk indicators, to prioritise our engagement. We have decided, for the time being, not to publish in detail where we set our risk indicators (beyond a high level description) in the funding policy document or in the annual funding statement.

So how will this work? Will the Regulator display charts like this one each year?

TPR graphWill they then berate the schemes and their advisers who were so bad at guessing where their secret line was? Because be in no doubt, with the speed of the revolving door operating between the Regulator and the industry it regulates, these indicators will get out and then gradually get disseminated through the pensions industry, from the biggest consultancies (who can easily fund having their consultants on secondment to Brighton) downwards, just as the Regulator’s previous “secret” link between assessment of covenant strength and “expected” discount rate assumptions did.

And what about the problem with small schemes? This is, in my view, considerably better handled by the Regulator. However, it does all comes down to its idea of proportionality.

Proportionality

The response to the consultation states:

Many respondents were concerned that proportionality did not follow through consistently in the consultation code or it was not explained clearly how it could be applied in practice. In particular, some thought our expectations around the extent of the analysis required to assess the covenant seemed disproportionate. The concern was that it would be difficult and costly for small schemes to apply the code’s principles.

I was one of those respondents. It continues:

We have reviewed the drafting to ensure that proportionality is properly referenced and emphasised throughout. We are looking to develop additional guidance to support the final code and will consider whether the proportionality principle can be explained further through illustrative examples.

On covenant assessment, we had already made clear (under the ‘Working with advisers’ section) that trustees may chose not to commission independent covenant advice as long as they can satisfy
themselves that they are sufficiently equipped, independent and experienced to undertake the work to the appropriate standard. In the section on ‘Employer covenant considerations’, we have emphasised the need for a proportionate approach (for instance, in-depth analysis may not be necessary if the scheme is relatively small or there has been no material change in the covenant since the last review). We also stress that assessment should focus on the knowledge gaps and where value can be added. Finally, we have made clear that the scope of any covenant review will depend on the circumstances of the scheme and it is, therefore, not always necessary for trustees to consider all the factors listed in the code.

In addition the Regulator has dropped the size of employer and strength of covenant as factors for trustees to consider in deciding on what is proportionate for their schemes, realising, rightly in my view, that the absolute size of employer and strength of covenant are much less important than the relative size of employer to scheme and risks to the scheme from failures of covenant which are already mentioned.

This all seems sensible. I do, however, think they will struggle to go further in setting out what proportionality means, since the problem of defining it has bedevilled the Solvency 2 project from the beginning and has still not been fully resolved. The IORP Directive is no clearer in this respect. What the Regulator could do is make a clear distinction between schemes with less than 100 members and the rest in terms of their responsibilities under the Code, reflecting the fact that the IORP Directive does not apply to these schemes.

Small schemes and risk-based prioritisation

But perhaps they have. Concerns were raised in the consultation about considering the size of the scheme in deciding whether to subject that scheme to greater scrutiny. It was argued that smaller schemes tended to be less well administered and advised (presumably by advisers and administrators of larger schemes!), more risky than larger schemes and should receive greater regulatory scrutiny. Some also questioned the usefulness of education without what they felt was the same prospect of regulatory scrutiny. I admit that I was one of those expressing concern about a lack of scrutiny coupled with a much increased regulatory burden for small schemes before the Regulator’s latest concessions on proportionality.

In their response the Regulator defended its actions by stating that large schemes all other things being equal, are of greater concern to us as they have the greatest impact on members and risk to the system (90% of members and liabilities are concentrated in the 1,210 largest schemes). However they expect the same standards of the small schemes that they aren’t scrutinising so hard. Bearing in mind that the Regulator regulates scheme managers rather than members (and many of those small schemes have just as many trustees as the larger ones) I don’t think this is a very convincing defence, but it seems to be preferable to admitting that they are just regulating schemes that fall under the IORP Directive.

Next steps

So a big raspberry for the secret FRI and a qualified welcome for the changes on proportionality. The final code has now been laid in Parliament and is expected to come into force in the next few months, subject to the parliamentary process. So if you think that there is more than a little tweaking left to do to this legislation, you need to start lobbying now.

Tweet about this on TwitterShare on FacebookShare on Google+Share on LinkedInPin on PinterestShare on TumblrEmail this to someone

The consultation on the proposals for pensions announced in the Budget, and contained in yesterday’s Queen’s Speech, ends on 11 June. I have set out my response below. I hope that it will sufficiently incense one or two more people into making their views heard, before the chance disappears.

A.1
The government welcomes views on its proposed approach to reforming the pensions tax framework.

1 Should a statutory override be put in place to ensure that pension scheme rules do not prevent individuals from taking advantage of increased flexibility?

Yes. Otherwise you are just writing cheques to pensions lawyers.

2 How could the government design the new system such that it enables innovation in the retirement income market?

Reform preservation rules, the TPR code on funding, HMRC rules and the PPF levy framework so as not to penalise different arrangements across the defined ambition spectrum. Remove the annual allowance, controlling the level of tax relief offered through the lifetime allowance only (I got this the wrong way round in my first draft – the annual allowance assumes regular incomes, many people now have incomes which bounce up and down alarmingly from year to year. It is also ridiculously cumbersome to administer).

3 Do you agree that the age at which private pension wealth can be accessed should rise alongside the State Pension age?

No. There is already an issue around healthy life expectancy and the state pension age in some regions of the UK.

4 Should the change in the minimum pension age be applied to all pension schemes which qualify for tax relief?

Yes. The arrangements need simplification.

5 Should the minimum pension age be increased further, for example so that it is five years below State Pension age?

No (see answer to 3).

A.2
The government welcomes views on its proposed approach to supporting consumers in making retirement choices.

6 Is the prescription of standards enough to ensure the impartiality of guidance delivered by the pension provider? Should pension providers be required to outsource delivery of independent guidance to a trusted third party?

There needs to be more clarity about the charges which can be levied for guidance or if it is to be remunerated in some other way.

7 Should there be any difference between the requirements to offer guidance placed on contract-based pension providers and trust-based pension schemes?

No. In most cases the scheme members have not chosen to receive lower levels of service.

8 What more can be done to ensure that guidance is available at key decision points during retirement?

I think there needs to be a right (but not requirement) to it for everyone at 50, 60, 70 and 80 as a minimum, at an agreed national nominal charge. I imagine that the £20 million available to develop resources for this will need to be increased significantly to make an impact on the quality of guidance materials provided.

A.3
The government would welcome views on the options outlined in point 5.15, including their likely complexity, and the burdens they might place on scheme sponsors and HMRC.

9 Should the government continue to allow private sector defined benefit to defined contribution transfers and if so, in which circumstances?

Yes. In all circumstances.

10 How should the government assess the risks associated with allowing private sector defined benefit schemes to transfer to defined contribution under the proposed tax system?

The reasons the Government have advanced for the changes to DC are equally compelling when applied to DB:

1. There is a lack of choice for people at retirement, which has become more of an urgent concern now that auto enrolment is boosting DC membership. This is even more the case for DB members who are already numerous (although getting less so daily), as their only choices are how much cash to take up to the 25% tax free limit and (up to a point) when to retire.
2. Current regulations deter innovation. This is, of course, why defined ambition as an idea has been so slow to get off the ground.
3. Restrictions on cash commutation imply a lack of trust of members to be able to decide how they spend their savings.
4. The concern that the annuity market has not maximised income for scheme members. This is mirrored by the high cost of de-risking via bulk annuities, which is the ultimate “flight path” for most DB pension schemes, and which many argue has resulted in a big drag on the growth of UK PLC.

All that would be required to extend the proposed freedoms would be to allow DB members to commute as much of their benefits at retirement, whether for cash or income drawdown, as they wanted, with the rest taken as pension as now. This could be applied to private and public sector schemes and would, I believe, at a stroke head off the rush to transfer.

Even if the Government does manage to stop people pouring out of the exits before April next year, this has to be bad policy. To provide more freedom and choice to one group of pensioners and at the same time to remove a longstanding freedom (and one available at the point members joined the schemes) from the other groups is clearly unfair. What is worse, with an election looming, it is likely to be unpopular.

A.4
The government would welcome views on any potential impact of the government’s proposals on investment and financial markets.

For private DB schemes, the Government says the decision is “finely balanced”. I think their fears are exaggerated and rather contradict the earlier declaration of trust in pensioners to make appropriate decisions about their retirement – after all appropriate investment in support of regular income in retirement (which would presumably be recommended by the “guaranteed guidance” to be offered to DC members) should not differ markedly from the equivalent investments in DB schemes. Whether DB schemes invest on a longer-term basis than individuals is, as the Kay Review made clear, uncertain.
The level of the Government’s concern about financial markets rather makes it look as if individuals can be trusted to look after themselves, with a slightly bigger safety net and a bit of advice, but financial markets cannot. This cannot be right.

Tweet about this on TwitterShare on FacebookShare on Google+Share on LinkedInPin on PinterestShare on TumblrEmail this to someone

I was introduced to a great piece of research by Tim Jenkinson, Howard Jones and Jose Vicente Martinez this week (Tim was speaking at the Workplace Pensions Live event in Birmingham). It looked at the performance of US active equity products recommended by investment consultants (a large sample covering 90% of the investment consulting market worldwide) compared to those not recommended by them over the period 1999-2011.

What they found was that:

  • Investment consultants’ recommendations seem less heavily influenced by return-chasing strategies than by more intangible personal assessments, eg of the capabilities of fund managers, the consistency of their philosophies and the usefulness of their reports (any of those explanations for recommendations sound familiar?);
  • People tend to follow the recommendations they are given; and
  • There is no evidence that investment consultants’ recommendations add value to plan sponsors.

IC underperformanceThe underperformance of recommended funds compared to unrecommended was 1% pa on average when all funds were given an equal weighting, falling to an underperformance of 0.26% pa when weighted by the size of fund recommended. This suggests that when investment consultants move away from recommending larger funds they are doing even worse.

 

There may be other reasons for using an investment consultant other than higher returns of course. People may appreciate “a narrative that provides comfort” (similar to the placebo effect in financial advice I previously discussed here) and which gives them ready-made explanations for their own stakeholders. However, bearing in mind the consistent underperformance, why do they follow the recommendations they are given?

One reason may be that the recommendations provide cover for decisions made. Another may be regulatory pressures, eg the Pensions Regulator in the UK requires pension scheme trustees to take professional investment advice (a requirement Tim Jenkinson believes is unhelpful) and it may be viewed as odd to then ignore it.

But the report concludes that a more likely reason is that people are generally unaware of how little value is being added. Certainly studies like this one that set the problem out in such stark terms are fairly thin on the ground. The investment consultants’ world is a very concentrated one (of the $25 trillion funds under management: $4.4 trillion are managed by Aon Hewitt, $4 trillion by Mercer and $2.1 trillion by Towers Watson) and the necessary information can be difficult to get hold of.

Another reason that the underperformance may be less obvious is the impact of the recommendation itself. As John Allen Paulos explains in his classic A Mathematician Plays the Market, for an over or underperforming stock you both need the performance itself and for someone to pick it. If you always pick what the investment consultant recommends the second condition is automatically met. Sometimes that will be the lucky stock and sometimes it won’t, but you will always choose it when it is, whereas a random choice of stocks will choose it in its lucky weeks less frequently.

What all this tells us is that you cannot assume that the additional complexity investment consultants’ appear to be biased towards is adding any value to your pension scheme or business. Tim Jenkinson suggests there should be a presumption of passive investment unless a very persuasive argument for active management can be advanced. And at the small end, as he says: “if you’re not big, be simple”.

Tweet about this on TwitterShare on FacebookShare on Google+Share on LinkedInPin on PinterestShare on TumblrEmail this to someone
The Comedy of Errors - with apologies to William Shakespeare in the week of his 450th birthday

The Comedy of Errors
– with apologies to William Shakespeare in the week of his 450th birthday

There are two ways that mistakes can happen when you are carrying out an experiment to test a hypothesis. Experiments usually have two possible outcomes: accepting a “null” hypothesis, which means concluding that the experiment does not challenge its truth, and rejecting a null hypothesis, which means concluding that the experiment does provide sufficient evidence to do so.

Type 1 errors, otherwise known as “false positives” are when you think there is evidence for rejecting the null hypothesis (eg deciding there actually is something wrong with a smear test) when there isn’t. Type 2 errors, otherwise known as “false negatives”, are when you accept the null hypothesis but you really shouldn’t (eg telling someone they are all clear when they are not).

Saddam Hussain once famously said “I would rather kill my friends in error, than allow my enemies to live”. This suggests that he was really very much more concerned about Type 2 errors than Type 1 errors.

He is not alone in this.

A recent widely reported academic paper published in Nature claimed to have a test that “predicted phenoconversion to either amnestic mild cognitive impairment or Alzheimer’s disease within a 2–3 year timeframe with over 90% accuracy”.

The latest statistics from the Alzheimer’s Society suggest that around 1 in 14 or 7% of over 65s will develop Alzheimer’s. Probably not all of these people will contract the disease within 3 years, but let’s assume for the sake of argument that they will. Even so this means that, out of 1,000 people over 65, 930 people will not get Alzheimer’s within 3 years.

Applying the 90% accuracy rate allows us to detect 63 out of 70 people who actually will get Alzheimer’s. There will be 7 cases not picked up where people go on to develop Alzheimer’s. However the bigger problem, the Type 1 error that Saddam Hussain was not so bothered about, is that 10% of the people who do not and will not get Alzheimer’s will be told that they will. That is 93 people scared unnecessarily.

So 63 + 93 = 156 people will test positive, of which only 63 (ie 40%) will develop Alzheimer’s within three years. The “over 90%” accuracy rate becomes only a 40% accuracy rate amongst all the people testing positive.

In statistical tests more generally, if the likelihood of a false positive is less than 5%, the evidence that the hypothesis is true is commonly described as “statistically significant”. In 2005 John Ioannidis, an epidemiologist from Stanford University, published a paper arguing that most published research findings are probably false. This was because of three things often not highlighted in the reporting of research: the statistical power of the study (ie the probability of not making a type 2 error or false negative), how unlikely the hypothesis is being tested and the bias in favour of testing new hypotheses over replicating previous results.

As an example, if we test 1,000 hypotheses of which 100 are actually true but with a 5% test of significance, a study with power of 0.8 will find 80 of them, missing 20 because of false negatives. Of the 900 hypotheses that are wrong, up to 5% – ie, 45 of them – could be accepted as right because of the permissible level of type 1 errors or false positives. So you have 80 + 45 = 125 positive results, of which 36% are incorrect. If the statistical power is closer to the level which some research findings have suggested of around 0.4, you would have 40 + 45 = 85 positive results, of which 53% would be incorrect, supporting Professor Ioannidis’ claim even before you get onto the other problems he mentions.

We would have got much more reliable results if we had just focused on the negative in these examples. With a power of 0.8, we would get 20 false negatives and 855 true negatives, ie 2% of the negative results are incorrect. With a power of 0.4, we would get 60 false negatives and 855 true negatives, ie still less than 7% of the negative results are incorrect. Unfortunately negative results account for just 10-30% of published scientific literature, depending on the discipline. This bias may be growing. A study of 4,600 papers from across the sciences conducted by Daniele Fanelli of the University of Edinburgh found that the proportion of positive results increased by over 22% between 1990 and 2007.

So, if you are looking to the scientific literature to support an argument you want to advance, be careful. It may not be as positive as it seems.

Tweet about this on TwitterShare on FacebookShare on Google+Share on LinkedInPin on PinterestShare on TumblrEmail this to someone

Typology of biasI found this diagram recently in a paper by John Adams from 1999 entitled Risk, Freedom and Responsibility. It attempts to summarise different people’s attitude to risk-taking based on their views about the kind of world they live in, represented by a ball sitting in very different types of landscape. It explains a great deal about pensions.

Much is often made about our seemingly inexorable shift away from collective solutions to problems to individualised ones, aided on the one hand by technology like tablets, smart phones and other devices which make it easier for us to create our own environments and cut ourselves off from each other, and on the other by a loss of trust in many of the traditional collective organisations, such as banks and governments, which have previously been used by us to pool our risks and protect the most vulnerable.

If this is true, then it would be represented in the diagram by a shift in world view from right to left.

Others focus on the triumph of the American business model or ABM as the dominant school of political and economic thought in the globalised world of today, just as socialism was in previous times. This model leads to a belief in low taxation, small government, minimal market regulation and the reliance of self-interested materialism of individuals within these markets to deliver what we need. Despite its name, it is not a description of how American business actually works, but just one of what Adams would call the “myths about nature” which often determine our thinking about risk and much else besides.

If the triumph of the ABM is true, then it would be represented in the diagram by a shift in world view from bottom to top.

Adams points out that most people exhibit several of these world views and move between them, sometimes very quickly, but I think that it is easy to see where the stereotypical figures from the UK pensions landscape might sit. For instance, many owners of SMEs are calculated risk-takers who believe that things tend to turn out okay on the whole. That is how they became business owners in the first place. So, in the diagram above, taking a few risks with the football is not going to lose it, but there might be a reasonable amount of bouncing around: ie they are individualists.

In the top right hand corner are the hierarchists. They do not believe that the environment in which they operate is fundamentally benign but they do think that it can be managed. This is why their landscape resembles a series of speed bumps: the football cannot be allowed too much freedom or the consequences might be serious and it is possible to deny the football that freedom. This is the world view of a large number of civil servants and actuaries, which is why the public sector is still running defined benefit pension schemes and the private sector (with the smaller schemes overwhelmingly sponsored by individualists) has largely retreated from them. The larger companies, which tend to harbour their fair share of hierarchists, have been the slowest to abandon such schemes.

In the bottom right hand corner are the egalitarians: people who believe that giving the football anything more than a light tap is likely to lose it forever. Nature is unforgiving and cannot be controlled, but the less we do to destabilise the environment, the longer she is likely to let us live. The resource and environment group of actuaries, with their focus on limits to growth and the implications of this, are likely to contain a number of egalitarians in their ranks.

And where are the pension scheme members? Well, even 15 years ago Adams reckoned on at least 40% of the population being fatalists. This is the perfectly flat landscape representing the idea that it does not remotely matter what you do with the ball, the end result will be the same. Adams cites a survey carried out in 1998 on young adults in England in which, when they were asked to imagine that they could only have one of two rights – the right to vote in an election, or the right to obtain a driving licence, 72% chose the driving licence. I think it is probable that this proportion would be higher now.

So we have pension schemes largely inhabited by fatalists and run either by individualists, in the case of smaller schemes, or by hierarchists in the case of larger and/or public sector schemes. The reason they have had to be auto-enrolled into schemes they did not choose to join themselves is because they do not fundamentally believe that it will make any difference, which makes the cost of it at any price too high.

However they are not comfortable being fatalists. The Pension Regulator’s survey of defined contribution (DC) pension scheme members in 2012 revealed that the three things they wanted most of all were:

  • Someone making clear to them how much they needed to save;
  • Being able to talk to someone to understand their pensions better; and
  • Clear communication from their employer and their pension provider.

All of which would make them less fatalistic and feel more in control. Whether you feel this would move them upwards into the individualist camp or diagonally across to the hierarchical camp (or even over to the egalitarian position) probably depends on your politics, but none of these positions are fixed. The recent floods have shaken many business people’s faith in things basically turning out okay in the end, and the credit crunch certainly moved many people out of hierarchist into either egalitarian or individualist territory.

What it suggests to me is that the way we organise pension scheme membership may be fundamentally flawed. Talking to members about their risk appetite or tolerance to risk is starting from an individualist perspective: that the world is a benign place, nothing too extreme is likely to happen and the only choice for you to make is how you want to invest your money. But it makes no sense if, assuming you can be coaxed away from the fatalist position, you turn out to be an egalitarian or a hierarchist. And this position probably makes no sense to the sponsor of the scheme.

When asked, sponsors of smaller schemes are very clear that they do not support the idea of collective schemes. They want to run their own schemes otherwise a large part of the benefits of the arrangements to them are lost. However, if auto enrolment is to deliver the changed relationship between the public and pensions everyone hopes for, I think prospective members are going to need choices about more than investment strategy. If members want to pool risk I think they should be able to, and collective schemes alongside firms’ own DC arrangements, perhaps with joint membership, may be the way to achieve this.

Individualists, hierarchists, fatalists and egalitarians. As Adams points out “the clamorous debate is characterised not by irrationality, but by plural rationalities.” It is a debate which has a long way to go yet.

Tweet about this on TwitterShare on FacebookShare on Google+Share on LinkedInPin on PinterestShare on TumblrEmail this to someone

doctorIn all the talk about annuities and the poor value they currently offer, nearly all of it has been based on standard annuity rates, ie where there is nothing sufficiently medically wrong with you to affect your life expectancy. However this is almost certainly not the rate you should be looking at.

Go to any of the annuity provider or broker websites, sometimes buried away a little, and you will find a link explaining what they can offer in the way of “enhanced” or “impaired lives” annuities. Legal & General’s web page on this looks like the kind of warning notice you find on the wall of your doctor’s surgery waiting room, with headings like Smoking, Type 2 Diabetes and High Blood Pressure. But in the upside-down world of buying annuities these become good things to do or have.

Just Retirement give some handy illustrations of what various conditions could mean for your income: up 20% for minor conditions like obesity and hypertension, up 30% for “moderate” ones like being a heart attack survivor with a bypass and 40% for serious medical conditions like stage 2 bowel cancer one year in. However, you don’t need to get anywhere near the frankly frightening conditions in the moderate and serious boxes to make a big difference to the income you can receive. annuitydiscount.co.uk provide a very long list of medications (covering every letter in the alphabet except J and Y) which could lead to an impaired life annuity if disclosed to the annuity provider.

As the BBC article from 2012 posted by the Better Retirement Group on enhanced annuities says: “At its simplest an annuity is a bet with the insurance company about how long you will live.”

So on that basis, it makes sense to stack the odds in your favour as much as you can. Which makes the 2007 article in the New England Journal of Medicine entitled, rather dully, Incidental Findings on Brain MRI in the General Population, such an interesting read.

They studied 2,000 people (mean age 63.3 years, range 45.7 to 96.7) from the population-based Rotterdam Study in whom high-resolution, structural brain MRI scans had been carried out. Asymptomatic brain infarcts (more commonly known as strokes) were present in 145 people (7.2%). Among other findings, aneurysms (1.8%) were the most frequent. Benign brain tumors also turned up reasonably often (1.6%). The most extreme case was someone with a large, chronic subdural haematoma, who was subsequently found to have had a minor head trauma 4 weeks before the MRI scan. Some of the scans are shown below.

brain scansBut the really amazing thing is this: only 2 of the 2,000 people scanned (the subdural haemotoma mentioned above and another who had a 12 mm aneurysm of the medial cerebral artery) had any idea that there was anything wrong with them!

Another huge area of undiagnosed disease (and on the annuity.co.uk list for enhanced annuities) is prostate cancer. According to a systematic review of prostate cancer biopsy schemes by the University of York in 2005, where they quoted from the NHS Centre for Reviews and Dissemination publication on screening for prostate cancer, Effectiveness Matters:

Post mortem studies show that 30% of men over 50, who had no symptoms of prostate cancer whilst alive, had histological evidence of prostate cancer at the time of death. This percentage rises to 60-70% in men over 80 years of age. In other words, most men with prostate cancer die with, rather than from, the disease.

The main reason these studies have been carried out is to determine whether screening for prostate cancer, which kills 3.8% of men with the disease, has saved many lives. The Prostate Specific Antigen (PSA) test that is commonly used to detect prostate cancer in the absence of symptoms is not only prone to false positives and negatives (ie telling you you have it when you don’t and don’t have it when you do – something all screening suffers from to some extent), but can lead to you being offered treatment which may well be worse than the disease. This is discussed further in the excellent The Norm Chronicles, by Michael Blastland and David Spiegelhalter, which questions whether, overall, screening is particularly effective in saving lives.

Effective in preventing death? Perhaps not. But effective in increasing retirement income? Almost certainly.

The latest Association of British Insurers (ABI) facts and figures on the UK annuity market suggest that enhanced annuities have grown in popularity, to 24% in 2012 from 2% in 2003. There is scope to make further large increases in these figures if more people can be persuaded to have themselves screened for some of the most common undiagnosed conditions before they retire.

So don’t necessarily accept a standard annuity rate. And consider getting yourself tested first.

Tweet about this on TwitterShare on FacebookShare on Google+Share on LinkedInPin on PinterestShare on TumblrEmail this to someone