We Know Zero

Two views of Park View

09. June 2014 · Write a comment · Categories: Birmingham, Regulation

Whatever you think about the risk of extremism being incubated in Birmingham schools or the political battles going on at the top of the Conservative Party, the contrast between the two Ofsted reports into Park View is startling to say the least. Just look at the key findings from the report issued today and compare them with comments from the 2012 report:

Park view Ofsteds

Are these really the same schools? The only summary findings which do not have a direct and emphatic rebuttal from the report two years earlier are those from a section of the current staff who appear to be concerned about leadership, feel the governors are involved inappropriately, and complain about unfair recruitment practices and feeling intimidated against speaking out. So are these courageous whistle-blowers, who were still summoning up that courage two years ago, or merely resentful staff?

What has changed? The new report refers to “considerable staff changes since the academy opened almost two years ago”. It then continues as follows:

The substantive principal, who was appointed in Spring 2012, is currently seconded to Golden Hillock School as acting principal. A Park View vice principal is currently the acting principal. The former academy principal is now the executive principal of the trust, she is due to retire at the end of the Spring Term 2014. She is being replaced by the substantive principal of Park View, who will continue in this capacity in addition to the post of executive principal of the trust. Other leaders have transferred to, or work part of their time in, the other two trust academy schools. A new vice principal and a new assistant principal have been appointed to start in April 2014.

One thing that has definitely changed is that in 2012 the school, Park View Business and Enterprise School as it was then, was a school. It then converted to academy status in April 2012, Park View School Academy of Mathematics and Science, the trust being managed by Park View Educational Trust, which then expanded into a multi-academy trust that includes Nansen Primary School and Golden Hillock School. All three have now been put into special measures.

So is it the academy programme which is the problem here rather than anything to do with extremism? A school subject to special measures can be subjected to further Ofsted inspections at very short-notice (although now we hear that no-notice inspections for all schools are being considered) to monitor its improvement. The senior managers and teaching staff can be dismissed and the school governors replaced by an appointed executive committee. This sounds very much like Park View’s recent history. It is unclear what benefit more short-notice inspections are likely to have, when their findings can differ as radically as this.

It seems fairly clear that something has gone wrong in the governance of these schools during the process of moving to a multi-academy trust, and that at least some of the school community feel that they have not had their views properly taken into account during that process. Raising the bogey of hardline Muslim extremism stalking the city’s schools, as Michael Wilshaw has done today, does not help anybody tackle this.

What also seems fairly clear is that the Ofsted inspection regime is considerably less objective than it would have us believe. Whatever changes may have occurred at this school over a two year period, at least some of the vertiginous decline from outstanding with no concerns to inadequate with few redeeming features (they came up with three in their summary) can ultimately only be explained by the last team liking the school and the new team (admittedly working in a very different climate) disliking the school. Those “likes” and “dislikes” appear to be what passes for a regulatory framework for the education of our young people. It is not nearly good enough.

Freedom and choice – a response to the Treasury pensions consultation

05. June 2014 · Write a comment · Categories: Defined Ambition, Legal, Pensions, Regulation, Risk

The consultation on the proposals for pensions announced in the Budget, and contained in yesterday’s Queen’s Speech, ends on 11 June. I have set out my response below. I hope that it will sufficiently incense one or two more people into making their views heard, before the chance disappears.

A.1
The government welcomes views on its proposed approach to reforming the pensions tax framework.

1 Should a statutory override be put in place to ensure that pension scheme rules do not prevent individuals from taking advantage of increased flexibility?

Yes. Otherwise you are just writing cheques to pensions lawyers.

2 How could the government design the new system such that it enables innovation in the retirement income market?

Reform preservation rules, the TPR code on funding, HMRC rules and the PPF levy framework so as not to penalise different arrangements across the defined ambition spectrum. Remove the annual allowance, controlling the level of tax relief offered through the lifetime allowance only (I got this the wrong way round in my first draft – the annual allowance assumes regular incomes, many people now have incomes which bounce up and down alarmingly from year to year. It is also ridiculously cumbersome to administer).

3 Do you agree that the age at which private pension wealth can be accessed should rise alongside the State Pension age?

No. There is already an issue around healthy life expectancy and the state pension age in some regions of the UK.

4 Should the change in the minimum pension age be applied to all pension schemes which qualify for tax relief?

Yes. The arrangements need simplification.

5 Should the minimum pension age be increased further, for example so that it is five years below State Pension age?

No (see answer to 3).

A.2
The government welcomes views on its proposed approach to supporting consumers in making retirement choices.

6 Is the prescription of standards enough to ensure the impartiality of guidance delivered by the pension provider? Should pension providers be required to outsource delivery of independent guidance to a trusted third party?

There needs to be more clarity about the charges which can be levied for guidance or if it is to be remunerated in some other way.

7 Should there be any difference between the requirements to offer guidance placed on contract-based pension providers and trust-based pension schemes?

No. In most cases the scheme members have not chosen to receive lower levels of service.

8 What more can be done to ensure that guidance is available at key decision points during retirement?

I think there needs to be a right (but not requirement) to it for everyone at 50, 60, 70 and 80 as a minimum, at an agreed national nominal charge. I imagine that the £20 million available to develop resources for this will need to be increased significantly to make an impact on the quality of guidance materials provided.

A.3
The government would welcome views on the options outlined in point 5.15, including their likely complexity, and the burdens they might place on scheme sponsors and HMRC.

9 Should the government continue to allow private sector defined benefit to defined contribution transfers and if so, in which circumstances?

Yes. In all circumstances.

10 How should the government assess the risks associated with allowing private sector defined benefit schemes to transfer to defined contribution under the proposed tax system?

The reasons the Government have advanced for the changes to DC are equally compelling when applied to DB:

1. There is a lack of choice for people at retirement, which has become more of an urgent concern now that auto enrolment is boosting DC membership. This is even more the case for DB members who are already numerous (although getting less so daily), as their only choices are how much cash to take up to the 25% tax free limit and (up to a point) when to retire.
2. Current regulations deter innovation. This is, of course, why defined ambition as an idea has been so slow to get off the ground.
3. Restrictions on cash commutation imply a lack of trust of members to be able to decide how they spend their savings.
4. The concern that the annuity market has not maximised income for scheme members. This is mirrored by the high cost of de-risking via bulk annuities, which is the ultimate “flight path” for most DB pension schemes, and which many argue has resulted in a big drag on the growth of UK PLC.

All that would be required to extend the proposed freedoms would be to allow DB members to commute as much of their benefits at retirement, whether for cash or income drawdown, as they wanted, with the rest taken as pension as now. This could be applied to private and public sector schemes and would, I believe, at a stroke head off the rush to transfer.

Even if the Government does manage to stop people pouring out of the exits before April next year, this has to be bad policy. To provide more freedom and choice to one group of pensioners and at the same time to remove a longstanding freedom (and one available at the point members joined the schemes) from the other groups is clearly unfair. What is worse, with an election looming, it is likely to be unpopular.

A.4
The government would welcome views on any potential impact of the government’s proposals on investment and financial markets.

For private DB schemes, the Government says the decision is “finely balanced”. I think their fears are exaggerated and rather contradict the earlier declaration of trust in pensioners to make appropriate decisions about their retirement – after all appropriate investment in support of regular income in retirement (which would presumably be recommended by the “guaranteed guidance” to be offered to DC members) should not differ markedly from the equivalent investments in DB schemes. Whether DB schemes invest on a longer-term basis than individuals is, as the Kay Review made clear, uncertain.
The level of the Government’s concern about financial markets rather makes it look as if individuals can be trusted to look after themselves, with a slightly bigger safety net and a bit of advice, but financial markets cannot. This cannot be right.

As easy as ABC?

01. June 2014 · Write a comment · Categories: Cartoons, Legal, Pensions, Regulation

With the recent revelations about the tax affairs of Gabby Logan and Gary Barlow in the news, it seems a good time to focus on the pension scheme equivalent.

Asset-backed contributions or ABCs have been lurking in the background of pension scheme funding for a while (Marks and Spencer set up such an arrangement in 2007), but have really only come to prominence since 2010. As you can see, they had quite a few takers over the next three years:

The total number of ABCs has now grown to more than 60, with a value of more than £6 billion. The value of each has tended to be around 10-20% of total scheme assets.

So why is this? For employers the answer is easy:

The employer can “pay” across an asset to the scheme while continuing to use it within its business.
The future stream of payments to the scheme is capitalised to make an immediate increase to the scheme’s funding level, which both makes the company accounts look better (although the Financial Reporting Council have been looking hard at a number of these) and reduces its Pension Protection Fund levy.
There is the potential to accelerate the tax relief on employer contributions if it is set up carefully.
The new effective “recovery period” (ie the period over which the stream of payments is paid from the special purpose vehicle known as a Scottish limited partnership (SLP) into the scheme) is usually longer than that of the recovery plan it replaces. It may also be more “back end loaded”, ie with a lump sum at the end allowing lower payments in the short to medium term.

But for trustees it is less clear:

The payments into the scheme are normally lower than they would be under a recovery plan which would not attract additional scrutiny from the Pensions Regulator.
The “asset” the employer is offering should already have been priced into the funding negotiations as part of the assets of the company included within the trustees’ covenant review. The ability to gain access to this asset on the occurrence of certain trigger events is, in principle, no different from the employer allowing the scheme to take a charge over that asset. However there are likely to be more hurdles to realising the asset under an SLP-type arrangement, as these arrangements are inherently more complicated than a simple legal charge.
There is usually no flexibility about the payments from such an arrangement which are targeted to meet a notional funding target many years in the future. By this time, the true funding target is likely to have changed, as will the value of the asset held in the SLP.
In order to make the arrangement work, they have to be a corporate trustee, even if they have not previously felt the need to incorporate to carry out their duties.

In summary, this is a vehicle for getting around the restriction on employer-related investment (ERI) of 5% of total assets which has existed since the Pensions Act 1995 came in. The only exceptions previously were small self-administered schemes (SSASs) which could use company property and loans to the company as assets on the basis that all the people in them were directors of the company. Whether it achieves this or not is as yet untested in the courts, although there have been some very confident legal opinions expressed about the fact that the letter of the ERI legislation only refers to shares or other securities, which cannot exist in this case because:

The SLP is an unincorporated body within the UK so it cannot issue shares. As one lawyer has said “the magic of a SLP is its distinct legal identity”.
A partnership interest is not generally considered a share (which is why, the confident legal opinion goes, along with the safeguards written into the agreements, Scottish independence would not make these deals suddenly illegal – although this obviously begs the question of why then you would go to such great lengths to create a SLP in the first place).

The Pensions Regulator is clearly uncomfortable with these arrangements, sensing that they are just devices for driving a coach and horses through its code of practice on funding. However, they are not illegal, so the Regulator has been able to do no more than issue guidance to trustees and their advisers on asset-backed contributions, with a long list of risks that they pose and advice trustees would need to seek before agreeing to one. They correctly point out that an ABC is not a bond-like investment, as some have suggested (unless by bond you mean a corporate bond issued by the sponsor of the scheme, ie an investment which becomes riskier the worse your sponsor is doing – which is not normally the point of bond investment). But the real kicker is the requirement they have set for a separate underpin that would protect the scheme’s position eg “in the event that the courts find that ABCs are void for illegality or where there is a change in the law”. This could turn out to be very expensive for the 60 such arrangements already in place.

However, just for a moment, despite all memories of other situations where lawyers have told us that scheme documents are copper-bottomed but which have subsequently proved to have traces of straw (equalisation, for instance), let us assume that the ABC drawn up in the way the Regulator has suggested will benefit the schemes which participate. These arrangements may observe the letter of the legislation but they clearly do not observe their spirit. Just look at the typical structure of one:

And then tell me that it bears no resemblance to the kind of “tax management scheme” we have seen punished recently. Here is Chris Moyles’ one as an example:

More and more voices are questioning the tax relief that pensions receive (the Institute of Fiscal Studies being one recent example). Steve Webb has also indicated that he would like to see a reduction in tax relief on pension contributions for higher rate taxpayers. Is this really the time to be championing schemes which accelerate that tax relief even more?

Investment consultants and value

15. May 2014 · Write a comment · Categories: Birmingham, Pensions, Regulation, Risk

I was introduced to a great piece of research by Tim Jenkinson, Howard Jones and Jose Vicente Martinez this week (Tim was speaking at the Workplace Pensions Live event in Birmingham). It looked at the performance of US active equity products recommended by investment consultants (a large sample covering 90% of the investment consulting market worldwide) compared to those not recommended by them over the period 1999-2011.

What they found was that:

Investment consultants’ recommendations seem less heavily influenced by return-chasing strategies than by more intangible personal assessments, eg of the capabilities of fund managers, the consistency of their philosophies and the usefulness of their reports (any of those explanations for recommendations sound familiar?);
People tend to follow the recommendations they are given; and
There is no evidence that investment consultants’ recommendations add value to plan sponsors.

The underperformance of recommended funds compared to unrecommended was 1% pa on average when all funds were given an equal weighting, falling to an underperformance of 0.26% pa when weighted by the size of fund recommended. This suggests that when investment consultants move away from recommending larger funds they are doing even worse.

There may be other reasons for using an investment consultant other than higher returns of course. People may appreciate “a narrative that provides comfort” (similar to the placebo effect in financial advice I previously discussed here) and which gives them ready-made explanations for their own stakeholders. However, bearing in mind the consistent underperformance, why do they follow the recommendations they are given?

One reason may be that the recommendations provide cover for decisions made. Another may be regulatory pressures, eg the Pensions Regulator in the UK requires pension scheme trustees to take professional investment advice (a requirement Tim Jenkinson believes is unhelpful) and it may be viewed as odd to then ignore it.

But the report concludes that a more likely reason is that people are generally unaware of how little value is being added. Certainly studies like this one that set the problem out in such stark terms are fairly thin on the ground. The investment consultants’ world is a very concentrated one (of the $25 trillion funds under management: $4.4 trillion are managed by Aon Hewitt, $4 trillion by Mercer and $2.1 trillion by Towers Watson) and the necessary information can be difficult to get hold of.

Another reason that the underperformance may be less obvious is the impact of the recommendation itself. As John Allen Paulos explains in his classic A Mathematician Plays the Market, for an over or underperforming stock you both need the performance itself and for someone to pick it. If you always pick what the investment consultant recommends the second condition is automatically met. Sometimes that will be the lucky stock and sometimes it won’t, but you will always choose it when it is, whereas a random choice of stocks will choose it in its lucky weeks less frequently.

What all this tells us is that you cannot assume that the additional complexity investment consultants’ appear to be biased towards is adding any value to your pension scheme or business. Tim Jenkinson suggests there should be a presumption of passive investment unless a very persuasive argument for active management can be advanced. And at the small end, as he says: “if you’re not big, be simple”.

Postcode mortality

09. May 2014 · Write a comment · Categories: Demographics

As Rowan Atkinson once said: “Life is one of those things that most of us find very difficult to avoid”. You would imagine that death would be similarly unavoidable, but not in all cases it would seem.

The Office of National Statistics (ONS) have just issued the latest figures (for 2012) on what they define as avoidable deaths (“mortality figures for causes of death that are considered avoidable in the presence of timely and effective healthcare or public health interventions”) and have concluded that these account for a staggering 23% of all deaths.

Avoidable deaths defined in this way can be further broken down into preventable deaths (ie those that can be avoided mostly or completely through public health interventions, eg traffic accidents) and amenable deaths (ie those that can be avoided mostly or completely through good quality healthcare, eg epilepsy). Some, eg ischaemic heart disease (22% of avoidable deaths in men), are seen as both.

One interesting aspect of this release is the regional breakdown of avoidable deaths as a percentage. These vary from 15% in the South (outside London) to 20% in the North West for women and from 24% in the South West to 31% in London and the North for men.

As we can see from the following graph, it is possible for the proportion of deaths by cause to change quite dramatically over time. Could this also be true for avoidable deaths by region?

The methodology for calculating the impact of improved survival rates of various conditions in terms of the number of deaths avoided is “age-standardised”, which means that we are all living in a population with the same structure as an average European population from 1976, when ABBA were in their pomp. The actual potential proportion of lives saved may therefore be more or less than 23%. This also means that some of the differences between regions could be due to how different their population structures are to ABBAland, eg there are a lot more over 65s as a proportion of the 2013 Standard Population (which the ONS should be moving to using soon) than there are in the 1976 version.

However, as David Spiegelhalter shows here, it is relatively straightforward to convert an improvement or deterioration in mortality rates into an increase or decrease in life expectancy. Making a few heroic assumptions (in this case: 1. that the most popular mortality table currently used by occupational pension schemes, but without any mortality improvements since 2002/03, is an appropriate way to estimate the effect of removing avoidable mortality on life expectancy; and 2. that the mortality is avoided equally at all ages from 65 onwards) we can therefore estimate the 7% difference for men and 5% difference for women in the degree of avoidable mortality to be equivalent to around 0.6 years for men and 0.4 years for women in the life expectancy at age 65.

If the mortality rates could therefore be improved in regions with currently the highest proportions of avoidable mortality, so that the avoidable mortality remaining was no higher than in those regions where it is currently at the lowest proportions, life expectancy at 65 would be increased in some regions by up to 0.6 years.

However, averaging by region smooths out much larger differences between postcodes (postcode mortality mapping is commonly used, eg by insurance companies in setting annuity rates). For instance, the current gap between the highest and lowest life expectancy at age 65 is 6 years for men (between Harrow and Glasgow City) and 5.5 years for women (between Camden and Glasgow City). So perhaps we should be shooting for the rather higher ambition of removing all of the avoidable mortality. If the overall average avoidable mortality of 23% could actually be avoided, this would lead to an average increase in life expectancy at age 65 of 2 years.

Finally, what about the differences between men and women? The ONS report states than “Men are more likely to die from potentially avoidable causes than women, with about 28% (67,548 out of 240,238) dying from avoidable conditions compared with 17% (44,945 out of 259,093) of women in 2012.” That 11% difference would equate to a life expectancy difference at 65 of around 1 year, which is about half of the current gap between men and women at 65. Women living longer than men? It would seem that it’s only half inevitable.

Public sector pensions and liabilities

30. April 2014 · Write a comment · Categories: Actuarial, Demographics, Economics, Pensions, Prediction, Regulation

Sometimes the best explanations of things come when we are trying to explain them to outsiders, people not expected to understand our particular forest of acronyms, slangs and conventions which, while allowing speedier communication, can also channel thinking down the same tired old tracks time after time. Such an example I think is the UK Government Actuary’s Department (GAD) paper on Pensions for Public Service Employees in the UK, presented to the International Congress of Actuaries last month in Washington.

Not a lay audience admittedly, but one sufficiently removed from the UK for the paper’s writers to need to represent the bewildering complexity of UK public sector pension provision very clearly and concisely. The result is the best summary of the current position and the planned reforms that I have seen so far, and I would strongly recommend it to anyone interested in public sector pensions.

There are two points which struck me particularly about the summary of the reforms, designed to bring expenditure on public service pensions down from 2.1% of GDP in 2011-12 to 1.3% by 2061-62.

The first came while looking at the excellent summary of the factors contributing to the decline of private sector pension provision. Leaving aside the more general points about costs and risks, and those thought applicable to the (mainly) unfunded public service schemes which have been largely addressed by the planned reforms, I noticed two of the factors thought specific to funded defined benefits (DB) plans:

A more onerous burden on trustees of plans, including member representation, and knowledge and understanding; and
Company pension accounting rules requiring liabilities to be measured based on corporate bond yields.

As the GAD paper makes clear, the Public Service Pensions Act will result in a significant increase in interventions on governance in particular in some public sector schemes. The Pensions Regulator’s recent consultation on regulating public service pension schemes is also proposing a 60 page code of practice be adopted in respect of the governance and administration of these schemes. This looks like the “onerous burden” which has been visited on the private sector over the last 20 years all over again.

The other point is not directly comparable, as company pension accounting rules do not apply to the public sector. However, as pointed out by the Office for National Statistics (ONS) this week, supplementary tables to the National Accounts calculating public sector pensions liabilities will be required of all EU member states from September this year onwards, to comply with the European System of Accounts (ESA) 2010. These are carried out using best estimate assumptions (ie without margins for prudence) and a discount rate based on a long term estimate of GDP growth (as compared to the AA corporate bond yield required by accounting rules).

The ONS released the first such tables published by any EU member state, for 2010, in March 2012. This for the first time values the liabilities in respect of unfunded public sector pension entitlements, at £852 billion, down from £915 billion at the start of the year.

I think there is a real possibility that publication of this information, as it has for DB pension schemes, will result in pressure to reduce these liabilities where possible. An example would be one I mentioned in a previous post, where mass transfers to defined contribution (DC) arrangements from public sector schemes following the 2014 Budget have effectively been ruled out because of their potential impact on public finances. If such transfers reduced the liability figure under ESA 2010 (which they almost certainly would) the Government attitude to such transfers might be different in the future.

The second point concerned the ESA 2010 assumptions themselves. There was a previous consultation on the best discount rate used for these valuations, ie the percentage by which a payment required in one year’s time is more affordable than one required now, with GDP growth coming out as the preferred option. Leaving aside the many criticisms of GDP as an economic measure, one option which was not considered apparently was the growth in current Government receipts, although this would seem in many ways to be a better guide to the element of economic growth relevant to the affordability of public sector provision. Taking the Office for Budget Responsibility (OBR) forecasts from 2013-14 to 2018-19 with the fixed ESA 2010 assumptions for discount rate and inflation of 5% pa and 2% pa respectively gives us an interesting comparison.

The CPI assumption appears to be fairly much in line with forecasts, but the average nominal GDP and current receipt year on year increase over the next 6 years of forecasts are 4.47% pa and 4.61% pa (4.72% pa if National Accounts taxes are used rather than all current receipts) respectively. A 0.5% reduction in the discount rate to 4.5% pa would be expected to increase the liability by over 10%.

Another, possibly purer, measure of economic growth, removing as it does the distortions caused by net migration, would be the growth of GDP per capita. If we take the OBR forecasts for real GDP growth per capita and set it against the long term ESA 2010 assumption of 1.05/1.02 – 1 = 2.94% the comparison is even more interesting:

In this case the ESA assumption is around 1% pa greater than the forecasts would suggest, making the liability less than 80% of where it would be using the average forecast value.

The ESA 2010 assumptions are intended to be fixed so that figures for different years can easily be compared. It would clearly be easy to argue for tougher assumptions from the OBR forecasts (although the accuracy of these has of course not got a great track record), but perhaps more difficult to find an argument for relaxing them further.

Whether the consensus holds over keeping them fixed when and if the liability figures start to get more prominence and a lower liability becomes an important economic target for some of the larger EU member states remains to be seen. However if the assumptions cannot be changed, since public sector benefits now have a 25 year guarantee in the UK (other than the normal pension age now equal to the state pension age being subject to review every 5 years), then the cost cap mechanism (ie higher member contributions) becomes the only available safety valve. So we can perhaps expect nurses’ and teachers’ pension contributions to become the battleground when public sector pension affordability becomes a hot political issue once more.

We can poke fun at the Government’s enthusiasm to take on the Royal Mail Pension Plan and its focus on annual cashflows which made it look beneficial for their finances over the short term, but we may also look back wistfully to the days before public sector pensions stopped being viewed as a necessary expense of delivering services and became instead a liability to be minimised.

The misbehaviour of mistakes

25. April 2014 · Write a comment · Categories: Cartoons, Risk

The Comedy of Errors
– with apologies to William Shakespeare in the week of his 450th birthday

There are two ways that mistakes can happen when you are carrying out an experiment to test a hypothesis. Experiments usually have two possible outcomes: accepting a “null” hypothesis, which means concluding that the experiment does not challenge its truth, and rejecting a null hypothesis, which means concluding that the experiment does provide sufficient evidence to do so.

Type 1 errors, otherwise known as “false positives” are when you think there is evidence for rejecting the null hypothesis (eg deciding there actually is something wrong with a smear test) when there isn’t. Type 2 errors, otherwise known as “false negatives”, are when you accept the null hypothesis but you really shouldn’t (eg telling someone they are all clear when they are not).

Saddam Hussain once famously said “I would rather kill my friends in error, than allow my enemies to live”. This suggests that he was really very much more concerned about Type 2 errors than Type 1 errors.

He is not alone in this.

A recent widely reported academic paper published in Nature claimed to have a test that “predicted phenoconversion to either amnestic mild cognitive impairment or Alzheimer’s disease within a 2–3 year timeframe with over 90% accuracy”.

The latest statistics from the Alzheimer’s Society suggest that around 1 in 14 or 7% of over 65s will develop Alzheimer’s. Probably not all of these people will contract the disease within 3 years, but let’s assume for the sake of argument that they will. Even so this means that, out of 1,000 people over 65, 930 people will not get Alzheimer’s within 3 years.

Applying the 90% accuracy rate allows us to detect 63 out of 70 people who actually will get Alzheimer’s. There will be 7 cases not picked up where people go on to develop Alzheimer’s. However the bigger problem, the Type 1 error that Saddam Hussain was not so bothered about, is that 10% of the people who do not and will not get Alzheimer’s will be told that they will. That is 93 people scared unnecessarily.

So 63 + 93 = 156 people will test positive, of which only 63 (ie 40%) will develop Alzheimer’s within three years. The “over 90%” accuracy rate becomes only a 40% accuracy rate amongst all the people testing positive.

In statistical tests more generally, if the likelihood of a false positive is less than 5%, the evidence that the hypothesis is true is commonly described as “statistically significant”. In 2005 John Ioannidis, an epidemiologist from Stanford University, published a paper arguing that most published research findings are probably false. This was because of three things often not highlighted in the reporting of research: the statistical power of the study (ie the probability of not making a type 2 error or false negative), how unlikely the hypothesis is being tested and the bias in favour of testing new hypotheses over replicating previous results.

As an example, if we test 1,000 hypotheses of which 100 are actually true but with a 5% test of significance, a study with power of 0.8 will find 80 of them, missing 20 because of false negatives. Of the 900 hypotheses that are wrong, up to 5% – ie, 45 of them – could be accepted as right because of the permissible level of type 1 errors or false positives. So you have 80 + 45 = 125 positive results, of which 36% are incorrect. If the statistical power is closer to the level which some research findings have suggested of around 0.4, you would have 40 + 45 = 85 positive results, of which 53% would be incorrect, supporting Professor Ioannidis’ claim even before you get onto the other problems he mentions.

We would have got much more reliable results if we had just focused on the negative in these examples. With a power of 0.8, we would get 20 false negatives and 855 true negatives, ie 2% of the negative results are incorrect. With a power of 0.4, we would get 60 false negatives and 855 true negatives, ie still less than 7% of the negative results are incorrect. Unfortunately negative results account for just 10-30% of published scientific literature, depending on the discipline. This bias may be growing. A study of 4,600 papers from across the sciences conducted by Daniele Fanelli of the University of Edinburgh found that the proportion of positive results increased by over 22% between 1990 and 2007.

So, if you are looking to the scientific literature to support an argument you want to advance, be careful. It may not be as positive as it seems.

Inflation and Age

16. April 2014 · Write a comment · Categories: Economics

A recent piece of research into how the way people spend their money changes as they age got me thinking about how age might currently influence a particular household’s own Consumer Price Inflation (CPI).

The Office of National Statistics (ONS) produces a monthly inflation report for each major sector of expenditure, alongside the updates for CPI. The latest of these, out yesterday, showed CPI falling to 1.6%, finally less than wage inflation in the UK. However CPI is based on an average basket of goods to determine how people’s cost of living is increasing and this of course differs from person to person. In particular, the spending habits of the population, when split by the age of the household reference person (the ONS term for what used to be known as the “head of household”, when it was always the man of the house even when that house was owned by the woman: this was changed in the 90s), differ markedly.

The graph is based on the latest splits available of spending by the 12 categories used by the ONS to determine CPI (these are from 2012, although the overall averages these figures provide give CPI figures of 0.1% or less different from the actual figures so I think they will do for illustrative purposes).

What we can see straight away is that the gap between CPI for under 30 year old households and the 65-74 year old households has shrunk from around 0.8% pa to around 0.5% pa. However, under 30 year olds are still experiencing consistently higher inflation than the rest of the population. In fact this section of the working population are still experiencing inflation well in excess of the average wage inflation of 1.7% announced today.

The big winners since July have been the 30-44 year olds, with their households down to 1.5% pa inflation from 2.7% pa at the start of the period. And if your head of household is between 50 and 74 you are now experiencing average inflation year on year of only 1.4% pa.

The people who have not won quite so much (other than the under 30s) have been the over 75s, with their households at 2.6% pa at the start of the period and at 1.6% pa (ie bang on the average) at the end. Interestingly, the under 30s and over 75s have an average inflation above the headline rate, and everyone in between is coming in below it.

Why the differences? Well the change in the inflation rate has varied hugely amongst the 12 categories of spending.

Food and non-alcoholic drink inflation has fallen from 3.9% to 1.7%, Clothing and footwear from 2.5% to 0.2% and Transport from 1.5% to -1.0%, so any sector with spending habits weighted towards these areas is going to have done better than average. The 30-64 year old households spend between 5.9% and 6.1% on Clothing and between 15.9% and 16.7% on Transport compared to the averages of 5.6% and 15.2% respectively. However the over 75s spend 18.8% of their income on Housing, fuel and power, which has seen inflation remain stubbornly high at 3.1% pa, and have higher than average spending on Health (4.6% against 1.5% average) and Household goods and services (8.3% against 6.8% average), both of which saw an increase in inflation over the period. Education appears to be in a world of its own, with inflation falling from 19.7% at the start of the period to 10.3% at the end. Perhaps fortunately for those placing a lot of faith in the headline rate, Education does not account for more than 3% of spending in any group (it’s 3% for the under 30s) and only 1.6% overall.

So, as always, the picture is more complicated than the headline rate. And, when it comes to inflation, age matters.

The Cost of Defined Benefits is Defined

10. April 2014 · Write a comment · Categories: Pensions, Regulation

The Pensions Regulator has just published a remarkable survey. As it says:

In August 2013, The Pensions Regulator (the regulator) commissioned quantitative research into the running costs of defined benefit (DB) pension schemes. The specific objectives of the research were:

To understand the costs of administering a DB scheme;
To contextualise and understand scheme costs against services received;
To compare and contrast scheme costs by size, specifically at what size do scale efficiencies become apparent.

What they found instead was that costs for what they termed small schemes (those with 12 to 99 members, schemes with fewer than 12 members were excluded from the survey) were so variable that, even ignoring the top and bottom 5% of schemes, they ranged from £264 per member per year to £2,744 per member per year (over 10 times as much). This is far larger than any variation by size of scheme: the average for a small scheme is £1,054 per member per year, whereas the average for a very large scheme (more than 5,000 members) is £182 per member.

So the conclusions are clear: the costs of running a DB scheme are not primarily dependent on how big your scheme is, but how well you administer your scheme, how well you manage your advisors and service providers and how disciplined you are in setting an investment strategy and managing its implementation. For a small scheme, the irrelevance of size to costs is further illustrated by the following scatter graph helpfully supplied in the report, showing no correlation between total running costs per member and scheme size for schemes with 12 to 99 members:

This is not necessarily a call for more independent trustees. The proportion of small schemes which used independent trustees (22%) was not so much less than the proportion of large schemes (1,000 to 4,999 members) which used them (35%) and yet the variation in costs for large schemes (£80 to £689 per member per year) was not nearly as great. But it is a call for a much greater weighing of costs and benefits in the services trustees procure for small schemes.

So there is considerable work to do for small DB schemes, particularly with the additional costs likely to result from the Regulator’s recent proposals, which were consulted upon earlier this year. Another point that comes out of the survey is that the vast majority of schemes, of all sizes, regards the year in question where these costs were measured (2012) as having higher costs than an “average” year. Perhaps this is true, or perhaps there is some denial going on here about what the new normal looks like.

If you want to see how your scheme compares to others of its size, the Regulator has provided a handy checklist to capture the information. This would seem to be an exercise which many small schemes, if they are not already aware of this as an issue, would be well advised to carry out as a matter of urgency.

Sponsors are not currently getting good value from some of these schemes, particularly small schemes, which account for 1,689 (or 36%) of the 4,696 DB scheme universe (excluding hybrid and public sector schemes). The cost of defined benefit has been defined. And it needs to come down.

Actuaries and Placebos

09. April 2014 · Write a comment · Categories: Actuarial, Economics, Pensions, Regulation

Source: Wikimedia Commons – Original work of the US Federal Government – public domain

Placebos are medicines or procedures “prescribed for the psychological benefit to the patient rather than for any physiological effect” according to the Oxford English Dictionary. Originating as a way of doctors to clear their consulting rooms of people they did not feel could be helped with real medicine, placebos’ status, as Ted Kaptchuk makes clear, underwent a dramatic fall from acceptability after the Second World War and the general adoption of the randomised controlled trial to establish the efficacy of medical treatments. The lack of research since into the various aspects of treatment collectively called “the placebo effect” (Kaptchuk is a notable exception to this) is bemoaned by Kaptchuk, who feels an important element of successful treatments is not getting the attention it deserves.

This may be changing. After all, the impacts of placebos and nocebos (from the latin meaning “I will do you harm”, these are as mysterious as placebos, but make you feel worse rather than better) can be dramatic. Ben Goldacre does a five minute routine on them here (warning: it’s a bit rude). A recent Horizon documentary also looked at placebos, with the suggestion that they might have had an impact on the UK Olympic Team GB cyclists as well as in more serious cases like those of Parkinson’s sufferers.

Why am I talking about them? Because, in a more general way, to quote Seth Godin: “A placebo is a story we tell ourselves that changes the way our brain and our body work”. Godin asks why, if a placebo can make wine taste better and improve the way your back feels, we should be squeamish about discussing them.

The main reason, of course, is the feeling that it is unethical to promote treatments and products which have no scientific basis. This also explains why people operating within professions – whether medical or otherwise – are so wary of them. Professions see themselves, in the Baconian tradition, as bodies of people with expert knowledge using that expertise scientifically for the benefit of society. Placebos do not fit into this world view at all.

Imagine two pensions actuaries: Actuary A is a very experienced practitioner, known for years by many of his clients and a trusted source of wisdom. Everything he says, which he conveys with a practised seriousness and sonorousness, interspersed with frequent not-completely-discreet stories about the antics of other people he has met in his long career, is accepted by his clients like tablets of stone brought down from the mountain.

Actuary B is a young relatively newly qualified actuary. He has just obtained his scheme actuary certificate after toiling away in the background providing much of the analysis and calculation work underpinning the consultancy provided by the more senior actuaries in the firm, including Actuary A. He is seeking his first scheme actuary appointment, and has been trouping along to trustee meetings behind Actuary A for some carefully selected clients which the firm would like to move from A to B. B realises quickly, confirmed by his first trustee meeting where one of the trustees looks him up and down quickly and tells him that he doesn’t trust anyone with shiny shoes, that these clients have been selected because of their particular reluctance to pay the elevated charge out rate of Actuary A. Unfortunately this does not mean that they are keen to see a cheaper actuary installed on their schemes, quite the reverse in fact. The trustees who are most incredulous about the fees associated with actuarial advice seem also to be those who set most store in the mystical wisdom of Actuary A and his booming voice.

Now if I say that I think there are placebos at work here I do not mean that these clients are not receiving carefully constructed advice, appropriate to their needs and in compliance with all legislative and regulatory standards. What I am saying is that, from the lack of shine on Actuary A’s shoes, to the gravitas (I think it used to be referred to by a different generation as “bottom”) brought to bear on any particular issue by Actuary A, there are many things which do not add anything to the quality of advice (which in some cases has been almost entirely constructed by Actuary B), but which are valued at least as much (if not more, in Actuary B’s view) by the client.

As Simon Carne has pointed out recently, supported by John Reeve in this month’s The Actuary, the physical advice is subject to an ever increasing body of regulation, to the point where some clients might be deterred from even asking an actuary the time. However, everything about the environment in which the advice is conveyed – from the tone of voice; to the way the actuary sits; to the degree of direct eye contact; to the choice of gestures used; to, where meetings are held at the firm, the whole experience of someone entering the building and being led into a room deliberately designed to make an impression – is not. In the same way that a presentation is not just the collection of slides put together on PowerPoint, we need to give more recognition to the fact that the advice that is valued by clients is a lot more than that which is written or even spoken.

Although, judging from the number of times I have had to be the bearer of bad news (with the expectation that that will be the case preceding me and therefore helping me in delivering that message in many cases), perhaps the more usual term for this element of actuarial advice should be nocebo rather than placebo.