I have written about false positives before, as have many others, but the media stories keep on coming. The latest reports, by the BBC most notably but also carried by the New Scientist and a number of other sources, display such an unfortunate ignorance of the issue of false positives that they risk raising false fears in the minds of sufferers of mild cognitive impairment and their families.

The BBC article makes a number of statements:

1. British scientists have made a “major step forward” in developing a blood test to predict the onset of Alzheimer’s disease.

2. Research in more than 1,000 people has identified a set of proteins in the blood which can predict the start of the dementia with 87% accuracy.

Neither of these statements are true. What the new test can do is guess right about patients with mild cognitive impairment (MCI) going on to develop Alzheimer’s disease (AD) within a year in 87% of cases. What it cannot do is predict whether someone with MCI will develop AD with 87% accuracy. The research article is here.

It all comes down to the likelihood of someone with MCI developing AD. It does not help that there is no general agreement of a definition for either term. The research reported on used the Petersen criterion for MCI. The likelihood of people within the population with MCI developing AD each year is again unknown, but 6 quite small studies (a total of 476 people across all the studies, with different average ages and sample sizes among the groups) carried out in North America analysed by Petersen et al in 2001 suggested that it lay between 6% and 25%, with an overall average rate of 12.5%.

So let’s assume that it is 12.5%. In a population of 1,000 people with MCI, we would expect 125 to develop AD. The test will identify 109 of them on average. Unfortunately, assuming the same accuracy rate of 87%, it will also give false positives for 13% (or 114) of the 875 not expected to develop AD in any particular year. That means that, of the 223 who test positive, less than half (49%) are actually expected to develop AD within a year. As the NHS choices website points out, in one of the few sceptical articles I could find, this is no better than tossing a coin.

If we assume the probability of moving from MCI to AD is at the higher bound of 25%, the positive predictive value (or PPV, as it is known) increases to 69%. However if we assume the lower bound of 6%, the PPV falls to 30%. In other words, if we get a positive blood test for this panel of 10 proteins, we currently do not know whether it is twice as likely to be a true positive than a false positive, or twice as likely to be false as true. Or anywhere in between.

Despite this, Dr Eric Karran, director of research at Alzheimer’s Research UK (clearly no conflict of interest there in promoting stories which promote the idea that Alzheimer’s research is highly effective) is widely reported as describing the study as a “technical tour de force”, while also acknowledging that the current accuracy levels risked telling many healthy people they were on course to develop Alzheimer’s.

In some reports it was pointed out that it was unlikely that the test would be used in isolation if it eventually made its way into clinics. A positive result could be backed up by brain scans or testing spinal fluid for signs of Alzheimer’s, they said. However if the test is no more predictive than a coin toss that is hardly encouraging.

There was more from Dr Karran: “This gives a better way to identify people who will progress to Alzheimer’s disease, people who can be entered into clinical trials earlier, I think that will increase the potential of a positive drug effect and thereby I think we will get to a therapy, which will be an absolute breakthrough if we can get there.”

This is simply untrue. Clearly it is important to support research into therapies for Alzheimer’s disease, but in raising funds for this the ends do not justify any means. Additional funding gained through false claims for any particular discovery will come at the expense of funding in other equally important areas. Like agreeing a definition of MCI and AD for instance, or better data on the transition probabilities from MCI to AD at different ages, without which a lot of the more laboratory-based research will be a waste of time as it will be unclear how best to apply it.

So let’s be careful how we report these things. People’s hopes and fears are at stake.