The latest revelations from Edward Snowden that the US and UK agreed in 2007 to relax the rules governing the mobile phone and fax numbers, emails and IP addresses that the US National Security Agency (NSA) could hold onto (and extending the net to people not the original targets of their surveillance) has increased the pressure on the Government to tighten controls on the activities of the security services. This extension apparently allowed the NSA to venture up to three “hops” away from a person of interest, eg a friend of a friend of a friend on Facebook.

I have an issue with the Guardian analysis here. They say that three hops from a typical Facebook user would rope in 5 million people. However, using actual ratios from the network in their source (43 friends have 3,975 friends of friends have 1,328,361 friends of friends of friends) and the median number of friends of 99 from the original study, would lead to a number closer to 3 million. Still, it is clearly altogether too many people to be treated as guilty by association.

So it might seem like a strange time for me to be advocating that we give the Government more of our data.

The Office for National Statistics (ONS) is currently consulting on the form of the next census and the future of population statistics generally. The two options they have come down to are:

1. Keep the 2021 census pretty much as it was for 2011, although with perhaps slight changes to the questions and a greater push for people to complete them online; or
2. Using administrative data already held by the Government in its various departments to produce an annual estimate of the population in local areas. In addition there would be separate compulsory surveys of 1% and 4% of the population for checking the overall population figures and some of the sub-grouping respectively, and the ‘residents of “communal establishments” such as university halls of residence and military bases’ which are difficult to reach by other means.

In my response to the survey, I suggested that they do both, increase the compulsory surveys each year to 10% of the population and reduce the time between full censuses to 5 years. This is why.

First of all, everybody needs this data to be available. If the Government does not provide it, someone else will. Not by asking you overt questions, but by buying information about your buying preferences or search engine activities or any number of other transactions without your informed consent (eg you ticked agreement to their terms and conditions on their website) and without your knowledge. I would prefer to give my data to the ONS.

The ONS is part of the UK Statistics Authority, which is an independent body at arm’s length from government. It reports directly to Parliament rather than to Government Ministers and has a strong track record of challenging the Government’s misuse of statistics. With the exception of requests received for personal information (which are filtered off to become Subject Access Requests under the Data Protection Act), they have provided copies of all information disclosed by the ONS under the Freedom of Information Act on their website. In my view the ONS has demonstrated that it is a safe custodian of our data. They are everything the NSA is not: overt, apolitical and committed to the appropriate use of statistics.

But there are problems with the current data, which brings me onto my second point. Ten years is too long to wait for updated information. As the ONS points out in its consultation document, because of the ten year gap between censuses, the population growth resulting from expansion of the European Union in 2004 was not fully understood until 2012. There were other problems with the population data everyone had been working with before 2011, 30,000 fewer people in their 90s than expected for instance, which had serious implications for all involved in services to the elderly and those constructing mortality tables too.

So we do need more frequent census information. Five years seems about right to me, provided the annual updates can be made more rigorous. I think the ONS are right to suggest that they need to be compulsory to achieve this, but 5% of the population does not seem a large enough sample to be confident about this to me. I would prefer to see 10% completing annual surveys. This would allow 50% of the population to be covered over every 5 year census period, or 40% if the requirement was dropped in census year. There are many recent examples (see Schonberger and Cukier below) to suggest that the gains in accuracy due to increased coverage would be far greater than the losses due to the ‘messiness’ of incomplete responses.

There is a lot in the consultation document about the relative costs of the different options, but nothing about the commercial value of the data being collected. Indeed the reduction of the consultation to these two, to my mind, inadequate options seems to be very greatly influenced by the question of costs and the current cuts in budgets seen throughout the public sector. This seems to me to be very short-sighted.

However, I think this displays a failure of imagination. According to Viktor Mayer-Schonberger and Kenneth Cukier in their book Big Data, data is set to be the greatest source of wealth and economic growth looking forward. Many others agree. By taking a fully accountable and carefully controlled approach to licensing the data in its care, the ONS should be able to finance its own activities, even at the level I am suggesting, at the very least.

The ONS is very nervous about becoming more intrusive in its collection methods, citing the 35% increase in cost of the 2011 census in achieving the same level of response. It also refers to the response rates to its voluntary surveys which have dropped from around 80% 30 years ago to around 60% today. The main reasons for this in my view are the incessant requests from companies’ marketing departments masquerading as surveys on everything from phone usage to our views on banking to the relentless demands for feedback on every online purchase making us all subject to survey fatigue. This makes it all the more necessary that an organisation which is not trying to sell you anything and which is scrupulous about the protection of your data should be attempting to increase its scope and maintaining its position as the go to place for statistical data rather than falling behind its commercial rivals.

So let’s not fall into the trap of conflating all official data with the mountains of bitty fragments collected by our intelligence agencies from their shady sources. That has nothing to do with the proper, accountable collection of information to allow government and governed alike access to what they need to make better decisions.

So take part in the consultation, it matters. And when the time comes give the ONS your data. You know it makes census.