ABSTRACT
National systems of official statistics are expected to provide governments, businesses and the public with data about the economic, demographic, social and environmental situation. Digitisation of data collection for official statistics is mooted as having a major potential impact on society. Greater use of administrative data held in government is proceeding slowly. Evidence is limited, but access to big data − from satellites, point-of-sale systems and social media − is being explored and trialled. Initiatives are driven by new requirements for official statistics as well as pressures on traditional data collection from households and businesses. Effective government use of new data sources has the possibility of creating the ultimate evidence base for policies intended to improve lives, and anonymised data are also being made available to other researchers. The article uses the UK as a case study to outline how the big data evidence-to-policy process is intended to work in that context, and to assess the challenges faced in making it work as intended. We conclude that technical developments need to be accompanied throughout with attention to the marketing of official statistics and engagement with all users and potential users. The goal is to produce trusted as well as trustworthy statistics.
Acknowledgements
The author gratefully acknowledges advice provided and extensive material published by ONS. Thanks also to David Hand, two reviewers and the guest editors for many helpful suggestions. Views expressed in the paper are the personal views of the author and do not necessarily represent those of ONS, the UK Statistics Authority, the Statistics User Forum or the What Works Centre for Wellbeing.
Disclosure statement
No potential conflict of interest was reported by the author.
Additional information
Notes on contributors
Paul Allin
Paul Allin is a Visiting Professor in Statistics in the Department of Mathematics at Imperial College London. His research interests are the measurement of national wellbeing and progress, and the use of these measures in politics, policy, business and everyday life. He lectures on official statistics and chairs the UK Statistics User Forum and the Advisory Panel of the What Works Centre for Wellbeing. He spent 40 years as a statistician, researcher and policy analyst in UK government departments and agencies, including directing the Measuring National Wellbeing programme at the UK Office for National Statistics (ONS).