In the latest in our series of discussions with government’s foremost leaders, the ONS chief reveals plans to revamp migration and mobility statistics, and explore the limitless potential of AI
According to his biography on GOV.UK, the role of national statistician professor Sir Ian Diamond is to serve as government’s “principal adviser” on all things statistical.
There has, it is fair to say, been quite a lot to advise on in recent years.
On top of the day-to-day duties ensuring departments’ effective and ethical use of statistics, Diamond’s position – which also comes with the responsibilities of serving as chief executive of the UK Statistics Authority, and head of the Government Statistical Service and Analysis Function – Diamond was among those appearing on Downing Street news conferences in the early weeks of the coronavirus to help the public better understand the daily data.
This is perhaps the most visible example of the many ways in which the UKSA – and its core agency, the Office for National Statistics, of which Diamond is permanent secretary – gathers and provides data to inform government’s response to issues such as the post-pandemic labour market, the cost-of-living crisis, and the impact of war in Ukraine.
All the while, statisticians in the ONS and throughout public service are playing a key role in delivering on the aims of the National Data Strategy, which was published in September 2020 and set out an ambitious objective for “data to positively transform the UK”.
Alongside which the ONS recently published its own Data Strategy and there is now also a government-wide plan – Transforming for a digital future: 2022 to 2025 roadmap for digital and data – for driving innovation and reform across all departments. The document was published last year by the Cabinet Office’s Central Digital and Data Office, which is now overseeing the delivery of its six key missions.
PublicTechnology is delighted to be working with CDDO on our ongoing Digital Leaders’ Download series of interviews with some of government’s most high-profile figureheads – both in the fields of digital, data and technology and beyond.
Having previously published interviews with the digital unit’s chief executive Megan Lee Devlin and former HM Prison and Probation Service leader Jo Farrar, the latest piece in our series features a discussion between the CDDO head and Diamond – who was knighted in 2013 for his services to social science and higher education.
Let’s begin.
Megan Lee Devlin: Ian, what does being a ‘digital leader’ mean to you?
Professor Sir Ian Diamond: Well, I come really from the data end of things and so my first answer is that data are a central enabler for the whole digital agenda. And, as such, being a digital leader for me is about being an ambassador for data.
But too often, in my view, we spend too much time talking about data as data rather than the use cases. So being a digital leader, for me, is going much further than simply thinking about data themselves, but actually how they are going to improve the lives of all of our fellow citizens, and indeed, citizens across the world, and how we can enable that. We must as leaders be tireless in advocating and giving the reasons why this is important.
MLD: It sounds like what leadership in a digital era means to you is about advocacy and understanding and unlocking the business benefit. And indeed you’ve been a brilliant advocate for the work we’re doing to drive delivery against the Transforming for a Digital Future Roadmap. What would be your tips for senior civil servants on how they can practically advocate for better use of data in their departments and teams?
ID: From a data perspective, advocacy is about ensuring you’re role modelling using data, and demonstrating your interest and awareness, for example, ensuring that every decision that is made within your department is evidence-based, and being very, very catholic about what data are. Data aren’t just the numbers and surveys, but textual data, visual data, all kinds of data. Leaders need to have that understanding and the understanding of how to make sure that the evidence that is coming to them is data-informed and has methodological rigour.
MLD: What excites you most about the role of digital and data in transforming government?
ID: The opportunities! I have been a statistician for a very, very long time, and the sorts of questions we can find answers to now I could only have dreamed about, even 20 years ago. Data can help us to understand the ways in which we can improve the lives of our fellow citizens, to improve the way they go about things, to improve the productivity of our economy, and to improve the entire well-being of our society.
The power of generative AI to support government is unlimited; it’s absolutely a fantastic opportunity
MLD: What’s your favourite new or emerging technology?
ID: Data linkage, no question. The fact that now, you can link data very smoothly, very easily, so long as of course you’ve got something to link it with is, to me, a marvel. The second thing is the technology to be able to estimate statistical models much more quickly. Much of what we do is through a systems diagram, because we work in multivariate, multidimensional, multidisciplinary teams. When I used to work out how to estimate systems diagrams in the mid-1990s, it took us months to estimate one diagram, but you can do it now in an afternoon. Technology means that we really can undertake the statistics quite quickly to impact on policy and that means that we can be ever more relevant.
October 2019
Date of Sir Ian Diamond’s appointment as national statistician; he began a second five-year term in April 2023
Integrated Data Service
The new government-wide service being delivered by the ONS to allow thousands of accredited researchers access to de-identified data sets
88.9%
Proportion of households in England and Wales that completed the 2021 census online – far ahead of the 75% target
2,945
Number of people in government’s statistics profession – who often work in concert with 23,155 colleagues in the DDaT profession
‘Local, longitudinal, linked’
The ‘three Ls’ that the ONS’s new data strategy believes will be ‘crucial features of the use of data going forward’
MLD: To be able to easily link data in hours not months, you do need some of the important foundational elements of data management, standardisation and data quality in place. What’s your view on that?
ID: I could not agree more. The first thing we need to do is to focus on what are the questions that we are trying to answer. If you understand those questions, then you can say: do we have appropriate data to address those questions? And do those data really address those questions? That is methodological rigour. For example, sometimes we use administrative data for research purposes, but they are not collected to answer research questions, they’re collected for administrative purposes, which is absolutely right and proper. And therefore we need to ask the question whether we can use them or whether we need to augment them. Or maybe we need to link data from different purposes to make the data more useful. It’s important to spend the time getting that right.
And more broadly, your point about data management is spot on. The great unsung heroes these days are data architects and data engineers. They often don’t get recognised at the level that they should, but you cannot do anything with data without a really top-class data architect. We should be celebrating top-class data engineers and their expertise needs to be valued as the heart of everything that we do.
MLD: I suppose that’s an important message for senior civil servants who are leading big organisations – whether they’re using data for operational or policymaking use cases – is to ensure that you have the right set of skills in your team and that your architects and engineers are empowered and celebrated.
ID: The very first statistics lecture I went to covered that your data must be cleaned and in good shape. Because data are so much more available now, it’s easy to forget important stuff such as ensuring that we are collecting the data in the right way to answer the question.
MLD: And that is particularly important in the world of generative AI when you’re getting data that’s been produced by an algorithm. What are your views on the power of generative AI?
ID: I think the power of generative AI to support government is unlimited. It’s absolutely a fantastic opportunity. For example, in the ONS we’ve already developed something called StatsChat. It’s brilliant and absolutely groundbreaking stuff, but I think we’re only just scratching the surface of what can be done.
But at the same time, I believe there will always be a real need for top-class digital people and top-class statisticians. Top-class digital people are needed to make sure that what you are asking the computer to do is grounded in good theory, and a good understanding of what data can do, and that what comes from AI is sensible and interpretable. Generative AI has unlimited potential, but we need to make sure that the questions that are being asked can be answered in the right way.
MLD: Yes, it’s important that we understand the limitations of generative AI in order to be intelligent users of it.
ID: It’s incredibly important that all senior civil servants have that understanding and confidence to ask questions of their director of analysis or their chief data officer who can speak to the quality of evidence created by AI.
Being a digital leader is about going much further than simply thinking about data themselves, but actually how they are going to improve the lives of all of our fellow citizens
MLD: What’s your department achieved in the last six to 12 months that you are especially proud of when it comes to digital and data?
ID: We will in November, for the first time, present research demonstrating how net migration can be disaggregated by the reasons for people coming so that we will be able to see the number of people who’ve come as students, against all other net migration.
So we will be able to do that. And being able, on a really regular basis, to use digital techniques to understand the number of people in different parts of the country, in a much more reliable and a much more timely way is, I think is absolutely brilliant.
We will then over the next year or so explore the use of mobility data to be able to answer for local authorities a question they’re always asking us, which is not just how many usual residents they have, but how many citizens they have to serve, shall we say, on a Wednesday at lunchtime? Those kinds of estimates, I think, are absolutely brilliant.
MLD: Fantastic. And finally: what do you hope we will have achieved by 2025?
ID: I really hope that the fabulous pace that CDDO have really led over the last couple of years will mean that we are truly a civil service that is data and digital led. I hope that Data Marketplace is a core tool for all civil servants and that the Integrated Data Service is able to provide linked data in a really rapid and ethical way, and that as a result, we have many examples of places where the digital community have improved the lives of our fellow citizens.
MLD: What a great thought to end our interview on, thank you so much, Ian.
PublicTechnology will publish lots more in our series of Digital Leaders’ Download interviews over the coming weeks and months, featuring exclusive conversations between Megan Lee Devlin and senior leaders including HMRC chief executive Jim Harra, Home Office permanent secretary Matthew Rycroft, and HM Treasury’s Cat Little, head of government’s Finance Function.
Catch up here on our interview with Lee Devlin – including insights into eliminating departments’ biggest legacy risks, unlocking £8bn of technology investment, and the rollout so far of government’s three-year digital strategy. Or here you can read her discussion with former HM Prison and Probation Service head Jo Farrar, including her thoughts on the potential of virtual reality and AI, and why it’s time for civil servants to stop accepting outdated IT.
