To ensure they can effectively contribute to the new Integrated Data Service, Whitehall agencies will need support for necessary work such as data cleansing, according to Professor Sir Ian Diamond
A lack of funding for departments could curtail their contribution of data to the new cross-government platform for government analysists, national statistician Professor Sir Ian Diamond has told MPs.
Diamond, who is also permanent secretary of the Office for National Statistics – which is leading on the £228.7m Integrated Data Service programme – said work was ongoing with chief data officers to “minimise frictions” related to the new “cloud native” platform. His observations came in a letter to members of the Public Administration and Constitutional Affairs Committee dated 18 December 2023, but only published last week.
The IDS platform has been created for accredited people – including an estimated 14,000 government analysts – with a view to better informing effective policymaking. It is currently in beta mode, and due to launch fully by 2025.
In his letter to PACAC, Diamond said the platform has been designed to address the lack of a central integration platform that can cater for the future needs of both data providers and analysts looking to make use of integrated data to develop cross-cutting analytical results. He added that the vision for IDS was for it to be a “rich and diverse data catalogue” with “indexed and linkable data, with the latest provision of data science and generative AI potential”.
Diamond said that at the time of writing there were 81 datasets available in the IDS, with “high-value data assets” on policy areas such as levelling up, climate change and net zero, and work on further datasets was under way.
But he also noted that, while departments have agreed to open up their essential shared data assets across government, in line with the Central Digital and Data Office’s roadmap for 2022-25, some officials fear political and financial pressures will affect departments’ behaviour.
Related content
- Experts voice fears over loss of government data experts
- NHS to hold ‘national conversations’ with public over patient data use
- Digital Leaders’ Download: National statistician Ian Diamond on how data can provide ‘answers we could only have dreamed about 20 years ago’
“Discussions with government analysts have highlighted a range of concerns about how current incentives for departmental data sharing fit with the needs of ministerial-facing departments,” he said. “There is also a wider financial risk regarding other departments’ ability to fund activity such as data cleansing, which may limit their ability to effectively share data.”
Diamond said that while the Treasury had set out the expectation that government departments would “support data sharing” in all 2021 Spending Review settlements, “no specific funding was provided, which may limit activity in some cases”.
The national statistician said the ONS was working with chief data officers across government to smooth data-sharing concerns related to IDS.
“One of the pilots in development is looking at data ownership and stewardship approaches to streamline the governance arrangements and make it quicker for departments to agree to share data via IDS, and for analysts to subsequently access that data for a broad range of analysis in the public good,” he said. “I would welcome support from the committee to share and promote the benefits of data sharing across government for the public good.”
Diamond’s letter to PACAC followed an evidence session for the committee’s Transforming the UK’s Evidence Base inquiry in September last year.
PublicTechnology recently interviewed ONS chief data officer officer Fiona James, who discussed how the Integrated Data Service is at the “front and centre” of the organisation’s recently-published Data Strategy.
