New set of guidelines aims to help not only expedite tech adoption, but to position the UK as a leader in the ‘stewardship of public data as critical national infrastructure’
The government has published a framework of measures intended to help public bodies ensure their datasets are ready and able to interoperate with artificial intelligence tools.
The guidance – which was created by the Department for Science, Innovation and Technology – is aimed at addressing the “true constraint” currently stymieing AI adoption: the data that underpins artificial intelligence systems.
The introduction to the notes that “in government, data collection has often prioritised operational delivery, reporting, or compliance, while considerations around reusability for advanced analytics or AI have sometimes received less attention”.
The document is intended for use by both data and AI professionals, as well as senior managers and policymakers. The guidelines set out by DSIT are intended to provide these stakeholders with “clear steps for preparing datasets to support various AI capabilities and promoting responsible data stewardship”.
The backdrop to this preparation is the government’s belief that “the value of any AI capability generally depends on… key questions”, the first and foremost of which is: “does it have the right data?” .
To help ensure that it does, the framework outlines 10 guiding principles for public bodies to adopt, grouped across four areas defined as “foundational pillars”: technical optimisation; data and metadata quality; organisation and infrastructure context; and legal, security and ethical compliance.
The area of technical optimisation encompasses the principles of data granularity, data complexity and diversity, and scale and performance.
In concert, these principles are intended to help support “the practical requirements of scale, interoperability, performance, and integration, recognising that AI capabilities place significantly different demands on data infrastructure compared to traditional reporting or statistical use”.
For the second of the four pillars, there are three further principles: data quality; data as a strategic product; and business logic and context.
“This pillar starts from the premise that organisations seeking to apply AI must first be able to clearly articulate what data they need, for what purpose, and at what level of granularity,” the guidance says. “Teams should be supported to discover what datasets already exist (for example through data marketplaces, catalogues, or registries) and to engage early with data owners and specialists to define access requirements, sensitivities, and intended uses.
The document adds: “This upfront clarity is critical not only for technical feasibility, but also for establishing lawful basis, scoping DPIAs, and setting appropriate governance controls. Building on this, the pillar addresses the trustworthiness and interpretability of datasets covering three AI-ready data principles. AI capabilities rely on both data values and metadata to learn effectively and to produce outputs that can be explained, audited, and defended.”
The third pillar, which concerns organisational and technical context, includes three further principles: data governance; information sharing and collaboration skills; and documentation and guidelines.
This set of guiding tenets “recognises that AI-ready data cannot be achieved through technical measures alone”.
“Organisational commitment, clear roles, and sustainable infrastructure are essential to maintaining readiness over time,” the framework adds.
In the final pillar, there is only a single unified principle for organisations to follow: to ensure security and compliance.
Related content
- New government chief data officer Aimee Smith starts role
- HMRC expands AI use to narrow revenue gap and ‘identify nascent issues with tax system’
- New digital centre of government prioritises ‘value of data and sustainable talent supply’
“Given the sensitivity of many public sector datasets and the potential impact of AI driven decisions, this pillar is foundational to maintaining public trust,” the document says. “Departments must implement appropriate technical and organisational controls, including role-based access control, encryption of data at rest and in transit, and comprehensive audit logging. These controls should apply consistently across file-based access, APIs, and analytical environments.
It adds: “Legal compliance requires clear evidence of the lawful basis for data processing, particularly where datasets may be reused for AI training or inference. departments should complete and maintain Data Protection Impact Assessments where required, documenting risks, mitigations, and decision rationales.”
Across each of the pillars and principles, the guidance goes on to set out a range of measures bodies can take, and includes a checklist that covers all areas and can be completed “to help assess dataset suitability for proof of concept to production grade AI use cases”.
‘Opportunistic to sustainable’
The conclusion to the guidelines begins with the observation that “the UK public sector stands at a pivotal moment in the evolution of artificial intelligence in government”.
“While the availability of advanced models and platforms continues to accelerate, these guidelines demonstrates that the true constraint on responsible, scalable AI adoption is not algorithms, but data: its quality, structure, governance, and legitimacy,” the conclusion says. “Releasing datasets for AI use is therefore not a technical publishing exercise, but a strategic act of public stewardship.”
It adds: “Critically, these guidelines position AI-ready data as a managed, end-to-end capability. Departments that are most advanced are those that treat datasets as products with named owners, explicit legal bases, quality obligations, and user support; that embed ethics, security, and risk appetite at the outset; and that recognise AI capabilities as continuing public services rather than finite technology deployments. Where this mindset is absent, early successes risk creating long term operational, legal, and reputational exposure. By adopting the principles, pillars, and action practices set out in this document, UK government organisations can move from opportunistic AI experimentation to a sustainable national capability for responsible AI. This will enable departments not only to release data more effectively, but to do so in a way that strengthens interoperability, accelerates safe innovation, and reinforces public trust. In doing so, HMG has the opportunity to establish global leadership not merely in AI application, but in the stewardship of public data as critical national infrastructure for the age of intelligent systems.”
