Whitehall’s digital record legacy poses ‘real and immediate risks’ to operations and compliance
Much of the digital information that government has amassed over the past two decades is poorly organised and “almost impossible” to search, a Cabinet Office report has said.
Although Whitehall was good at managing paper records, the government said it had not kept up with digital demands - Photo credit: Pixabay
The report, Better Information for Better Government, said the current state of record-keeping creates “real and immediate risks” for operations, information compliance and the government’s reputation, for instance when responding to inquiries and FOI requests.
It said that there was an “immediate need” to address the situation, and recommended that departments assess the level of risk they face, while teams at the Government Digital Service and The National Archives should focus on preparing government for future digital record-keeping.
Civil service chief John Manzoni urges change as data revolution ‘comes for government’
GOV.UK to take more responsibility for local links as Local Directgov is retired
Unlocking the power of big data
Government information is held in departmental computer systems, databases or paper files, but some of this must be retained long-term in order to maintain the public record, in which case it will usually be transferred to The National Archives.
The government said it had been “generally good” at managing information held on paper – when it resided in only one place and was either destroyed or moved to the archives – but that the shift to digital information “undermined the rigour of information management across much of government”.
This is partly because the move to digital happened at different rates in different departments, meaning that information management was not wholly re-assessed.
Many attempted to use paper practices for digital records, for instance printing out digital records, but this became more difficult as processing power increased and the amount of information generated grew.
Other problems included difficulties in identifying valuable material because emails would also include out-of-office messages and junk mail, and the fact many civil servants began to save information in their own different ways.
“Only now, a decade later, is government beginning to understand the implications of this systemic information management failure,” the report stated.
“While little information has been lost altogether, much of what has accumulated over the past fifteen to twenty years is poorly organised, scattered across different systems and almost impossible to search effectively,” the report said.
This poses problems for government when responding to inquiries – with the report noting that sometimes departments have found important information after an inquiry had finished – and in compliance with data security measures.
It also means that civil servants risk re-inventing the wheel because they can’t access information on previous solutions – the report estimated that this wasted effort could cost the government £500m a year.
Support and culture change
The report – which comes as civil service chief John Manzoni has said Whitehall needs to boost its data expertise to cope with the challenges posed by big data – was published in response to a November 2015 review of government digital records.
That review, carried out by Alex Allan, former chairman of the Joint Intelligence Committee, said that although policies and guidance on capturing and managing information were “sound; the problems come in the implementation”.
He recommended greater co-operation between GDS, The National Archives and departments, and a change in culture across government.
The government report aims to address these concerns, setting out plans to create bespoke support plans for each departments, with the focus being to engage with staff to encourage, rather than mandate, change.
“Our aim should be simply that departments get better at managing their legacy collections and improve current information management by drawing on all the levers at their disposal,” the report said.
Its main recommendations are for departments’ accounting officers and senior leaders to evaluate the risk they face from legacy digital information and “take specific steps” to improve information management.
The most appropriate way for departments to deal with legacy digital collections, it said, would be to embrace data analytics tools to assess which material to get rid of, while making it simpler for staff to save things in the right places in the first place.
Meanwhile, in addition to supporting individual departments’ efforts, the Cabinet Office, The National Archives and GDS will carry out cross-government initiatives to improve information management in the future and increase capability within the civil service.
Over the next six to twelve months, a team from the three organisations will aim to drive change across the whole of government, while retaining a “supportive, collaborative engagement model”.
Commenting on the report, Allan said: “The government has taken the issues I identified very seriously and come forward with a strong set of recommendations which, if put into place by individual departments, should greatly improve digital records management across the civil service.”
Paul Maltby claims councils must first renew ageing infrastructure before realising the benefits of machine learning and automation
Both the government and human rights group Liberty claim victory after judges agree that the so-called snoopers' charter is incompatible with EU legislation
Move follows the earlier transfer of data policymakers
Department reportedly provides list of 39 technology and other reform programmes that are being paused or delayed
Calm has turned a section of the 57,509-word EU document into a sleep-inducing audio book
BT's Konstantinos Karagiannis explains ethical hacking and why it's important to exploit vulnerabilities
The Federation for Small Businesses has urged regulators be patient as firms try to comply with the General Data Protection Regulation
BT answers some common questions on the new data privacy laws that come into force on Friday