Whitehall’s digital record legacy poses ‘real and immediate risks’ to operations and compliance
Much of the digital information that government has amassed over the past two decades is poorly organised and “almost impossible” to search, a Cabinet Office report has said.
Although Whitehall was good at managing paper records, the government said it had not kept up with digital demands - Photo credit: Pixabay
The report, Better Information for Better Government, said the current state of record-keeping creates “real and immediate risks” for operations, information compliance and the government’s reputation, for instance when responding to inquiries and FOI requests.
It said that there was an “immediate need” to address the situation, and recommended that departments assess the level of risk they face, while teams at the Government Digital Service and The National Archives should focus on preparing government for future digital record-keeping.
Civil service chief John Manzoni urges change as data revolution ‘comes for government’
GOV.UK to take more responsibility for local links as Local Directgov is retired
Unlocking the power of big data
Government information is held in departmental computer systems, databases or paper files, but some of this must be retained long-term in order to maintain the public record, in which case it will usually be transferred to The National Archives.
The government said it had been “generally good” at managing information held on paper – when it resided in only one place and was either destroyed or moved to the archives – but that the shift to digital information “undermined the rigour of information management across much of government”.
This is partly because the move to digital happened at different rates in different departments, meaning that information management was not wholly re-assessed.
Many attempted to use paper practices for digital records, for instance printing out digital records, but this became more difficult as processing power increased and the amount of information generated grew.
Other problems included difficulties in identifying valuable material because emails would also include out-of-office messages and junk mail, and the fact many civil servants began to save information in their own different ways.
“Only now, a decade later, is government beginning to understand the implications of this systemic information management failure,” the report stated.
“While little information has been lost altogether, much of what has accumulated over the past fifteen to twenty years is poorly organised, scattered across different systems and almost impossible to search effectively,” the report said.
This poses problems for government when responding to inquiries – with the report noting that sometimes departments have found important information after an inquiry had finished – and in compliance with data security measures.
It also means that civil servants risk re-inventing the wheel because they can’t access information on previous solutions – the report estimated that this wasted effort could cost the government £500m a year.
Support and culture change
The report – which comes as civil service chief John Manzoni has said Whitehall needs to boost its data expertise to cope with the challenges posed by big data – was published in response to a November 2015 review of government digital records.
That review, carried out by Alex Allan, former chairman of the Joint Intelligence Committee, said that although policies and guidance on capturing and managing information were “sound; the problems come in the implementation”.
He recommended greater co-operation between GDS, The National Archives and departments, and a change in culture across government.
The government report aims to address these concerns, setting out plans to create bespoke support plans for each departments, with the focus being to engage with staff to encourage, rather than mandate, change.
“Our aim should be simply that departments get better at managing their legacy collections and improve current information management by drawing on all the levers at their disposal,” the report said.
Its main recommendations are for departments’ accounting officers and senior leaders to evaluate the risk they face from legacy digital information and “take specific steps” to improve information management.
The most appropriate way for departments to deal with legacy digital collections, it said, would be to embrace data analytics tools to assess which material to get rid of, while making it simpler for staff to save things in the right places in the first place.
Meanwhile, in addition to supporting individual departments’ efforts, the Cabinet Office, The National Archives and GDS will carry out cross-government initiatives to improve information management in the future and increase capability within the civil service.
Over the next six to twelve months, a team from the three organisations will aim to drive change across the whole of government, while retaining a “supportive, collaborative engagement model”.
Commenting on the report, Allan said: “The government has taken the issues I identified very seriously and come forward with a strong set of recommendations which, if put into place by individual departments, should greatly improve digital records management across the civil service.”
Des Ward, information governance director at Innopsis, reflects on the real story behind the WannaCry cyber-attack.
A fresh look at data protection and backup best practice, particularly when it comes to ransomware.
A motion to appoint lawyer Daren Fitzhenry is to be put before the Scottish Parliament
"At the end of the day, the services are all the same, so if you’re doing bins from Shetland to Glasgow to the Borders, we all do similar services"
BT looks at turning points within the UK defence sector, the evolving nature of warfare and how new cyber-attacks pose new questions for our national defence
BT's Phil Brunkard on technological innovation and how it's affecting the public sector
BT's Phil Brunkard on brain implants, parking spaces, and takeaways from BT Innovation Week
BT finds that IT Directors disagree over whether Artificial Intelligence will create or displace jobs