How The National Archives is digitising 1,000 years of history

Written by Sam Trendall on 5 October 2017 in Features

Digital director John Sheridan tells PublicTechnology how new technology is changing not only how the UK’s official archive operates, but is altering the very nature of archiving

The National Archives has about 600 staff - one in six of whom are focused on digital  Credit: PA

There can be few, if any, public bodies for whom the concept of ‘digital transformation’ will be quite so transformative as The National Archives.

The organisation, a non-ministerial government department sponsored by the Department for Digital, Culture, Media and Sport, manages hundreds of millions of documents spanning 1,000 years of British history. Its headquarters in Kew, in west London, requires well over 100 miles of shelving to store all these records – with another mile needed each year.

All of which adds up to a pretty hefty digitisation project.

Of course, not all of those miles and miles of historic documents can be digitised, and the organisation is not attempting to put a millennium of data online in one fell swoop.

But, in the last few years, the organisation has redoubled its focus on digital platforms and services, beginning in 2015, when it laid out a four-year plan to “become a digital archive by design”. The same year, head of legislation services John Sheridan was promoted to become The National Archives’ first digital director, taking his place in a six-strong senior-management team.

This year brought the publication of an updated digital strategy for the period up until 2019. The plan encompasses three phases on the way to achieving its digital ambitions: Reshape; Grow; and Accelerate. The Reshape stage has been successfully completed, and the Grow programme will run until late next year, before 2019 brings the Accelerate agenda.

The organisation’s main digital service for the public is Discovery, which allows users to search descriptions of 32 million records held not only by The National Archives, but by a further 2,500 archives across the UK. Nine million of these records are already available for online download.

Related content

In the 2016/17 year – across Discovery and the archives’ other services – almost 230 million individual documents were delivered to the public online, compared with a little under 600,000 that were supplied in physical form.

The work to digitise records has been taking place for about 15 years. And, although hundreds of millions of documents are now available digitally, only “between 5% and 6%” of The National Archives’ records have already been digitised, Sheridan tells PublicTechnology.

The process has been partly funded by grants, but has primarily come as a result of members of the public – or, more commonly, the genealogy websites that have risen to popularity in recent years – paying the archives to create digital copies of records.

“We have a really successful model for digitisation, which is commercially led – that may well be by a genealogy company, or a company that is developing a product,” Sheridan says. “We will grant you the right to digitise the collection, we will facilitate that, and you will give us the [copy].” 

He adds: “We are then free to do what we like in terms of [usage], and we get royalty payments as well. It creates a really sustainable model of digitising the records –  and we are not spending any of our money on digitising the collection. We have a model that works particularly well for large-scale digitisation – we are doing that through our partners.”

Being digital by design means rethinking some big ideas that have been part of the archival process for a century or more

While digitisation brings opportunities to open up and disseminate archival information in a much broader way, it also presents challenges in how collections are maintained and presented, Sheridan says. The way in which paper archives are kept has, in essence, changed little for a couple of centuries. But the means of storing and protecting digital items changes every few years. 

“Being a digital archive, as opposed to being a physical archive, is a big deal, and a different deal,” he says. “In a physical archive, records are provided to us in a form that we can preserve and present. With digital, we cannot easily or readily present them. And we have to keep doing that preservation work with each generation of technological change. We have to make decisions to maximise the benefits and the capability [for years to come].”

Sheridan adds: “The big thing to understand with digital records is that they are going to outlive the digital-preservation system. [We need to consider] how do I get the records out this thing, and onto the next thing? We are very clear: the record is the thing that will outlive the system.”

Context and caution
At the moment, The National Archives is a “first-generation digital archive”, Sheridan says, meaning that so far it has solely been digitising existing physical records. The next stage of the organisation’s evolution will bring much more elemental change.

“We are a first-generation digital archive; we are implementing, in a digital way, the archival practices that were designed for managing physical records,” he says. “But a big part of our strategy is not just being a first-generation digital archive, but being a second-generation digital archive that is designed as digital.”

Sheridan adds: “It means rethinking some big ideas that have been part of the archival process for a century or more, and how we understand the evidence of the record, and how we evidence the life of the record.”

Percentage of The National Archives’ records that have been digitised to date

229.8 million
Number of documents supplied to the UK public online last year – compared with fewer than 600,000 supplied in physical form

Year to which The National Archives can trace its
history, when the Her Majesty’s Stationery Office was founded

West London location of The National Archives headquarters, and its physical archive

Four years
Duration of its plan to become ‘digital by design’, spanning the period from 2015 to 2019


The key to understanding a record’s worth and meaning has long been the context in which it is archived and presented, according to Sheridan.

“Context has always been the archivist’s special sauce – how do we display a record so that its evidential value can be understood? That is something that the [commercial] marketplace is not addressing,” he says.

And, so, The National Archives – alongside the University of Surrey, the Open Data Institute, and tech companies Methods, Guardtime, and Voetek – is currently working to try and develop distributed-ledger technology that ensures the historic sustainability of digital archives. The 18-month project, which is dubbed ARCHANGEL, has been funded by the Engineering and Physical Sciences Research Council.

Alongside its investments in technology, the organisation is also significantly boosting its digital headcount. Currently about 100 staff – one in six of its total – are focused on digital. Over the next two years this figure is expected to grow to 150, Sheridan says. 

A key part of this growth will be developing its own talent via an apprenticeship scheme. The National Archives has already taken on five apprentices, and is the only public-sector body currently taking part in the programme run by the Ada National College for Digital Skills, which was founded last year.

This influx of young minds and new ideas will, perhaps, be particularly beneficial in a profession that Sheridan says can tend towards staidness.

“Archivists are steady Eddies,” he adds. “But, with the challenge of digital archives, they have no choice but to evolve their practice, and to move from relatively slow evolution, to much more rapid development.”


About the author

Sam Trendall is editor of PublicTechnology

Share this page




Please login to post a comment or register for a free account.


David Matthew (not verified)

Submitted on 7 October, 2017 - 01:28
It is not quite accurate to say that the percentage of records that have been digitised or have been produced digitally rather by original documents as it gives a misleading impression and the comparison is not like-for-like. Many of the documents that have been digitised (e.g. soldiers documents) are about 200 documents to each file, so individual soldiers documents would count s 200 whilst the original box of documents was originally listed as one document. I would doubt even the figure of 5-6% of documents being digitised and they will unlikely to ever be because of the economic cost or from a preservation point of view they cannot.

Mark Conrad (not verified)

Submitted on 10 October, 2017 - 19:56
Great article. Many good ideas for large scale digitization projects. What is the National Archives doing about born digital records of the UK?

Jim Brennan (not verified)

Submitted on 3 November, 2017 - 01:39
If the National Archives has been stuck with a partnership model which is heavily biasses in favour of the digitalising private sector partner that's not its fault, but it should not allow its partners to present the digitised material to users in a way that maximises the profit opportunity, or inhibits the correction of indexing error or allow them the easy option of choice of easy access to more obviously lucrative material. Currently, for example, users of digitised genealogical information from Caribbean parish registers or slave returns have to suffer all of these consequences of ill-thought out partnership agreements, and the National Archives is clearly not in the least interested in ensuring that the taxpayers who fund it get proper online access to tis material It would be nice to know that more thought is being given to this in future agreements. And Discovery itself shows no sign of improving the cataloguing of existing undigitised holdings, sometimes of collections well over a century and more old. It would be reassuring to know exactly what future priorities are in relation to content, rather than just to be fed press releases about process.

Related Articles

ICO issues warning over ‘emotion analysis technologies’
31 October 2022

Regulator claims new systems come with inherent risk of ‘systemic bias, inaccuracy and even discrimination’

Home Office seeks software developers to meet ‘daring targets’ for use of automation
2 December 2022

Department looks for trio of experts to join specialist team

NHS Digital cites transformation ‘turning point’ after £750m investment in FY22
17 November 2022

Annual report of tech agency – which will shortly be wholly merged into NHS England – details progress in many areas

DWP looks to embed machine-readable laws into digital ‘Universal Credit navigator’
24 October 2022

Department works with government lawyers and specialist tech firm to pursue ‘rules as code’ project