DH trials algorithm tool to analyse consultation responses in effort to handle ‘click democracy’

Written by Rebecca Hill on 9 January 2017 in News

Whitehall needs to make better use of machine learning to make sense of the deluge of information generated by greater online participation, the digital strategy manager at the Department of Health has said.

'Digital avalanche' of data from online consultation and petitions needs to be tackled with algorithms - Photo credit: Pixabay

“The rise of digital social movements…has led more people to use digital tools to connect with government more quickly and in ever greater numbers,” Laurence Erikson wrote in a blogpost.

He said that, although greater participation should be “good news for government”, every webchat, online consultation form and petition promoted on social media and over email, “there’s someone on the other end who needs to read and make sense” of the inputs.

Describing this as a “digital avalanche”, Erikson argued that they could be a useful way of making better use of people’s opinions, assuming that the civil service has the “methods and the tools to cope”.

One answer, he said, may lie in machine learning algorithms that can be trained to spot patterns from data – and so used on large datasets to analyse unstructured text.

Related content

Changing the culture: What government must do to make the most of AI
Google’s DeepMind and NHS restart data-sharing deal with greater transparency
GDS expands data science training programme for civil servants

Erikson said that DH had been trialling a learning algorithm tool to help analyse public consultation responses, starting by carrying out a controlled trial on a consultation with a lot of digital responses, which the team compared with the manual analysis.

“The findings so far are intriguing,” Erikson said. “We found that the machine learning approach reinforced some of the findings of the manual approach, but also identified new insights from the consultation responses.”

These included being able to look at the issues that were consistently raised across different questions, and being able to better understand what different groupings of audiences said with the algorithm approach.

The value of these insights increased as more data was fed in, he added. However, he noted that this still required “lots of human input” and still took time.

Erikson’s comments and his department’s trial of machine learning to analyse consultation responses echo moves made by the Government Digital Service’s GOV.UK team to increase use of machine learning to classify user comments.

Last month, Matt Upson said that the GOV.UK team had automated some of the process for analysing the responses to one of its user surveys, which asks people about their journey through GOV.UK and gets around 3,000 responses a month.

“Until now, a group of volunteers from various GOV.UK teams have manually classified the responses into a number of pre-defined classes,” Upson said. It takes volunteers around 45 minutes to assign classes – such as ‘ok’, ‘check-status’ and ‘compliment’ – to the free text sections for 100 surveys.

This, he noted, would be a “significant investment of time” if the team was to classify all 3,000 each month. He said around 4,000 surveys have been manually classified to date.

However, because algorithms need a lot of pre-classified samples to begin with, and there is a lot of variation in the free text people use, “it could take us several years before we have sufficient data at the current rate our volunteers are able to classify them”.

Instead, the team used the machine learning process to identify the largest class – ‘ok’ – by training it to base its assessment on a combination of features in the free text.

“After generating newer, simple features and curating a more objective dataset, our machine learning algorithm was able to identify 'ok' responses 88% of the time,” Upson said. “This saves a huge amount of time for our volunteer classifiers, although there is still room for improvement.”

In the cases where surveys that should have been classified in that way “slipped through the net”, it was correctly classified by the volunteer coders and fed back into the algorithm to improve its accuracy.

Upson said that it would probably be impossible to entirely automate the process, but that this wasn’t the end-goal because the volunteers come from different teams, which makes sure the feedback is returned directly to the teams working on the relevant parts of GOV.UK.

Share this page



Please login to post a comment or register for a free account.

Related Articles

National police systems to be moved to AWS
16 August 2018

Home Office seeks suppliers for one-year project to migrate services from datacentre to public cloud

GDS working with departments to mitigate reported leak of sensitive data
23 July 2018

Report reveals that information has been made publicly available online via an information-sharing tool widely used by government developers

Hull to trial smart bins
18 July 2018

Council hopes pilot will show potential for enabling more efficient routes and timetables

Related Sponsored Articles

Intelligent Connectivity: Boosting Flexibility and Control
13 August 2018

At BT, we realise that digital technology is changing the way we all do business. Make smart decisions with intelligent connectivity.

BT: Intelligent Connectivity is where it all begins. Smarter decisions are the end result
7 August 2018

At BT, we realise that digital technology is changing the way we all do business. Make smart decisions with intelligent connectivity.

Building nation-level defences to fight cyber crime
30 July 2018

BT's Mark Hughes argues that nation states should act now to put in place cyber defences to protect themselves from the most advanced threats ever seen.

The inaugural Chamberlain Lecture at the iconic BT Tower
23 July 2018

On the 4th June 2018, BT partnered with Cratus Communications and Public Policy Projects for the inaugural Chamberlain Lecture given by Lord Heseltine at BT Tower.