DH trials algorithm tool to analyse consultation responses in effort to handle ‘click democracy’

Written by Rebecca Hill on 9 January 2017 in News

Whitehall needs to make better use of machine learning to make sense of the deluge of information generated by greater online participation, the digital strategy manager at the Department of Health has said.

'Digital avalanche' of data from online consultation and petitions needs to be tackled with algorithms - Photo credit: Pixabay

“The rise of digital social movements…has led more people to use digital tools to connect with government more quickly and in ever greater numbers,” Laurence Erikson wrote in a blogpost.

He said that, although greater participation should be “good news for government”, every webchat, online consultation form and petition promoted on social media and over email, “there’s someone on the other end who needs to read and make sense” of the inputs.

Describing this as a “digital avalanche”, Erikson argued that they could be a useful way of making better use of people’s opinions, assuming that the civil service has the “methods and the tools to cope”.

One answer, he said, may lie in machine learning algorithms that can be trained to spot patterns from data – and so used on large datasets to analyse unstructured text.

Related content

Changing the culture: What government must do to make the most of AI
Google’s DeepMind and NHS restart data-sharing deal with greater transparency
GDS expands data science training programme for civil servants

Erikson said that DH had been trialling a learning algorithm tool to help analyse public consultation responses, starting by carrying out a controlled trial on a consultation with a lot of digital responses, which the team compared with the manual analysis.

“The findings so far are intriguing,” Erikson said. “We found that the machine learning approach reinforced some of the findings of the manual approach, but also identified new insights from the consultation responses.”

These included being able to look at the issues that were consistently raised across different questions, and being able to better understand what different groupings of audiences said with the algorithm approach.

The value of these insights increased as more data was fed in, he added. However, he noted that this still required “lots of human input” and still took time.

Erikson’s comments and his department’s trial of machine learning to analyse consultation responses echo moves made by the Government Digital Service’s GOV.UK team to increase use of machine learning to classify user comments.

Last month, Matt Upson said that the GOV.UK team had automated some of the process for analysing the responses to one of its user surveys, which asks people about their journey through GOV.UK and gets around 3,000 responses a month.

“Until now, a group of volunteers from various GOV.UK teams have manually classified the responses into a number of pre-defined classes,” Upson said. It takes volunteers around 45 minutes to assign classes – such as ‘ok’, ‘check-status’ and ‘compliment’ – to the free text sections for 100 surveys.

This, he noted, would be a “significant investment of time” if the team was to classify all 3,000 each month. He said around 4,000 surveys have been manually classified to date.

However, because algorithms need a lot of pre-classified samples to begin with, and there is a lot of variation in the free text people use, “it could take us several years before we have sufficient data at the current rate our volunteers are able to classify them”.

Instead, the team used the machine learning process to identify the largest class – ‘ok’ – by training it to base its assessment on a combination of features in the free text.

“After generating newer, simple features and curating a more objective dataset, our machine learning algorithm was able to identify 'ok' responses 88% of the time,” Upson said. “This saves a huge amount of time for our volunteer classifiers, although there is still room for improvement.”

In the cases where surveys that should have been classified in that way “slipped through the net”, it was correctly classified by the volunteer coders and fed back into the algorithm to improve its accuracy.

Upson said that it would probably be impossible to entirely automate the process, but that this wasn’t the end-goal because the volunteers come from different teams, which makes sure the feedback is returned directly to the teams working on the relevant parts of GOV.UK.

Share this page


Add new comment

Related Articles

Lack of digital skills biggest hurdle for governments to overcome, Gartner analysis finds
26 April 2017

Research vice president says public sector needs to embrace start-ups and move workforce out of low-value repetitive...

TechUK: City mayors should commit to making digital the 'new norm'
25 April 2017

Industry body calls on mayors to appoint a chief digital and innovation champion and use their influence to boost smart...

Don’t abuse stats during election campaign, UK Statistics Authority chair tells party leaders
24 April 2017

David Norgrove writes to ask parties to make sure their campaign materials don’t mislead the public

Robotics and AI get £93m in Industrial Strategy Challenge Fund
24 April 2017

Business secretary Greg Clark announces spending breakdown for first areas to receive investment through the fund...

Related Sponsored Articles

UK SMEs showcase projects in competition final to help millions of businesses and citizens stay safe from online crime
19 April 2017

BT, TechHub and the Cabinet Office have announced the winners of their Securing the Nation competition at an event at the iconic BT Tower

BT appoints senior executive to lead public sector business in London and the South East
5 April 2017

BT has appointed a new senior executive, Mark Sexton, to head up its public sector business in London and the South East and implement a new strategic direction to increase its local presence

BT appoints senior executive to lead Public Sector business in Scotland
27 March 2017

BT has appointed a new senior executive, David Wallace, to head up its public sector business in Scotland and implement a new strategic direction to increase local focus nationwide