DfT uses machine learning to create news round-up tool for policymakers

Written by Sam Trendall on 31 August 2018 in News

Program trained to identify stories likely to be of interest to policy professionals

Credit: Sollok29/CC BY-SA 4.0

The Department for Transport has created a machine learning-powered tool for providing its policymakers with a roundup of relevant news stories.

To keep its policy professionals up to date with news stories in which they might be interested, the department has previously relied on tools that simply performed keyword searches on articles published by the UK’s 700-plus local and national news publications. The lack of sophistication of this model invariably leads to a number of irrelevant stories being flagged up, said DfT data scientist Will Bowditch in a blog post.

To sift through news archives more effectively, the DfT’s data science team embarked on a project to “teach a computer” to read through newly published news stories and identify which ones were likely to be of interest to policymakers. The prototype that has been built uses basic filters suggested by the policy experts themselves. It then relies on “machine-learning algorithms to transfer the heavy processing to the computer”, Bowditch said.

Related content

He added: “Computers aren’t equipped to deal with language, but they can process increasingly vast amounts of numbers. To convert text into something a computer can digest we convert sentences into a matrix of word counts, referred to as a ‘bag of words’. If we do this for a selection of words… we can begin to spot patterns in the data.”

If data scientists can identify a number of articles that they know were useful to a particular policy unit, the words used in these stories can be used to “train” a program to identify similarly relevant articles in the future.

“The process involves calculating the probability that a word, for example ‘park’, will appear in documents we’ve tagged as belonging to a [policy] team,” Bowditch said. “And using those probabilities, [the tool] can calculate the most likely category for new documents. We can see from the word counts table the probability that articles with the word ‘park’ belong to the parking category is much higher than the probability for the word ‘electric’. In reality, we apply this process to thousands of words and hundreds of articles.”

The News Roundup service is currently in its alpha stage, and is still being managed by human experts. But the long-term ambition is “to make the tool self-sufficient”. This could be achieved by allowing people who receive emails sent by the program to vote on whether the stories provided were relevant or not – and using this voting information to further hone the tool’s algorithms.

DfT’s data-science team has compiled a record of 40,000 articles with transport-related keywords and is now engaged in “exploring other ways we can use this data”, Bowditch said. Anyone interested in taking part in such a project can email the team by clicking here.

About the author

Sam Trendall is editor of PublicTechnology

Share this page




Please login to post a comment or register for a free account.

Related Articles

HMRC seeks digital platform for field fraud officers
15 August 2019

Tax agency issues procurement alert for ‘notebook application’ for evidence-gathering

Can the GDS innovation strategy deliver a lasting legacy for government?
14 August 2019

Government's new Innovation Strategy set out ambitious proposals to update processes, eliminate ageing kit, and embrace emerging technologies. PublicTechnology caught up with...

Government to build digital service to connect top public sector leaders
13 August 2019

Recently launched National Leadership Centre to create online platform allowing leading officials to seek each other out

Related Sponsored Articles

Why you need to understand your application landscape before you transform your network
13 August 2019

Migrating to the cloud or moving to a future network can be a risky business. BT explains how managing applications is important for end user experience, productivity and for understanding and...

Build the network your business needs to thrive
6 August 2019

BT presents a new eGuide, looking at how to build infrastructure able to support growth both now and into the future 

Trends to Watch: Network Services
30 July 2019

BT spoke with Ovum's Brian Washburn about the network trends taking place in 2019, covering SD-WAN, NFV, hybrid networking and cloud connectivity services 

How CIOs are shaping their networks for a digital future
23 July 2019

BT walks through how CIOs are riding the wave of new network technology like SD-WAN, NFV, Cloud and Hybrid to shape their network for a digital future