‘Unless you get the ethics of data right, you'll fail on the effectiveness as well’
The head of the Royal Statistical Society Stian Westlake talks algorithms, politicians and the current ‘favourable wind’ gathering in government behind his profession
Stian Westlake is more surprised by one of this reporter’s questions than one might expect the chief executive of the Royal Statistical Society to be. It is, of course: what is his favourite official statistic?
But after some thought, he provides an answer – prefaced with the apologetic caveat that it is “very geeky”.
“The ONS used to publish an information sheet showing the ratio of house prices to the new-build cost of housing. What it shows is – unsurprisingly for anyone who's immersed in the UK property market – it costs an awful lot more to buy a house than to build a house,” he says. “The reason why that's so interesting is because it's the smoking gun for the fact that we have a real problem with the supply of housing and planning in the UK. I think it’s one of the small statistics that sums up a very big debate and points to something that really matters to people.”
Westlake took up the role at the head of the RSS, an industry body representing statisticians and data analysts, in July of this year. He says he has long had a fascination with “the world of data and the use of statistics to make the world a better place”.
“Ministers, like anyone, respond to incentives”
He has spent the last decade at the innovation think tank Nesta, after stints at McKinsey and social investment firm The Young Foundation. At Nesta, he set up the Alliance for Useful Evidence, a network to champion “the smarter use of evidence in social policy and practice", as well as the Innovation Growth Lab, a randomised control trials lab for economic policy, and a data science team.
And in his new post, Westlake says he is keen to help statisticians that want to use their powers for good to make the biggest impact they can. That means ensuring that “in public life, in public debate, we make the best possible use of high-quality statistics that we can”.
His arrival at the RSS this summer has coincided with a growing awareness among the public of the role official statistics play in policy and public services – with recent events serving as a useful reminder. “The news stories around the Covid-19 pandemic have made people realise that actually statistical analysis is really important to how we tackle big challenges – how government can make high-quality policy and how people can hold government to account.”
Asked whether he believes government departments make the best use of the statisticians they employ – and the data they have – he says things “are improving”.
“I think the current government is genuine in its desire to improve how it uses data, and the opportunities it gives statisticians,” he explains – but adds that getting it right is a “real challenge”.
For one thing, he says, using data well requires investment in computing power, data gathering – “as we're seeing with the response to Covid, having spent a massive amount of money building effective test and trace capability” – and skills.
“And investing in people means making sure that we are respecting people who have statistical skills, but also building skills and nurturing people whose job descriptions might not have ‘statistics’ in them. Statistics is important for everyone.”
Transparency and trust
Then again, it is not simply a case of throwing money at the problem, he adds. And it is certainly not simply a case of buying “whizzy technology”.
“It's also about transparency, openness and good data. So, trustworthy statistics have got to be high quality, which is sort of a technical challenge. But it's also got to be appropriately open,” he says.
Essentially, this becomes an ethical and communications issue.
“I think sometimes people put those two things [effectiveness and transparency] in opposition to one another. But actually, unless you get the transparency right, unless you get the ethics of data right, you'll fail on the effectiveness as well.”
He has been thinking about those issues “a hell of a lot” recently. As if to illustrate his point, a few weeks into his new job, an enormous row erupted over the use of a controversial algorithm to award grades for A-Levels, GCSEs and other qualifications, after exams were cancelled because of Covid-19.
Westlake says: “Here we have something where the government has designed what is basically an algorithm methodology for dealing with data to try and answer a really hard statistical question: how should you provide grades for people who would normally sit an exam and who aren’t sitting the exam?”
The controversy led to something of a public spat between Ofqual and the RSS, which has been lobbying the regulator to address potential problems with its approach for months. In letters to Ofqual, since made public, the society warned that relying on schools’ historical performances and student rankings to calculate grades was bound to lead to some inaccurate grades being awarded. It was later proven right, leading to a major backlash, protests, a U-turn and the departure of two top officials.
“Had the transparency issues [of the exams algorithm] been addressed earlier, it would have helped address some of the questions before they became an urgent, problematic issue”
The RSS even put forward statisticians to help navigate what it acknowledged would be a tremendously challenging process – but was told they would need to sign a non-disclosure agreement to do so. The society objected, saying it would prevent independent experts from commenting on any discussions that Ofqual did not choose to make public itself for five years.
Ofqual has since published its methodology – which itself raises “quality questions”, Westlake says – but no minutes from the meetings of the technical panel that formulated it.
“There are questions of transparency. Had the transparency issues been addressed earlier, it would have helped address some of the questions before they became an urgent, problematic issue for ministers; and also, because it would have allowed more work to be done on the quality. Some of the concerns that have been all over the news would have been recognised earlier.”
One of the biggest lessons government must learn from the A-Level row is therefore that transparency and external scrutiny is critical, both for public trust and quality.
“It's the kind of thing if you're a politician, or an official working in the more political side of Whitehall, you may think this is a narrow technical issue for someone else to deal with. [The recent row] has really shown that data is political,” Westlake explains.
Although the RSS has been critical of the way the A-Level fiasco was handled, Westlake is keen to stress that “Britain is very lucky that we have a highly professional cadre of government statisticians, and statisticians working with, but not in, government. They are extremely dedicated. I think the spirit of professionalism in the Government Statistical Service is incredibly strong”.
He points to the Bean review, a 2016 review of UK economic statistics, and the steps that followed – the setting up of ESCoE, a consortium of research institutions to provide analysis of emerging and future issues in measuring the economy – and the Office for National Statistics Data Science Campus.
“Those are signs of a self-critical profession, which is something really worth protecting and nurturing. I think the public responds well to that.”
Politics and the public
Of course, there are myriad opportunities to dent public trust in official figures – not least, when politicians massage stats to suit their agenda. Westlake’s predecessor, Hetan Shah, has chastised ministers and departments on several occasions, including on a series of claims by education ministers about schools performance and funding.
Holding government to account is, Westlake (pictured right) says, “a really essential part” of the RSS’s role. Its call for a regulatory investigation into the Ofqual row is an example of this, he says.
Westlake has worked closely with politicians in the past, as a policy adviser to three universities ministers between 2017 and 2019 and previously in the Treasury. So. what does he think is the best way to keep them in line?
“I think ministers, like anyone, respond to incentives,” he says.
Campaigning organisations like the RSS and Full Fact that call out inaccuracies can help, as can an awareness of statistics among journalists; the RSS offers training, as well as awards, for the use of stats in journalism.
“All of those things create an environment where people understand it when ministers use data in nuanced ways – but equally, where the costs of misusing data in the public domain are higher.”
The RSS has been working recently with businesses and other research organisations and learned societies to create a framework for what it means to be a data analyst, which it hopes could be of use to government. Next, it will work on developing professional frameworks and accreditation.
Because the use of data goes much further than just Whitehall’s highly specialised analysts, Westlake notes.
“There are people from NHS trusts, across the government and in departments, who deal with data, but that don't have that structured skills development and professional support that chartered statisticians or members of the GSS benefit from. This is really a historical accident.”
Westlake is encouraged that there seems to be a “favourable wind” in government at the moment, with statistics featuring heavily in Cabinet Office minister Michael Gove’s plans for civil service reform.
What about Dominic Cummings’s fascination with big data, and his desire to bring in more misfits and weirdos – including statisticians – as part of his vision for civil service reform?
“It's always interesting to bring new skills into government,” Westlake says, adding that “continual ventilation” is important – by bringing in external experts, government can gain new perspectives and crucial skills. He points again to the ONS’s Data Science Campus.
But, he adds: “Some of those skills are already there – it depends how you define misfits and weirdos. The really important thing is to make sure that we respect, value and correctly deploy the vast wealth of skills that already exist in government.”
Leaders from two of government’s core digital and data units – the CDDO and CDEI – introduce new guidelines intended to promote transparency in the public sector’s use of algorithms
A reminder of the shocks, scandals and success stories that shaped the world of government technology in 2022
Jeff James reflects on delivering major digitisation work while working in lockdown conditions
Issue with Home Office database means tens of thousands could have erroneously received payments