In its annual report, department makes good on a pledge to develop, undertake and reveal the outcome of a fairness analysis, but declines to provide detailed findings of the process
The Department for Work and Pensions is growing its trials of artificial intelligence tools while revealing that an internal “fairness impact assessment” of algorithms used to help detect benefit fraud found no “immediate concerns of discrimination, unfair treatment or detrimental impact”.
Since it first began using an anti-fraud algorithm three years ago to help assess claims for Universal Credit Advances, the DWP has faced criticism and concern over what onlookers have claimed is a potential for bias and a lack of transparency about the technology’s operation.
The National Audit Office report included in the department’s annual financial statements for the 2022/23 year revealed that the DWP’s preliminary testing of the machine learning system “found some evidence of bias toward older claimants in some of the models”. But at the time, the DWP broadly believed that “its ability to test for unfair impacts across protected characteristics is currently limited”.
At the request of the NAO and parliament’s Public Accounts Committee, the department committed to develop and then undertake a full fairness impact assessment and include the results in its report the 2023/24 year – a pledge which it has now made good upon.
The DWP’s FY24 report reveals that the department conducted this investigation – which was based on “best-practice analytical methodologies” – in early 2024.
The report adds: “DWP’s assessment of the results of the fairness analysis do not present any immediate concerns of discrimination, unfair treatment or detrimental impact on customers. Analysis confirms payment timeliness of legitimate Advances requests is not disproportionately affected by the model. The model does not impact payment of the associated UC claim.”
While this outcome is now public, the department said that it “has considered the benefits and risks of publishing the [full] results of fairness analyses [and] concluded it is not in the public interest to do so, because it will undermine the effectiveness of the model as a fraud prevention control and therefore erode the ability to protect the public purse”.
The DWP did, however, add that it “is committed to continue iterating the fairness analysis method” and that further assessments “will be completed at regular intervals”.
The department has previously faced calls to use government’s Algorithmic Transparency Recording Standard to publish details of how the fraud-detection system works – but has declined to do so. This seems set to remain the case, but the annual report says that the DWP has created an internal “AI inventory” to maintain records of how its algorithms work – and that this “has been aligned” to the framework set out by the recording standard.
The document does reveal that the anti-fraud technology “is designed to risk assess Advances requests and refer those assessed as high-risk to a colleague for a fraud-prevention intervention”. It added that it represents “the first machine learning model deployed into live service” by the DWP.
New generation
But many more AI and automation tools have seemingly been deployed since, the annual assessment reveals. During the 2023/24 year the department “rapidly and successfully tested multiple generative AI proofs of concept”, according to the report.
The yearly review picks out three particularly noteworthy examples, beginning with ‘Whitemail’, a technology that “scans documents and quickly identifies vulnerable customers, allowing DWP to fast track and intervene”, the report says.
Another generative tool tested last year was ‘AIgent’, which supports agents administering Personal Independence Payments “by summarising evidence for inclusion in decision letters”.
The ‘A-cubed’ system, meanwhile, “provides work coaches with quick access to advice to help to support customers move closer to the labour market”.
“The DWP has considered the benefits and risks of publishing the results of fairness analyses, and concluded it is not in the public interest”
Between them, these three technologies have allowed the department to sort though millions of documents, as well as helping those most in need obtain support quickly, the report claims.
“DWP has introduced technology that quickly triages the high volumes of correspondence it receives,” it says. “Every day, 22,000 documents are processed in real time in what previously used to take weeks. To date, DWP has processed 2.2 million documents using generative AI to read, understand and summarise correspondence. The full information is then shared with colleagues for decision making. It also uses capability to flag potentially vulnerable citizens and expedite their correspondence to the relevant colleague who can help them.”
Board games
PublicTechnology had contacted DWP requesting comment on these use cases and was awaiting comment at time of going to press.
During 2023/24 the department launched its Lighthouse programme to help support and guide its explorations of generative AI.
Along with the programme, the DWP “has developed a new cross-departmental assurance framework for assurance and governance of AI” and has also “extended its existing digital governance to include an AI Steering Board, an AI assurance and Advisory Group, and an AI Delivery Board”.
The steering and delivery boards are chaired by DWP chief digital and information officer Richard Corbridge, while the assurance and advisory collective is led by the department’s deputy chief data officer.
Across all its deployments of AI, the department strives to ensure that its work “adheres to six principles”, which dictate that all uses should be: explainable; mitigated; controlled; understood; value-led; and governed.
