Children taking part in a project that works with schools to gather pupils’ feedback on government policy have resoundingly concluded that the risks of public sector AI outweigh the benefits

An initiative designed to gather policy feedback from school pupils has found that a large majority of children aged believe that the benefits of government using artificial intelligence are outweighed by the risks – including the danger that the technology will “take over the world”.

The Pupils 2 Parliament project was created in 2015 by Roger Morgan – who previously spent 13 years in the statutory public office of national children’s rights director. The initiative works with a range of schools in England and Wales, from which it collects and collates the views of pupils, before submitting them to select committee inquiries or government consultations.

The project’s most recent submission features feedback on the use of AI in the public sector – a topic that, prior to the dissolution of parliament this week, was the focus of an inquiry by the Public Accounts Committee.

Pupils 2 Parliament worked with the Knighton Church in Wales Primary School in the Welsh county of Powys to gather input from 26 children aged from nine to eleven.

The pupils were asked to assess the comparative risks and benefits of several example use cases of AI. The evidence submission reveals that “in all three examples, the children assessed the stated risk as outweighing the potential benefits of the AI application”.

Related content

• AI firms must address hallucinations before GOV.UK chatbot can roll out, digital chief claims
• AI Safety Summit: Major nations sign cooperation agreement recognising technology’s ‘potential for catastrophic harm’
• Experts warn UK could still be ‘powerless’ to tackle AI risks despite government’s £100m plan

In two proposed uses of AI – namely answer citizens online questions for government, and diagnosing health conditions and recommending appropriate NHS treatments – the young respondents voted “by a majority of 13” that the risks are greater than the prospective benefits. This equates to a margin of 19 votes to 6, with one abstention or otherwise neutral answer.

In the case of using AI to write public-information documents, the children voted 22-4 that the dangers outweigh the possible advantages.

After issuing their verdicts on these specific potential deployments, children were given an open invitation to provide further feedback on the possible use of AI by government. This

“The main view expressed was the concern that AI systems might ‘go wrong’ or gather errors in their learning and so make errors in their tasks, resulting not only in the consequences of those events themselves but also in serious costs to government,” the Pupils 2 Parliament submission said. “[One child said:] ‘The AI might stop working and the government would have to reboot it and it would cost a lot of money’. Because of errors, introduction of AI ‘could cause more damage’ [according to another]. Children also spontaneously raised the worry that AI might one day ‘take over the world’. However, they saw this as most likely to be the result of error or malfunction – ‘more likely malfunction than on purpose, take over the world and cause lots of problems’.”