New section on measuring benefits on agile projects added to document that helps government create services that comply with GDS standards

The government Service Manual has added guidance on how to measure and demonstrate the benefits of agile development projects.

The manual, which was first published by the Government Digital Service two years ago, provides advice in a range of areas to help government departments develop services that comply with the GDS Digital Service Standard. The ‘Measuring Success’ section of the document has this week been furnished with a new section titled ‘Measuring the benefits of your service’.

The new guidance aims to help departments overcome the problem of it being “sometimes hard to make the case for building the right thing – especially if you’ve been funded to address a problem in a specific way”.

To do so, developers should understand the benefits that their proposed product or service delivers, and be able to demonstrate and articulate “how those benefits help address wider departmental or governmental objectives”, according to the updated manual.

This process should begin – during the discovery phase of the project in question – with the calculation of a “baseline figure” that quantifies the problem to be addressed by the proposed service. This could be related to the cost of providing the existing service, or the length of time it currently takes citizens to complete a process or transaction.

Related content

• Government’s Digital Service Standard to get major overhaul and rebrand
• GDS publishes new user research guides in Service Manual
• Final version of Digital Service Standard for councils published

Once this research has been completed, the next stage should be to work out the cost of running an alpha exploration phase. 

During this alpha phase, projects should “test different ideas and scrutinise your riskiest assumptions”. 

The potential benefits of a project can, at this stage, begin to be grouped into three areas, the first of which is cashable benefits. These are benefits that result in an organisation spending less or generating more money.

Non-cashable benefits, meanwhile, “are changes that don’t lead to an immediate benefit, but address problems you’d have had to fix eventually”. 

The final category is wider economic benefits, which cover any benefit felt by external parties – including users, and government partners in the private or charity sectors. 

During the alpha phase, project teams should also consider how fixing the problems they have set out to fix would help contribute to the wider strategic goals or broader objectives of their organisation. 

At the conclusion of the alpha phase, projects should “create an economic model”. This should include a comparison between the baseline figure calculated at the start of the process to quantify the problem to be solved ,and the estimated benefits of each of the possible solutions tested 

These comparisons should take care not to overpromise, and should take into account “what happens if things don’t go to plan once you’re in public beta”. This might include analysing the impact of service uptake being both as low and as high as could conceivably be expected. 

During the process of creating an economic model, project teams should also fully interrogate all their assumptions and predictions, the manual said.

The model should contain estimates of how much it will cost to maintain a team to run the service in beta and live mode.

“Sometimes projects are impacted by issues you couldn’t possibly have predicted,” the manual said. “So, it’s worth overestimating how much a project will cost or how long it’ll take to deliver.”

At the conclusion of this process, teams should find themselves in a position to “advocate for the right solution” – even if they have been given a brief with which they do not agree.

Going live
If and when projects progress into beta and, ultimately, live mode, continued monitoring and evaluation of benefits should take place. At the start of the beta phase, all stakeholders should “agree a set of metrics” for measuring the progress of the project, its goals, and its benefits.

It’s worth overestimating how much a project will cost or how long it’ll take to deliver

Data sources for measuring these metrics might include user research, case studies, questionnaires, and analytics data, the manual said.

During beta and live phases, teams should continue to “iterate your models”, GDS advises.

The manual said: “Working in an agile way means you can test things and get rich data fairly quickly – especially in beta and live, when your service is being used by a high number of users.”

If teams follow the new guidance, upon reaching the conclusion of the beta phase they should be able to say with certainty whether to progress the project into live mode.

“At the end of beta, you should have an iterated economic model and benefits map that demonstrate the value for money your service will deliver if progressed into live,” the manual said.