The telecoms giant has been issued with a hefty penalty after a regulatory investigation found that the firm did not have adequate contingency procedures, which was compounded by human error
BT has been hit with a multimillion-pound fine over failures that contributed to more than 10 hours of disruption to calls made to the UK’s emergency services 999 phone line.
Announcing a financial penalty of £17.5m, regulator Ofcom said that its investigation into the outage – which took place on 25 June last year – had identified three “key stages” during the incident.
Problems began at 6.24am and, until 7.33am, “BT’s emergency call handling system was disrupted by what was later found to be a configuration error in a file on its server, [which] resulted: in call handling agents’ systems restarting as soon as a call was received; agents being logged out of the system; calls being disconnected or dropped upon transfer to the emergency authorities; and calls being put back in the queue,” according to Ofcom.
The regulator added that the telecoms firm was “initially unable to determine the cause of the issue and attempted to switch to its disaster recovery platform”.
However, from 7.33am until 8.50am these attempts to move to the backup system were “unsuccessful due to human error… as a result of instructions being poorly documented, and the team being unfamiliar with the process”.
It was during this timespan that “the incident grew from affecting some calls to a total outage of the system”, Ofcom found.
Although the 999 call handline platform was successfully switched to disaster recovery provisions shortly before 9am, disruption to the emergency services phoneline persisted until a few minutes before 5pm, as the backup “platform struggled with demand” over the course of the day.
Related content
- DHSC spends £40m on ambulance handsets to work across Airwave and ESN
- HMRC extends Airwave device deal for criminal investigation units until 2027 to cover ESN delays
- Emergency Services Network: Home Office claims majority of technical groundwork ‘to start mass transition’ has been done
A total of 14,000 calls were impacted during a period of 10.5 hours. There was particular disruption to text relay call services, meaning that “people with hearing and speech difficulties were unable to make any calls, including to friends, family, businesses and services, [which] left deaf and speech-impaired users at increased risk of harm”.
The regulator said that the problems demonstrated that BT lacked “sufficient warning systems… for when this kind of incident occurs, nor did it have adequate procedures for promptly assessing the severity, impact and likely cause of any such incident, or for identifying mitigating actions”. Ofcom added that its investigators “found that BT’s disaster recovery platform had insufficient capacity and functionality to deal with a level of demand that might reasonably be expected”.
These failures, coupled with a “potential degree of harm [that] was extremely significant extremely significant”, informed the watchdog’s decision to impose such a hefty fine.
“Being able to contact the emergency services can mean the difference between life and death, so in the event of any disruption to their networks, providers must be ready to respond quickly and effectively,” said Suzanne Cater, Ofcom’s director of enforcement. “In this case, BT fell woefully short of its responsibilities and was ill-prepared to deal with such a large-scale outage, putting its customers at unacceptable risk. Today’s fine sends a broader warning to all firms -– if you’re not properly prepared to deal with disruption to your networks, we’ll hold you to strict account on behalf of consumers.”
In response to the penalty, a BT Group spokesperson: “We take great pride in underpinning the national 999 service and recognise the critical importance our infrastructure plays. The level of disruption to the service on Sunday 25 June last year has never been seen before and we are sincerely sorry for the distress caused. We accept the specific points raised in Ofcom’s findings, and have put in place comprehensive measures to prevent this series of events reoccurring and improve end-to-end resilience of the system as a whole. While no technology is 100% resilient, we have built a highly robust network with multiple layers of protection to connect the public to blue light services in their time of need. We take our responsibility to the emergency services and the public seriously, and on this occasion we fell short of our own high standards for the 999 service.”
Please tell me more about your excellent articles
My programmer is trying to convince me to move to .net from PHP. I have always disliked the idea because of the costs. But he’s tryiong none the less. I’ve been using WordPress on a number of websites for about a year and am worried about switching to another platform. I have heard good things about blogengine.net. Is there a way I can transfer all my wordpress content into it? Any kind of help would be really appreciated!
It抯 really a great and useful piece of info. I抦 glad that you just shared this helpful info with us. Please stay us up to date like this. Thank you for sharing.