Aberdeen City Council solves four-month IT problem in two days

When internal due diligence procedures were unable to identify the exact problem, Aberdeen engaged Advance7's REACT problem investigation and diagnosis service. Working with Aberdeen's IT department to identify and resolve the problem, Advance7's unique Rapid Problem Resolution (RPR) methodology determined the root cause of the recurring problem in just two days.

The problem was affecting staff that travelled off-site during the day and then logged on to the Council's web access email system to catch up on emails. Staff experienced inconsistent or very slow access to the email system, severely impacting their ability to work from home.

Michael Campbell, Technology Analyst at Aberdeen City Council, commented: 'This issue was particularly frustrating because it was an intermittent problem which meant we could not predict when it would occur, making it very difficult to log issues each time. Even when using the standard logs provided for the application we were unable to identify where the problem lay. Having seen the Advance7 team in action previously, we were confident that their RPR approach would help us resolve the issue quickly. They did not disappoint – identifying the root cause in just two days.'

Endemic amongst many companies both in the private and public sector is a tendency to allow IT problems to run for far longer than they need. There could be many reasons for these problems, but essentially they fall into two categories: major incidents that may have serious immediate impact; and ‘grey' problems, which are intermittent or transient. It is the latter that often to tend to run without being resolved, in many cases because the root cause is difficult to pinpoint. However, the longer they are left to run, the more they will impact on business productivity. IT departments need to be more forward thinking and embrace problem management. If they can't resolve problems themselves, they need call in external experts.

Advance7's first step was to get a detailed debrief from Michael's team and identify the key symptoms. Next, network analysers were installed, hardware that captures data at a ‘low level', and in one day Advance7 was able to collect more detailed data than the standard logs had allowed Aberdeen to do. On day two, analysis of this ‘low level' data gave an initial indication of what the root cause could be.

In addition, all remote users were given a ‘tick sheet' that they had to complete each time they had an issue. The tick sheet gathers scene-setting information such as, time, location, internet connection, user name, etc, as well as information on the action being attempted, such as, logging on, downloading email, sending attachments, etc. Overnight, two users had reported a problem and recorded the incident on their tick sheets. By stripping out all of their data and looking closely at what these users where trying to do, Advance7's initial suspicions were confirmed.

Mark Bairstow, Service Delivery Manager at Advance7 said: 'A major part of the RPR Methodology is to follow the correct process to gather a 360° picture of the problem and undertake a thorough post-capture analysis of the data. Only by analysing and processing the low level data and correlating with the tick sheet data were we able to identify the root cause.'

What Advance7 discovered was that remote users were using HTTP 1.1 enabled web browsers, the newest version of web browser, to access the Application. Unfortunately, the WebAccess server was not configured to support persistent sessions of HTTP 1.1. When the remote user's initial attempts to connect to the application timed out, they would try to re-connect, leading to an overload of the firewall, and a failure of the application to respond.

'Advance7 brought their trouble-shooting expertise to the table – knowing where to look and how to look at it,' said Michael Campbell. 'Once the REACT team had identified the root cause my team was able to apply a one line fix in the command – and our remote users have been logging on problem-free ever since. We have been so impressed by Advance7's approach that we have booked a course of RPR training.'

Many companies resign themselves to an unnecessary expensive system improvement programme (SIP) without fully determining what the root cause is. Advance7's REACT service operates a no fix no fee policy, meaning that these companies have nothing to lose in engaging specialist experts before embarking on a costly SIP.

Advance7 has very recently completed training of Aberdeen City Council's IT department in its unique RPR methodology. In addition, Aberdeen City Council engaged Advance7's IMPACT Data Centre relocation service.

Related links to this article:
Aberdeen City Council
Advance7