| 
In 2001, NTL commissioned external consultants to provide high-level reports on the viability of using Capacity Planning to monitor and measure the level of service provided to the company’s interactive TV users. Whilst Capacity Planning tools had been employed for several years within the networks division of NTL, this was the first time that their usage had been considered beyond the confines of the IT department. Andrew Wolffsohn, Capacity Planning Manager at ntl:home recalls: “Considerable overcapacity had been built into the NTL cable network when it was first installed some five years ago. However, the move towards a more customer-focussed and cost-conscious culture within NTL meant that it was becoming increasingly important for us to have access to hard facts which would enable us to manage our future investment in the network infrastructure and to optimize service level delivery.”
The traditional Capacity Planning approach is to install a dedicated PC on the network to collect data from the designated servers. However, given the ‘sealed’ environment of NTL’s set top boxes with their integrated cable modem, it was impossible to collect data from the network using this approach. It became evident that the only common view of both ‘ends’ of the NTL network was provided by the company’s thirty-one proxy servers located at five server farm locations throughout the UK. The proxy servers are the conduit through which every set-top box must pass in requesting a page, and the route by which pages are requested from the network.
Andrew Wolffsohn: “Although there are a number of toolsets available via free download for measuring the performance of proxy servers, these only provide information on the number of hits and the countries from which the visitor has come. In essence, they are designed for an ‘Internet view’ of the world and were not suitable for our needs. For example, these free tools are unable to cope with portal services, such as our ‘walled garden’ where we have some fifty suppliers each with their own urls.”
NTL started to collect data from their proxy servers by interrogating them with a Unix-based toolset, but this approach proved to be extremely manual, time-consuming, inefficient and the results – based purely on hits – could be misleading. Andrew Wolffsohn was determined to find a better solution in his quest to proactively monitor and manage the quality of experience and service delivered to NTL’s interactive TV customers.
In November 2002, and having looked around the market for companies capable of dealing with such a novel approach to the role of Capacity Planning, Andrew Wolffsohn found only a small number of companies with what appeared to be the right type of tools and the appropriate experience.
Each company was asked to submit a proposal describing their solution for the monitoring and management of the NTL proxy servers.
Specifically they were asked to provide information on how they would handle the following metrics:
• hits – by url
• sessions – based on thirty minute time spans and designed to measure how many different sites the user visited using url’s as the method of counting
• session length – again based on thirty minute time blocks, measuring the length of time a user stayed on any one site
• stickiness – the number of times a user returned to a specific site
• quality of service – the speed at which information was displayed on the user’s screen, thus providing a measurement of the performance of the network between the proxy server and the end user
Andrew Wolffsohn: “Undoubtedly our request for this type of Capacity Planning – particularly the ‘stickiness’ and our desire to measure ‘quality of user experience’ – was not something that any of the companies had ever been asked to tackle before. All of them were willing to modify existing tools to meet our needs, but only one company – Metron – already had a database (CustomDB) which had been specifically designed to collect data from any non-standard source and bring it into a central performance database.
|
“Data collection was an essential part of our requirement. The solutions offered by the other companies would simply have provided the information on the data monitoring and collection as instant reports on screen. We needed to be able to collect the data, store it and then have the facility to ‘cut it’ in numerous different ways depending on the audience for the different reports.”
The contract to develop the system for ntl:home was awarded to Metron in May 2003 with the challenge to produce a working solution within two months such that it could be used to measure the level of service (and other metrics) during the week of the Wimbledon tennis tournament.
In addition to this requirement, NTL’s head office was also showing considerable interest in the new Capacity Planning system, believing that it could provide valuable commercial information for the company’s partners. At the same time NTL’s ETV (Extended TV) division wanted to use the Athene-based solution to provide them with a range of up to date (and previously inaccessible information) including details of how many people were using the ‘press the red button’ facility.
The Athene-based solution, designed and implemented by Metron, collects data on the metrics specified by NTL on a 24 x 7 basis. Graphical and tabular reports are produced automatically, which are published to NTL’s intranet or made available as hard copy.
The project was divided into two phases: the initial phase involved setting up the system to collect data from the thirty-one proxy servers and then import it into Athene’s performance database using Metron’s CustomDB product. By parsing the data collected from the proxy servers, Metron’s solution was able to derive the new metrics for set top box performance. The second phase of the project is concerned with automating the report production and providing access to partners and other interested parties in the resultant information.
Andrew Wolffsohn: “Metron bent over backwards to design and deliver a system in time for Wimbledon. Quite simply, they were brilliant. For the first time, we were able to have hourly reports on how many people were using the Wimbledon interactive service, compared with the previous approach which only provided daily reports some time later.”
Metron is currently working with NTL to automate the reporting aspects of the system which will allow Andrew Wolffsohn to present reports, with minimal effort, on any aspect of the ntl:home service. A ‘wild card’ facility will ensure that as new partners, urls or services are added; these are automatically added to the Athene performance database for data 24 x 7 collection.
Andrew Wolffsohn: “The reaction to the new system, even before we have all the facilities available, has been one of great excitement within NTL. We will be able to provide quick, accurate information on a wide range of metrics – including, uniquely, customer experience – for the first time at the touch of button. From a commercial and performance point of view, we believe that this capability will be very important to NTL. In addition, I know that those people within NTL currently using a data warehousing system can’t wait to replace it with the information provided by Metron’s Athene!
“The responsiveness of Metron has been a major contributor to the success of the system. I very much doubt whether other companies could have been as flexible in their approach and as fast in their development and delivery as Metron has been. I believe that once the second phase is completed in November, we will have a unique and extremely effective solution for measuring levels of customer service and experience in a live, interactive environment.”
Next Case Study |
