ASPAdvisor hed: Outsourcing accountability dek: how do you know that your xSPs measure up? dek: to get the most out of SLAs, spend up to 8 percent of your outsourcing cost on overseeing your contract. by Don Fitzwater

As more businesses decide to outsource some or all of their IT infrastructure and services to xSPs, companies are requiring accountability from their service providers and guarantees about whether they are receiving the performance promised by their service-level agreements (SLAs). This becomes increasingly complicated as businesses use multiple outsourced services, with each side pinning any performance problems on the other. SLAs alone don't necessarily mean your business operations are properly protected or provisioned. Unless firms take care not only in negotiating but also in monitoring their SLAs with outsourcers, there is a very real danger that the outsourced services will be less than satisfactory. Indeed, outsourced services could prove less reliable and more expensive than in-house systems. Because Web site or network downtime can cost a company lost revenue and productivity, finding the source of the problems and getting them solved quickly is absolutely essential. Given this situation, analysts say that time and money spent on managing SLAs can reap large rewards. To do so, the IT consultancy Meta Group suggests that firms should spend 7 to 8 percent of the cost of any outsourcing deal on overseeing the contract.

Who watches the watchers?

The sad state of affairs today is that most SLAs are negotiated and then forgotten because, to date, there hasn't been a reasonably practical way to identify and deal with outsourcing violations from the customer's viewpoint. As a result, SLAs are often only based on infrastructure availability metrics, resulting in imperfect end-user performance guarantees, poor customer satisfaction, and a whole lot of finger-pointing.

Monitoring these flawed SLAs involves an assortment of approaches, each of which has limitations. Automatic monitoring tools are one option for companies wanting to enforce their SLAs. Another is to use undercover staff to test the service at the end-user level. Some companies farm out this type of monitoring activity to a third party that performs spot checks on various parts of the company's systems. These checks can involve submitting dummy orders for goods on e-commerce systems or just testing the quality of service overall. For example, there are companies that do Web site performance monitoring by pretending to be users logging on to buy items on the company's system, then reporting on how well the system did (or didn't) perform for them.

Automated tools and site checkers can be useful in spotting at least the most obvious of SLA violations, but these tools often don't go far enough. The traditional metrics for network-based services--availability, latency, and throughput-are often accompanied by additional measurements such as Mean Time to Repair (MTTR) or Mean Time to Problem Resolution. But the industry is evolving. Metrics such as application availability and response time are gaining importance, particularly among xSPs. Unfortunately the availability has largely received more attention than response time. This is a shame because response time is a crucial factor in the end-user experience. An application's response time is directly proportional to user productivity.

Furthermore, the industry is moving away from using those traditional IT metrics to enforce SLAs, as discussed above. IDC's research indicates that senior managers outside the IT department prefer SLAs to be measured in a way that shows the impact on the overall business process rather than metrics such as response time and uptime. In the IDC study, 82 percent of organizations surveyed said they would prefer to have SLAs specified in terms of end results for the business process.

So how does one go about monitoring your SLA from that sort of perspective? Well, you can use some or all the aforementioned approaches alone or in combination with each other, depending on your needs. Or you can do it by hiring another kind of xSP: There is a whole new emerging class of xSPs that specializes in monitoring SLAs.

Taking your xSPs' pulse

This new class of xSP provides SLA performance-monitoring tools and infrastructure to other xSPs, the xSPs' customers, or both. Performance-monitoring xSPs use a combination of traditional automated network-monitoring technologies and "intelligent agents." Together, these tools allow the xSPs to simulate a company's users by doing the same business-critical activities. These xSPs accurately and systematically measure true end-to-end performance every step of the way-from client through the network and up into servers, Web pages, applications, databases, and back-end transactions-even across the Internet or an outsourced service-provider network.

One such provider is ResponseNetworks ResponseNetworks, which offers its Pulsar xSP, and SLA performance monitoring service for xSPs. Through this service, ResponseNetworks says that xSPs can develop meaningful performance SLAs driven by customer-business transactions across the entire supply chain.

Pulsar xSP is designed for just about any provider-class organization (AIPs, ASPs, MSPs, WIPs, etc.) that need to manage SLAs across any part of an application-hosting supply chain. The product is a Web-based application with functionality that includes a policy engine for creating granular, business-driven SLA compliance rules. Additionally, the application can perform baseline and trend analysis, real-time monitoring and notification, diagnostic triage, report creation, and publishing. It includes an integrated, secure portal allowing different customer, provider, and partner user groups to sign in and view customized information about service levels.

The product automatically generates application, database, server, or network transactions from distributed agents that act as simulated end-users. It then measures transaction performance against defined service-level goals and policies to provide a true end-user perspective.

By measuring the total end-to-end response time of application transactions, the product allows the operator to monitor against service levels in real-time. By breaking down end-to-end business transactions into their component parts, it can help diagnose faults by location and type, as well as assign accountability and proof of performance for managing service-level agreements.

Another similar provider, AperServ, helps companies by automatically managing and measuring the network, system, and application performance of their IT outsourcers and related SLAs. While ResponseNetworks' product is focused more on providers, providers' clients are intended to use AperServ.

Much like the Pulsar xSP service, AperServ's Technology Outsourcer Management (TOM) system monitors the performance of the outsourced enterprise systems, networks, and applications. It uses automated alerts and an integrated problem resolution system, and notifies the appropriate staff (both in-house and at your outsourcer's operation) when performance issues arise. It also provides the detailed metrics needed to solve the problem quickly.

TOM makes use of multiple, geographically dispersed point-of-presence intelligent agent servers. The system's real-time performance reports show xSP clients historical trends to track recurring problems over time. Technical reports detail more than just system up and down time; they also provide information on speed of data transfers (latency), accuracy and relevancy of data, packet loss, and so on. In addition, TOM generates business reports that present the bottom-line financial impact of xSP problems.

To help motivate your company's service providers, the product's SLA Credit Request System lets your providers know when a credit-worthy problem has occurred. An SLA Credit Assessment Report automatically compares your monthly system performance with the terms of your IT outsourcers' Service Level Agreements. In other words, it systematically lets xSPs know they are being actively monitored and will be held accountable if they come up short.

SLAs are becoming increasingly common in today's Internet economy. For an SLA to be truly useful, it should include precise details of the service being given and the penalties to be imposed for failures. But perhaps even more important, SLAs need to be monitored diligently. Unless SLAs are monitored, they may result in outsourced services being more expensive and less reliable than in-house systems.

And the management of SLAs should involve more than just traditional IT-related technical measures. The SLA management process should also include procedures to demonstrate whether outsourced services improve or hurt overall business performance at the bottom line--where, ultimately, it matters most.