Case Study How INOC Worked with Aqua Comms to Achieve Detect-to-Ticket Times of 5 Minutes, Reduce Ticket Volume by 20%, and Manage a Growing Professional Services Pipeline

The Challenge

To support its growth and reputation as a carrier of high-bandwidth transport-layer services across the Atlantic, Aqua Comms needed to establish and maintain guaranteed service level agreements (SLAs) across all of its network elements.

This required a highly capable 24x7 network operations center (NOC) to monitor this infrastructure around the clock, and catch and dispatch performance and availability issues to the appropriate vendors and customers. Aqua Comms determined that outsourcing its NOC to INOC enabled it to access the requisite support capabilities and resources to support its major carrier customers while avoiding the costs, operational complexities, and business distractions of building and maintaining such a NOC in-house.

Aqua Comms initially engaged INOC to augment its limited support function and become its primary NOC, monitoring and managing basic notification and escalation activities across its subsea and terrestrial cable networks.

As time went on, however, it became apparent that there were some significant underlying issues undermining INOC’s ability to provide efficient support. It was also apparent that much more would soon be asked of the NOC as Aqua Comms continued to build out its critical infrastructure services and support investors’ networks to exceedingly demanding SLAs.

It was during this period—about two years into the engagement—that Charles Cumming, Senior Global VP of Operations, and Andy Hudson, Chief Network Officer, joined Aqua Comms. Both had designed, built, installed, and owned network operations centers throughout their careers, and drew on this experience to lead an effort to investigate and solve these fundamental support challenges before adding more capabilities. We summarized these specific challenges below.

• Inflexible runbooks led to confusing external customer notifications: The runbooks INOC had used until this time predated its engagement. Upon this in-depth analysis, however, these runbooks were found to be limiting in their flexibility and insufficient to the demands of the day. Their continued use resulted in messy, confusing, and ultimately un-actionable information being passed onto customers via tickets. Accurate customer communications is a significant challenge when equipment and infrastructure capacity are shared across hundreds of customers. Mapping devices, modules, ports, and services to each customer needs to be done accurately and kept up-to-date on an ongoing basis.
• A lack of ITSM onboarding and pricing support for a pipeline of new professional services: More recently, Aqua Comms rolled out a set of professional services to support other investors’ networks running on its cable systems. When these services were first being designed and planned for the market, pricing the service menu appropriately was a struggle. While Aqua Comms had a clear vision for what their professional services catalog should contain, it didn’t have the unique ITSM experience to present and price those services accurately.

Aqua Comms needed a highly capable and operationally mature NOC through which to onboard, monitor, and support these customers—a challenge that INOC was well-suited to overcome as a larger vendor.

The Solution

Aqua Comms and INOC first worked to address systemic problems that were limiting the effectiveness, efficiency, and accuracy of the support coming from the NOC before attending to the need for a better-defined and supported professional service catalog.

The two companies have worked together to update their runbooks and have continuously refined their internal messaging to ensure that alerts sent to customers and vendors are clear, complete, and accurate while internal teams have the information they need to perform an investigation. These solutions arose through a few specific projects, which we’ve summarized below.

• Updating runbooks to better analyze alarm data, establish internal messaging, and conduct investigations prior to external messaging: To address the communications issues caused by the underpowered runbooks, Aqua Comms and INOC worked closely to refine the procedures. The CMDB was rebuilt with accurate relationship information between devices, modules, ports, services, and Aqua Comms’ customers. This allowed for multiple automated workflows to be built to improve the accuracy and timeliness of customer notifications. Additionally, the NOC team was trained to better segregate, analyze, and triage incoming alarm data. As part of this refinement, the two companies developed inward messaging to help Aqua Comms and network vendor Ciena perform internal investigations and prevent a significant number of unnecessary external notifications.
• Collaborating on pricing and support onboarding for professional service customers: INOC has also worked closely with Aqua Comms to develop accurate pricing and technical support for parts of its professional services catalog, with pricing being a critical consideration for submarine systems.

The Results

After implementing these changes and new service development, Aqua Comms has been able to:

• Reduced ticket volume by 20%
• Provided improved external customer-specific notifications in the form of executive summaries
• Achieved a comfortable degree of Ciena interoperability
• Created a well-defined submarine and terrestrial cable professional services catalog
• Achieved and maintained a tight SLA of five minutes from alarm detection to ticket creation
• Maintained solid second-line engagement

Despite the successes, Aqua Comms stresses the importance of continual service improvement, hoping to better refine their processes as they expand their global services.