AIOps lessons learned: Be careful when selecting a vendor

The concept of AIops is simple: Infuse artificial intelligence(AI)  into IT to make operations speedier and more efficient. In theory, AIops at its best should lead to an autonomous IT environment in which functions can run themselves with little or no human intervention. In practicality, the path to this nirvana state is anything but straightforward and raises several questions. Where should you start? How do you measure the value? Is AI ready to scale across production environments? Do I need new tools? 

Although there is no easy button for AIops, it’s certainly worth exploring because IT operations have grown significantly more complex, particularly over the past couple of years as IT staff have been sent home to do their jobs. Businesses are becoming increasingly dynamic, mobile, cloud-centric and distributed, which is creating a massive gap between budgets and the role of IT. AIops can help close that gap. 

Research firm ZK Research, in partnership with Masergy, conducted an AIops study to shed light on where the industry is and to provide some insights as to how to answer some of these questions, enabling businesses to move forward with AIops. The study was a total sample size of 510 enterprise-class companies across a wide range of industries. 

The study focused primarily on the intersection of AIops with network operations, given the number of significant market transitions happening in the network and the fact that networks touch all parts of an IT system. Software-defined WANs (SD-WAN), secure access service edge (SASE), remote and hybrid work, 5G and other trends make managing the network extremely challenging using legacy operations. The goal of the survey was to  find out whether networking was ripe for AIops.

Here are some of the key findings of the survey.

Adoption rates offer warning to IT pros: AI analytics don’t equal automation

Don’t be fooled by AI-based analytics. Many IT leaders think that because they’ve implemented AIops, they are on the path to full automation. However, that isn’t always true. The study findings indicate that it’s highly possible buyers are getting caught in this misconception.

A whopping 64% of IT leaders said they are currently using AIops, compared to 37% who are simply evaluating or exploring the technology. Anecdotal research indicates the 64% number is too high, because  ZK Research was expecting about 50%. While it may seem that this is contradicting the  survey data, it underscores that there is a misunderstanding of what AIops actually is.  ZK Research said that over-reporting adoption is often the case with emerging tech. 

In the early days of the cloud, businesses using a basic hosted service were claiming to use the cloud. This is a step on the way to the cloud, but this is not a cloud implementation. A more recent example is SD-WAN. Early in the buying cycle, businesses that had adopted broadband for corporate connectivity considered this to be SD-WAN, but it isn’t.   

The data presents a good news/bad news scenario for the industry. It’s certainly a positive that such a high percentage of companies are interested in AIops, but it’s a negative that many will go through some growing pains as IT pros better understand which vendors are using AI and which others are using the term for marketing. 

It’s critical that evaluators of AIops systems understand what they are using and if the AIops toolset they’re evaluating can improve a user’s autonomy. AIops is not a network management tool with a few recommendations, nor is it a security dashboard with a clever color scheme or a basic rules-based engine that requires constant updating. 

One way to tell is to look at what is powering the engine. AIops should be built on machine learning, behavioral and predictive analytics that do more than point to problems, generating tickets for IT staff. AIops should offer verified solutions and help IT teams carry out necessary modifications. AIops engines should come front-loaded with network management and security intelligence, but more importantly, they should be observing real-time data 24/7 to learn from existing environments. 

Companies and threat landscapes are always evolving, so AIops should move with them, getting smarter over time. Ask the vendor about the product’s problem-solving accuracy rate today compared to a year ago. Other good questions to ask are: How long it will take to learn your environment, and how often do the models need to be retrained? Buyers need to be diligent when selecting an AIops tool.

IT operational efficiency is a good starting point  

One of the first questions asked in the survey was: “What are the primary reasons for your interest in AIops tools?” because this would provide some insight into common use cases. The top responses were stack-ranked in this order:

1. Gain IT operational efficiency/productivity
2. Faster security threat response
3. Faster security threat identification 
4. Improve network reliability
5. Improve network/application performance 

This data shows that companies are using AIops platforms for basic blocking and tackling first before using them for more advanced tasks. This is the right approach because the network needs to be secure and optimized before companies can look to advanced capabilities, such as intent-based networking.

AIops is for network and security 

One of the pleasant surprises from the study was the coming together of network and security. Typically, large enterprises keep a walled garden between the two teams. After years of talking about bringing these teams together, it’s good that businesses are finally looking for a single tool that can be used by both groups. 

This trend was highlighted by the question, “What are the top criteria by which you will select an AIops provider?” The top response was “AIops features,” which include analytics, predictions, recommendations, and integration. This supports the above data points that the technology is used to improve IT operations. The second-highest response was”Ability to address network and security.” In fact, the study found 55% of those already using AIops leverage it across both IT domains. 

Think twice before taking a DIY approach 

The survey asked: “What deployment model will you choose for AIops?” Topping the list was in-house management at 47%, followed by comanagement at 32%, and fully managed services at 21%. Given it’s so early in the AIops cycle, ZK Research expected  a do-it-yourself preference, which is common with new technology. A word of caution: Getting AI models trained and retrained to work in a production environment is not trivial. 

In this case,  ZK Research recommends seeking a managed service provider to at least assist in ensuring that the data sets are good. In data sciences, there’s an axiom that states, “Good data leads to good insights,” but bad data can lead to bad insights. Managed services can help ensure the best results quickly. 

AIops is no longer the stuff of science fiction. The technology is real and works today. Businesses that are considering AIops should learn from the study and use it to solve the basics first before endeavoring to shift to more advanced capabilities. Furthermore, choose your vendor carefully and challenge the company to provide metrics and data on the performance of the product. Finally, if your company isn’t filled with data scientists – and most are not – then seek a managed services partner that can help. 

Chris Preimesberger is a former editor of eWEEK and a regular VentureBeat contributor who has been reporting on and analyzing IT trends and products for more than two decades.

Zeus Kerravala is the founder and principal analyst with ZK Research. He spent 10 years at Yankee Group and prior to that held a number of corporate IT positions. Kerravala is considered one of the top 10 IT analysts in the world by Apollo Research, which evaluated 3,960 technology analysts and their individual press coverage metrics.