Operational Intelligence (OI) is a process of creating relevant actionable business insights from operational data. Behind your IT infrastructure systems are massive streams of machine-generated data. This data is incredibly valuable to the overall efficiency of your business. OI is designed to enable organizations to:
Gain deeper understanding using all relevant information, especially machine data.
Reveal important patterns and analytics by correlating events from multiple sources.
Dramatically reduce the time to detect important events.
Leverage live feeds and historical data in order to understand what is happening now.
There exist huge amounts of valuable machine-generated data. Here are a few examples:
Online businesses providing transaction monitoring 24/7/365.
Web activity data to expand knowledge of customers, capacity, and digital asset usage.
Service level monitoring information from Managed Service Providers.
Call and event records to uncover more profitable services for Communication Service Providers.
GPS and other data that enriches customer behavior information with location data.
Most organizations use a complex mix of business applications / reporting / analysis tools. However, many challenges and questions remain, such as:
Why does it take so long to get answers re: key business metrics?
Why can’t I see what is actually happening right now?
Why is it so hard to handle exceptions when things go wrong?
Why can’t we capture and preserve knowledge so we can be more effective in the future?
Simply put, the speed of business has increased beyond the capacity of the previous generation of IT, which focused on the tracking and automation of transactional activity. The new generation of IT must not only capture what has happened – it must tell us what is happening now and facilitate timely action.
Business Intelligence (BI) draws on data sources that are historical, batch-loaded and structured. OI is typically used with time-series, unstructured or semi-structured data, e.g., specific machine events or transactions that have a timestamp association. The data in OI systems enables you to see what is happening now and compare it to what happened in the past. OI answers questions that traditional BI systems are not designed to answer.
Rather than thinking of OI as an alternative it is helpful to view BI and OI as complementary. Usually, a successful BI implementation provides one with long term direction, while OI implementation provides one with short term direction. With the right tools you can exploit the explosion of data and gain new insights for running your business while supplementing it with the best available customer analytics along with the knowledge of past successes.
Data doesn’t have to be real-time to offer valuable business insights. Years of historical logs can be mined quickly that can reveal trends. There are countless ways OI can make an organization more effective, productive and secure. Fundamentally, OI helps you take advantage of new categories of rich real-time data whose business value you probably have not begun to exploit.
Machine data is a class of data being generated by web servers, apps, machines, SaaS systems, etc. This data includes GPS readings, call logs, fleet locators, etc., and is one of the fastest growing categories of ‘big data’.
Until now, analysis of this data focused on machines and their operations, and not upon what it could tell us about our businesses. Today, massive amounts of information can be used for finding and fixing problems, or leveraged for strategic business advantage. This machine data explosion requires a new way of analysis that sits alongside established practices.
How can you observe machine data? The semantics of machine data are complex and CIOs must make clear its value to the organization. Little good comes from handing off raw data to your business staff. IT must make the data usable so the business staff can analyze operational data in order to gain valuable insights.
The fastest road to OI comes through creating business value from the explosion of machine data.
CEOs, CFOs and senior managers are accustomed to seeing historical data. They analyze past performance and predict future results based upon data points such as sales figures, buying trends, raw material fluctuations, etc. It requires experience and intuition to drive business this way, but it is a bit like celestial navigation, which tells you where you were and directs you toward where you would like to go, but it doesn’t reliably tell you where you are. Operational Intelligence is more like GPS. Managers who use data in motion can make real-time course corrections or quickly chart new directions.
Bringing Operational Intelligence practices to an organization is a gradual process.
Figure 1. The Roadmap to Operational Intelligence
Let’s look at each step along this roadmap in greater detail.
Search & Investigation. The journey begins as IT uses machine data as a means to determine what is happening during an incident in a data center. To find the root cause IT examines not only each data set of the system that produced it, but also for information it offers about customers, key events, or performance of business processes.
Proactive Monitoring. IT proactively monitors data to avoid previously identified risks. Simplified forms of predictive models can be created at this stage. Events and trends that may lead to trouble are identified so failures can be avoided. At this point, IT usually understands machine data well enough to start proposing business improvements.
Operational Visibility. IT starts measuring SLAs and KPIs across the organization to engage the business. As business interest grows, users are able to answer questions and track consumer behavior in ways previously not possible without machine data. Now the conversation begins in earnest. IT begins to understand the real business needs and the Business staff begins to understand the real value of machine data. More sophisticated customer behavior models and business processes begin to emerge. At this point, Business staff presents IT with additional questions and IT responds with a quick custom dashboard (instead of a pointer to raw, unintelligible machine data or a three-month wait for a new report).
Real-Time Business Insights. The pinnacle of Operational Intelligence comes when machine data is used to track and correlate activity in real-time and to predict behavior. Dashboards are put in place, events are recognized that spur activity and predictive models can forestall problems and/or identify opportunities. At this stage, Operational Intelligence can be used broadly across the organization – often with more Business users than IT users.
Organizations often rely upon today’s leading analytic applications to answer questions using static, historical data. In the past, when new questions arose, new applications had to be designed.
Organizations now have the tools to examine this new class of data in order to understand its raw form. Operational Intelligence presents the opportunity to gain new, exciting insights from the massive volumes of data machines are creating.