Glitches, Blips, and Outliers. Real-Time Anomaly Detection Using Machine Learning

12th June 2017

Written by Rebecca Herson

Part 1

Companies are simply bursting with data. And as the volume of global data grows into the “trillions-of-gigabytes” level, organizations have the ability to leverage information in ways never before seen. Companies can capture and analyze vast quantities of metrics, such as key performance indicators (KPIs), mobile data, weather, infrastructure operational performance, and product sales patterns, all in an effort to gauge the health and success of their organization.

When analyzed rapidly and correctly, this information has the ability to provide unique and powerful insights that can aid organizations in developing useful business strategies. At the same time, with so much data, the smallest glitch has tremendous power to do significant financial or reputation damage.

However, as data gets bigger and more complex, we are finding that organizations struggle to analyze information quickly and effectively using only existing staff and conventional methods. And, if an unexpected or non-conforming change occurs within these data patterns, many companies find themselves simply without the means to determine the source of the problem.

Why Worry About Anomalies?

Time series data, KPIs, and variables are numerous, complex, and sometimes interconnected. And these data offer companies a real-world story about their services, products, and solutions: A celebrity endorsement has bumped sales in Asia; a flaw in a single line of code is causing failures in an online check-out system; or manufacturing production equipment is running hot, causing a slow-down in processing. A single medium-sized business might have thousands of different variables and sub-variables that influence operations, such as device type, customer satisfaction, location, shipping, internet usage, conversion, etc. A single incident could be the result of one blip in one variable, or blips in hundreds of different, but interconnected variables. And, when tens of thousands or millions of dollars are riding on the success of a product or service, companies desperately need accurate information, so money and manpower can be utilized more effectively and efficiently.

It isn’t possible to correctly attribute a specific anomaly to an underlying business incident if the source of that anomaly isn’t understood.

How are Anomalies Detected?

Imagine that every morning for months you watch your neighbor leave his house at seven AM (regardless of the weather) to walk his dog. Then one morning, you look out your window expecting to see your neighbor walking down the street, but neither your neighbor, nor his dog are anywhere to be seen. Your brain notes this as unusual—or as an anomaly.

An anomaly detection system is a piece of software written to do the same thing that our brains do. The machine learning algorithms that power Anodot’s automated anomaly detection system utilize the latest in artificial intelligence (AI)—or real-time ‘autonomous analytics’—to provide immediate business intelligence, with the added benefit of scalability to process the increasing volume of business metrics. Anodot’s online machine learning algorithms work in a way similar to the human brain:

  1. Based on the data, Anodot’s algorithms create a model.
  2. This model, in turn, predicts the value of the next data point.
  3. If the model predicts behavior, but the next actual data point deviates significantly from the prediction, it is flagged as a potential anomaly.
  4. New data points are noted and captured to intelligently build-upon and update the model.

ABOUT ANODOT

Anodot was founded in 2014, and since its launch in January 2016 has been providing valuable business insights through anomaly detection to its customers in fintech, ad-tech, web apps, mobile apps, e-commerce and other data-heavy industries. Over 40% of the company’s customers are publicly traded companies, including Microsoft, Waze (a Google company), AppNexus, and many others. Anodot’s real time business incident detection uses patented machine learning algorithms to isolate and correlate issues across multiple parameters in real time, supporting rapid business decisions. Learn more at:  http://www.anodot.com/.


Tags:
Blogs

BACK TO BLOGS