Anomaly Detection

BlazeMeter supports anomaly detection techniques that can help teams maintain high-quality standards, ensuring that web applications perform reliably and efficiently.

Purpose

An anomaly, in the context of data analysis and system monitoring, is a data point or pattern that significantly deviates from the expected behavior or average trend.

Anomaly detection is part of the root cause analysis process. It surfaces issues that might not be identified by the other layers of defense, such as baseline, comparison, failure criteria, and so on. Anomaly detection involves identifying data points, events, or patterns that deviate significantly from the expected behavior. These anomalies can signal various issues such as performance degradation, security breaches, or system errors, allowing teams to address problems before they escalate.

BlazeMeter uses statistical models to implement anomaly detection.

Benefits

  • Detecting anomalies early in the testing process helps in resolving potential issues before they impact the user experience, ensuring smooth and reliable application performance.

  • Continuous monitoring for anomalies helps maintain optimal performance by identifying and addressing bottlenecks and other performance-related issues promptly.

BlazeMeter integrates anomaly detection into its continuous testing platform in the Timeline Report.

How BlazeMeter detects anomalies

BlazeMeter uses a statistical model that analyzes the response time metrics (such as average, median, 90th percentile, and so on) for every label in the test, including the All label. The model identifies abnormal behavior, defined as a significant increase or decrease in response time that deviates from the mean and exceeds a predefined threshold.

The model analyzes data points collected every second from all load generators in the test. For each data point, BlazeMeter calculates its distance from the mean, taking into account the distribution of data points by using the standard deviation. If the distance is greater than a certain threshold (X standard deviations from the mean, where X is defined by BlazeMeter), the data point is marked as abnormal.

However, a single abnormal data point does not constitute an anomaly, as occasional spikes are expected in some cases. Instead, BlazeMeter examines groups of data points over a specified time window. For a time window to be classified as an anomaly, a certain percentage of data points within that window must be marked as abnormal.

When a time window (group of data points) is identified as an anomaly, it is highlighted in Timeline charts with a bold purple line on the relevant label.

Video demo

The following video quickly summarizes how anomalies are displayed in the Timeline report.

Review detected anomalies

An Anomalies Detected notification in a report header indicates that the test run contains at least one anomaly.

To review each detected anomaly for further insights, follow these steps:

  1. Navigate to the Timeline tab.

    Anomalies are displayed on this tab. On the left-hand KPI Selection panel, anomalies are grouped under a purple Anomalies - Response Time category. This group includes all labels and response time metrics for which anomalies were detected.

  2. Choose a label from the “Anomalies - Response Time” group to display it on the chart.

  3. Zoom in.

    Once the label is displayed on the chart, you may find multiple anomalies detected for that label. To focus on a specific anomaly, zoom in on the time range where the anomaly was detected.

  4. Adjust the chart resolution.

    Change the chart resolution to 1 second to view the individual data points that make up the anomaly. This detailed view allows you to observe the deviation and assess whether it indicates an issue that requires further investigation.