Validate responses with assertions

Assertions allow you to specify expected data in the response to a request made in a test run. When a test is run, the outcome is determined by whether or not all the assertions pass. If any assertion fails, the test fails. Assertions can be made against response header values, status code, response time/size, and content (like JSON or XML).

Define assertions

You can define zero or more assertions for any request in a test. The response data from the original request and the result of the last run are provided below the assertion editor for reference.

Each assertion consists of four items:

Source
The location of the data to extract for comparison. Data can be extracted from HTTP header values and JSON, XML or text body content. You can also create assertions based on the response status code, time and size.

Property
The property of the source data to retrieve. For HTTP headers, this is the name of the header. For XML and JSON content, see below. Unused for text content, status code, response time and response size.

Comparison
The type of operation to perform when comparing the extracted data with the target value. For more information, see Compare values in standard assertions.

Target value
The expected value used to compare against the actual value. The target value can contain a static value or a variable using the template syntax.

You can assert against data from JSON or XML:

  • Asserting against data from JSON body content
    Data from a JSON response body can be extracted by specifying the path of the data using standard JavaScript notation. View sample JSON expressions.
  • Asserting against data from XML body content
    Data from an XML response body can be extracted by specifying the path of the data using XPath. In the 'Property' box of an assertion definition that uses 'XML Body' as the source, enter an XPath expression to locate the data to extract.
You can also create an assertion for DNS error. The custom status code for DNS-specific failures is 50000.

Use semantic assertions

Semantic assertions let you validate the meaning of a response instead of matching an exact text value. They are useful for dynamic responses, such as AI-generated content, where the wording can vary while the intent remains correct.

Create a semantic assertion from the Assertions tab in the test editor. Semantic assertions compare the actual response value with the expected value and apply a sensitivity preset to determine whether the response passes.

The available sensitivity presets are:

  • Lenient is the default and works best for most paraphrased responses, such as two differently worded answers that have the same intent.
  • Moderate requires a closer match between the expected and actual meaning, such as standard support or workflow messages with small acceptable variation.
  • Strict is best for high-stakes cases such as payment, authentication, or approval confirmations, where only near-duplicate wording should pass.

After a test run, semantic assertions appear in the results with the compared values, a similarity score, and a status label.

Use standard assertions for exact status codes, short keywords, or other values that must match literally.

Do not use semantic assertions when negation or numeric differences must be interpreted exactly. For example, responses such as "Your order was confirmed" and "Your order was not confirmed" can appear semantically similar even though their meanings differ. Very short inputs can also reduce accuracy.

Compare values in standard assertions

The following comparison operators define how standard assertions compare the Actual Value with the Target Value. The Actual Value consists of the data located by Source and Property.

is empty The actual value exists and is an empty string or null.
is not empty The actual value exists and is a value other than an empty string or null.
equals A string comparison of the actual and expected value. Non-string values are cast to a string before comparing. For comparing non-integer numbers, use equals (number).
does not equal A string comparison of the actual and target value.
contains The actual value contains the target value as a substring.
does not contain The target value is not found within the actual value.
has key Checks for the existence of the expected value within a dictionary's keys. The actual value must point to a dictionary (JSON only).
has value Checks a list or dictionary for the existence of the expected value in any of the list or dictionary values. The actual value must point to a JSON list or dictionary (JSON only).
is null Checks that a value for a given JSON key is null.
is a number Validates the actual value is (or can be cast to) a valid numeric value.
less than Validates the actual value is (or can be cast to) a number less than the target value.
less than or equal Validates the actual value is (or can be cast to) a number less than or equal to the target value.
greater than Validates the actual value is (or can be cast to) a number greater than the target value.
greater than or equal Validates the actual value is (or can be cast to) a number greater than or equal to the target value.
equals (number) Validates the actual value is (or can be cast to) a number equal to the target value. This setting performs a numeric comparison: for example, "1.000" would be considered equal to "1".

More advanced assertions can be created by using Scripts.