Synthetic Test Data for Negative Tests and Chaos Testing

There are only few ways for users to get a process right (the Happy Path), and countless ways of getting it wrong (the Unhappy Paths). In practice, the majority of your tests will be so-called negative tests of Unhappy Paths. Negative assertions verify that your app is indeed catching exceptions that you know to avoid, and that it offers helpful validation or error messages and ways for users to recover gracefully.

However, even negative and positive tests merely test the well-trodden paths that users take, whereas reality is unpredictable. Chaos testing refers to testing unexpected errors that occur, for example, when someone sends requests in the wrong formats or when responses are delayed. Coming up with chaotic test data takes even more effort than writing tests for straight-forward Happy and Unhappy Paths.

BlazeMeter's test data integration helps you save time by generating negative and chaotic test data as part of the Probabilistic Distribution settings. You control how the negative cases are pseudo-randomly distributed.

  1. Open any test or virtual service transaction.

  2. Open the Test Data/Service Data pane, and open a Data Entity.

    If you want to toggle between positive and negative, create a Data Variant for the negative cases now.

  3. Find the Data Parameter for which you need negative test data and click its Distribution button.

    The Synthetic Data Generation By Distribution dialog opens.

  4. Click the Suggest Negative Data button and choose a suggestion:

    • Out of bounds numbers, extreme values.

    • Negative numbers where positive numbers are expected, and vice versa.

    • Text where numbers where expected, and vice versa.

    • Strings that are too long or too short for the text field.

    • Malformatted text, wrong delimiters, or text with invalid characters.

    • Extra numbers or characters at the beginning or the end.

    • Credit cards with correct checksum but for the wrong vendor.

    • Times and dates with invalid formats, or with valid formats but not the requested one.

    • Empty values, and so on.

      If there are no suggestions for your test data, contact Support and describe your use case.

  5. Click Save.

Run the test to see how your app reacts to chaotic and negative test data.

For more information on providing alternative values and their probabilities, see Random Distributions.