Planning Your Tests

Creating and running effective performance tests require proper planning.

This section, dedicated to making sure that you are properly implementing these various elements into your performance testing process, is important to read before running any performance tests.

Things To Consider During Performance Test Planning

As you can see so far, a lot goes into planning an effective performance test.

Below are the most important elements that need exploring while planning a performance test. Focusing on them will ensure the test is effictive and efficient.

  • Identify the types of performance tests required (stress / load / endurance) based on the business requirements provided.
  • Ensure that you have up-to-date performance targets and any Service Level Agreements (SLAs) determined before starting the performance tests.
    • Note: If your organization does not have Service Level Agreements for performance in place, you should work on identifying and documenting one.
    • Note: Your technical team(s) must be aware of, and discuss SLAs and their feasibility with all stakeholders before plans are final, or updated.
  • Performance tests should not occur on production environment sites and servers except for acute or specific scenarios;
  • Ensure that the performance test environment is an exact (or close to exact) replica of production environment;
  • Base all performance tests on business targets, objectives, or potential issues that require validating - don't try to conduct tests without a clear objective or scope;
  • Add "ThinkTime" and "pacing time" to scripts. They should replicate the production scenarios;
  • Use co-relation points and data files to make scripts more dynamic and the test scenario more like production;
  • The test environment should be manually tested to ensure that workflows execute properly with one user;
  • Clear all logs from the controller and load generator machines after every 3-4 tests so that the test numbers are not affected in a negative manner

Stakeholders Involved And Their Roles

As performance testing varies greatly, depending on your application and industry, it's important to consider the various roles and stakeholders in a test, and who to include.

Here are some key stakeholders you should consider and the roles they play throughout the performance testing process:

Customers or "End Users"

  • Customers help shape the requirements of the application through feature requests, reported bugs, or general feedback.
  • Customers play a key role in shaping performance testing requirements by helping your business analysts determine the appropriate service level agreements for the application.

Business Analysts, Project Managers, And Product Owners

  • Business analysts, project managers and product owners provide business level targets to test managers. These targets can include elements like:
    • The application should be able to meet a user load greater than X users while meeting the service level agreements;
    • All pages should take less than one second to load upon request.
  • These stakeholders should help ensure that test managers understand all performance level requirements (geography, application behaviour expected, device use, etc)
  • These stakeholders should help test managers finalize all tests.

Performance Test Managers

  • These stakeholders play a key role in the service level agreement identification process;
  • These stakeholders create the performance test strategy and plan the necessary performance tests in the study;
  • These stakeholders communicate all the requirements to the performance test analysts;
  • The performance test manager's role, and the performance test analyst role, are the same individual.

Performance Test Analysts

  • A performance test analyst plays a key role in performance testing projects as they understand the performance test requirements and convert those into business scripts, work flow models & documents;
  • These stakeholders setup the test architecture, manage tool infrastructure, do performance test pre-setup tasks, execute tests, monitor tests and analyze the final results;
  • These stakeholders compare the most recent results with the results from earlier versions and provide summary points from the performance test to each stakeholder;
  • In the case of performance bugs, they log the issue and follow-up with developers and performance engineers to get those resolved;

Performance Engineers And Developers

  • Performance engineers also play a key role in performance tests. In the case of any performance issues, they debug the system and resolve performance issues;
  • These stakeholders check the technical performance of the service level agreements during the planning phase.

Quantifying Performance Acceptance Criteria

Defining your performance testing criteria is essential to performance testing success.

By determining your performance service level agreements (SLAs), you're ensuring that the performance requirements of the business is quantifiable and that the performance testing teams are aware of the goals so they have a target to achieve.

It's not always a simple task as this can require a lot of communication and negotiation between project stakeholders. Business analysts should first try to understand the application, the functionality required, and the performance expectations. They then document these and discuss them with business stakeholders and customers.

For performance testing, here are the steps to follow in order to quantify criteria and have the testing up to individual standards:

  • Business analysts, project managers, and product owners interview customers and business stakeholders to understand performance expectations.
  • Analyze competitors (if business counterparts haven't already) in the market to understand existing performance targets that competitor products have set and are achieving.
  • Contact technical teams and discuss the business performance targets with them. Understand the feasibility of these targets.
  • If there are conflicting expectations, connect the business and technical stakeholders to determine a universal, absolute performance expectation for all stakeholders.
  • Document the final service level agreements and share it with the stakeholders.