The following example shows a possible first day experience in Perforce Autonomous Testing.
Since the answers are generated by the assistant, every first-day conversation will be different. Simply ask the assistant what it can do for you. If something is unclear, ask it to explain.
Example - Ask for clarification any time
You are a first time user who is curious what Perforce Autonomous Testing can do, so you ask it questions about itself.
User: I don't understand the structure of all this, applications, scenarios, policies?
Assistant:
Think of it like this:
- Application: This is the highest-level container, representing the software you want to test (e.g., your e-commerce website or your mobile banking app).
- Scenario: This is an individual test case that describes a single path a user might take. For example, "User logs in successfully" or "User adds an item to the cart."
- Test Suite: A group of individual scenarios that you run together. For example, you could have a "Login" suite that includes scenarios for a valid login, an invalid login, and a forgotten password.
- Policy: These are rules that are checked during a test run. They aren't part of the test steps, but rather a measure of quality. For example, you could have a Performance Policy that says "no page should take longer than 3 seconds to load." If a page violates this, the test might still pass functionally, but the policy violation will be flagged.
- Environment: This is the configuration your tests run on. It defines the devices, browsers, and network conditions for the test execution.
So, you run a Test Suite (a collection of Scenarios) against your Application in a specific Environment, and the whole process is checked against the rules in your Policies.
Do you want me to help you create your first application to get started?
Example - Create your very first test
No application has been configured yet. You are a first-time user and want to test your web site's performance. You have not provided details in your first request and therefore you will receive helpful guidance. You see that you can ask the assistant clarifying questions any time.
User: How do I run a test?
Assistant:
I can help you create testing resources. First, let's create your new application. Please provide a name and an optional description for it.
User: What exactly do you mean by creating an application?
Assistant:
In Perforce Autonomous Testing, an "application" is a container that represents the software you want to test. It's the top-level organizational item that holds all of your other testing resources, including:
-
Environments: The specific URLs, devices, and configurations you'll test against (e.g., "Staging," "Production").
-
Test Scenarios: The individual test cases or user stories you want to validate (e.g., "User Login," "Add to Cart").
-
Test Suites: Collections of test scenarios that you run together (e.g., "Smoke Tests," "Regression Tests").
-
Policies: The rules and standards for performance, UX, and security that your application must meet.
Think of it as the main folder for a specific project or app you are testing.
Do you want me to go ahead and create your first application?
User: yes
Assistant: Let's create one now. Please provide a name and an optional description for your new application.
User:
-
Name: ExampleCom
-
Description: the customer facing documentation portal
User clicks the Create Application button.
Assistant:
Excellent! The application "ExampleCom" has been created. I've updated your settings to make it the active application.
Now, let's move on to the next step: defining a test environment. Please use the form below to configure your environment.
User:
Name: ExampleCom website
-
Enter Application web URL: http://example.com
(Note: If you were testing a mobile app, you would upload an .apk, .aab, or .ipa file here.) -
Mobile Devices: Samsung Galaxy 9, iPhone 8, and the latest iPhone
-
Web Devices: the latest Firefox
-
Load: Small (10VUs)
-
Duration (min): 10
-
Ram Up Time (minutes): 2
-
Max Users Per Engine: 4
-
Load distribution: Location: Sandbox 100% of traffic, 10 users.
The user clicks the Add Environment button.
Assistant:
Great, the "ExampleCom website" environment has been created.
Next up is defining test policies. Please use the form to create a new policy for your application. You can write your policy in the editor, and I will help you convert it into structured rules.
User: What do you mean by policy?
Assistant:
In Perforce Autonomous Testing, a policy is a set of rules that define the expected performance, user experience (UX), and security standards for your application. When you run a test, these policies are automatically checked, and any violations are flagged in the test report. [...]
By using policies, you can automate the process of checking that your application meets your quality standards with every test run.
Please use the form to create a new policy for your application, and I will help you convert it into structured rules.
User:
Policy name: ExampleCom page load
Policy description:
-
The system needs to respond to requests in less than 2 secs. (Performance)
-
Only less than 5% of requests can take longer than 500 ms. (Performance)
The user clicks Update Policy, then Save Policy.
Assistant:
Excellent. I have created the new policy "ExampleCom page load" with the rules you defined.
Now that the policy is created, I'll mark that task as complete and move to the next one: creating test scenarios.
To get started, please describe a user flow or a test you'd like to create. For example, you could say "test the login process" or "verify that a user can add an item to the cart". Do you want me to help you write the first test scenario?
User: yes
Assistant:
Great. Please describe the test scenario you have in mind. For example, you could describe the steps to log in, search for a product, or add an item to the cart. I will then convert your description into a structured test case.
Do you want me to start with a common scenario like user login?
User: No, I want to test the search filter. Use the search to find out how to restore an archived report.
Assistant:
Of course. I've broken down your request into the following test scenario.
Scenario Name: Search for Archived Report Restoration
Test Steps:
-
Action: Navigate to the documentation homepage.
-
Action: Locate the search bar.
-
Action: Enter the text "how to restore an archived report" into the search bar.
-
Action: Initiate the search.
-
✅ Validation: Verify that a list of search results is displayed.
-
✅ Validation: Confirm that the results include at least one article related to "restoring an archived report".
Summary: 6 steps (4 actions, 2 validations)
Do you want me to create this test scenario with the steps listed above?
User: Add a step after step 2, select reporting as search filter.
Assistant:
No problem. I've added that step to the test scenario.
Here is the updated version:
Scenario Name: Search for Archived Report Restoration
Test Steps:
-
Action: Navigate to the documentation homepage.
-
Action: Locate the search bar.
-
Action: Select "reporting" as the search filter.
-
Action: Enter the text "how to restore an archived report" into the search bar.
-
Action: Initiate the search.
-
✅ Validation: Verify that a list of search results is displayed.
-
✅ Validation: Confirm that the results include at least one article related to "restoring an archived report".
Summary: 7 steps (5 actions, 2 validations)
Does this updated scenario look correct? Do you want me to create it now?
User: yes.
Assistant:
I have successfully created the test scenario "Search for Archived Report Restoration".
We are currently working on creating test scenarios for your application. Do you want me to add another scenario?
User: No. How do I run this test now?
Assistant:
To run this test, we first need to add it to a test suite. Test suites allow you to group and execute one or more scenarios together.
Since we're now organizing scenarios, I'll update our to-do list.
Let's create a new suite for this test. What would you like to name it?
User: ExampleCom navigation
Assistant:
I have created the "ExampleCom navigation" test suite. Now I will add the "Search for Archived Report Restoration" scenario to it.
The scenario has been added to the suite.
Now, I will run the "ExampleCom navigation" test suite on the "ExampleCom website" environment.
The test suite execution has started. I will let you know once the results are in.
Do you want me to check the status of the execution now?
User: Show me the test report when it's ready
Assistant:
Of course. I will monitor the test execution and show you the report as soon as it's ready.
The test suite is still running. I will check the status again in a few moments and update you as soon as it's complete.