Overview.
"Codeless Load Generator" is the most straightforward load generator to start with.
Such a load generator allows you to prototype load test(s) quickly without complex scenarios programming.
The only skills you need are understanding YAML format and basic knowledge of css or xpath selectors.
Such a load generator allows you to prototype load test(s) quickly without complex scenarios programming.
The only skills you need are understanding YAML format and basic knowledge of css or xpath selectors.
All integrations with the Perforator platform are open-sourced.
Please take a look at our GitLab repository "Perforator SDK Java" if you are interested in the internal implementation of the load generator.
Please take a look at our GitLab repository "Perforator SDK Java" if you are interested in the internal implementation of the load generator.
Configuration of this load generator is done in a single YAML file,
where you specify target load settings and actions to execute in remote browsers.
The example below can give you a sense of what a real-world configuration might look like.
The example below can give you a sense of what a real-world configuration might look like.
---
loadGenerator:
apiClientId: YOUR_API_CLIENT_ID
apiClientSecret: YOUR_API_CLIENT_SECRET
projectKey: YOUR_PROJECT_KEY
suites:
50 users for 5 mins:
concurrency: 50
duration: 5m
rampUp: 1m
rampDown: 30s
steps:
Open landing page and await to be loaded:
- open: https://verifications.perforator.io/
- awaitElementToBeVisible: '#async-container'
Verify page 1:
- click: ul.navbar-nav li:nth-child(1) a
- awaitElementToBeVisible: '#async-container'
Verify page 2:
- click: ul.navbar-nav li:nth-child(2) a
- awaitElementToBeVisible: '#async-container'Jump Start.
1.
Download and unarchive
prebuilt example.
2.
Start editing config.yml with your favorite text editor.
3.
Update apiClientId
property with your own value.
Please navigate to the API Settings page to get your API Client ID.
4.
Update apiClientSecret
property with your own value.
Please navigate to the API Settings page to get your API Client Secret.
5.
Update projectKey
property with your own value.
Please refer to the "Projects" chapter to learn more about projects and get your project key.
6.
Execute cloud-full-run.sh(on Linux/macOS) or cloud-full-run.cmd(on Windows)
using your command line terminal - it should start a lightweight load test
with 50 concurrent browsers in the cloud running for 5 minutes.
7.
Load generator prints a link to view performance test statistics
when all cloud resources are ready, and the actual load test begins.
Please copy such a link and open it in your browser.
Note: Please make sure that JDK 11+ is installed on your computer and available in your terminal PATH.
Execute `java -version` in your terminal to validate it.
Execute `java -version` in your terminal to validate it.
examples-codeless-release-1.4.2 % ./cloud-full-run.sh
[2022-01-26 06:35:49][INFO] - Created browser cloud 59d05c73-30a7-4a49-b71d-06a6ab8d6dc2
[2022-01-26 06:35:49][INFO] - Browser cloud is not ready yet. Status = QUEUED, RequestedBrowsers = 50, StartingBrowsers = 0, ReadyBrowsers = 0
[2022-01-26 06:35:55][INFO] - Browser cloud is not ready yet. Status = PROVISIONING, RequestedBrowsers = 50, StartingBrowsers = 50, ReadyBrowsers = 0
[2022-01-26 06:36:32][INFO] - Browser cloud is not ready yet. Status = PROVISIONING, RequestedBrowsers = 50, StartingBrowsers = 38, ReadyBrowsers = 12
[2022-01-26 06:36:52][INFO] - Browser cloud is ready: selenium hub = https://selenium.perforator.io/---/---/wd/hub
-------------------------------------------------------------------------------------------------------------------------------
| |
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||| |
| |
| Load generation is about to start. |
| Please open the link below in the browser to see statistics in real-time. |
| |
| https://app.perforator.io/statistics/73df630c-86b4-40e0-8ab4-a4ac2e9ec373/c1f8e6a0-7a41-4d72-88fb-c6c9f553b75e |
| |
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||| |
| |
-------------------------------------------------------------------------------------------------------------------------------
[2022-01-26 06:36:52][INFO] - suites: active = 1, failed = 0, successful = 0; transactions: active = 2, failed = 0, successful = 0
[2022-01-26 06:37:12][INFO] - suites: active = 18, failed = 0, successful = 25; transactions: active = 36, failed = 0, successful = 175
[2022-01-26 06:37:37][INFO] - suites: active = 39, failed = 0, successful = 137; transactions: active = 78, failed = 0, successful = 889
[2022-01-26 06:37:53][INFO] - suites: active = 50, failed = 0, successful = 249; transactions: active = 100, failed = 0, successful = 1563
[2022-01-26 06:38:33][INFO] - suites: active = 50, failed = 0, successful = 583; transactions: active = 100, failed = 0, successful = 3568
[2022-01-26 06:41:21][INFO] - suites: active = 50, failed = 0, successful = 1953; transactions: active = 100, failed = 0, successful = 11807
[2022-01-26 06:41:26][INFO] - suites: active = 11, failed = 0, successful = 1996; transactions: active = 22, failed = 0, successful = 12014
[2022-01-26 06:41:29][INFO] - Terminating browser cloud -> 59d05c73-30a7-4a49-b71d-06a6ab8d6dc2
[2022-01-26 06:41:30][INFO] - Browser cloud has been successfully terminated -> 59d05c73-30a7-4a49-b71d-06a6ab8d6dc2
[2022-01-26 06:41:30][INFO] - suites: active = 0, failed = 0, successful = 2007; transactions: active = 0, failed = 0, successful = 12042Configuration.
Load generator configuration file can be logically divided into the following major sections:
- loadGenerator - general settings applied on a whole load test, for example, API credentials, project key, etc.
- suite - suite level settings, for example, concurrency, duration, rampUp, etc.
- steps - a list of logical units to execute as part of the load test.
loadGenerator config section has three required settings:
- apiClientId - client ID for API authentication. Please navigate to the API Settings page to get your API Client ID.
- apiClientSecret - client Secret for API authentication. Please navigate to the API Settings page to get your API Client Secret.
- projectKey - identifier of the project to create new execution records. Please refer to the "Projects" chapter to learn more about projects and get your project key.
Additionally, the load generator supports many more optional settings allowing you to fine-tune load test execution.
Feel free to review all available options at the Settings tab paying attention to parameters with the "Scope: loadGenerator".
- loadGenerator - general settings applied on a whole load test, for example, API credentials, project key, etc.
- suite - suite level settings, for example, concurrency, duration, rampUp, etc.
- steps - a list of logical units to execute as part of the load test.
loadGenerator config section has three required settings:
- apiClientId - client ID for API authentication. Please navigate to the API Settings page to get your API Client ID.
- apiClientSecret - client Secret for API authentication. Please navigate to the API Settings page to get your API Client Secret.
- projectKey - identifier of the project to create new execution records. Please refer to the "Projects" chapter to learn more about projects and get your project key.
Additionally, the load generator supports many more optional settings allowing you to fine-tune load test execution.
Feel free to review all available options at the Settings tab paying attention to parameters with the "Scope: loadGenerator".
---
loadGenerator:
apiClientId: YOUR_API_CLIENT_ID
apiClientSecret: YOUR_API_CLIENT_SECRET
projectKey: YOUR_PROJECT_KEY
# ...
suites:
# ...
Your load test can contain one or more suites,
where a suite defines what kind of steps and browser actions to execute,
how long to repeat it,
and how many threads to use in parallel.
Every suite section has three required properties:
- concurrency - concurrency level of the suite, i.e., how many browsers to run in parallel.
- duration - duration of the suite.
- steps - logical units to execute sequentially as a part of every suite instance.
There are many more optional suite parameters.
Feel free to review all available options at the Settings tab, paying attention to "Scope: suite" parameters.
Every suite section has three required properties:
- concurrency - concurrency level of the suite, i.e., how many browsers to run in parallel.
- duration - duration of the suite.
- steps - logical units to execute sequentially as a part of every suite instance.
There are many more optional suite parameters.
Feel free to review all available options at the Settings tab, paying attention to "Scope: suite" parameters.
---
loadGenerator:
# ...
suites:
50 users for 5 mins: # This is your suite name
concurrency: 50
duration: 5m
steps:
Open landing page and await to be loaded: # This is a step name
# ...
Verify page 1: # This is a step name
# ...
A step is a logical unit of a suite instance processing.
You can treat a step as a sequence of actions towards completing a specific goal on the target website.
For example, you can have a step named "login", and such a step might have three sequential actions: 1. enter username 2. enter password 3. press submit button.
Every step defines a list of actions to execute in the remote browser.
Please take a look "Browser Actions" tab to learn more about available actions.
You can treat a step as a sequence of actions towards completing a specific goal on the target website.
For example, you can have a step named "login", and such a step might have three sequential actions: 1. enter username 2. enter password 3. press submit button.
Every step defines a list of actions to execute in the remote browser.
Please take a look "Browser Actions" tab to learn more about available actions.
---
loadGenerator:
# ...
suites:
50 users for 5 mins: # This is your suite name
# ...
steps:
Open landing page and await to be loaded: # This is a step name
- open: https://verifications.perforator.io/
- awaitElementToBeVisible: '#async-container'
Verify page 1: # This is a step name
- click: ul.navbar-nav li:nth-child(1) a
- awaitElementToBeVisible: '#async-container'Overrides.
You can override almost all configuration settings from the command line or via environment variables.
Load generator uses the following order while resolving specific setting values:
1. Command line arguments.
2. Environment variables.
3. config.yml setting.
This approach allows you to prepare a configuration file once and then execute it with different options.
A real-world example is when you store your config.yml in a version control system like GIT, and you don't want to expose security keys in such config.
So you can configure a new environment variable dedicated to the specific property, and a load generator automatically resolves it.
All Settings mention command-line argument and environment variable where an override is available.
For example, apiClientId has the following overrides:
- Command line arg: -DloadGenerator.apiClientId=...
- Environment variable: LOADGENERATOR_APICLIENTID=...
Load generator uses the following order while resolving specific setting values:
1. Command line arguments.
2. Environment variables.
3. config.yml setting.
This approach allows you to prepare a configuration file once and then execute it with different options.
A real-world example is when you store your config.yml in a version control system like GIT, and you don't want to expose security keys in such config.
So you can configure a new environment variable dedicated to the specific property, and a load generator automatically resolves it.
All Settings mention command-line argument and environment variable where an override is available.
For example, apiClientId has the following overrides:
- Command line arg: -DloadGenerator.apiClientId=...
- Environment variable: LOADGENERATOR_APICLIENTID=...
Logging.
Load generator uses Log4j
logging framework.
All codeless examples come with the prebuilt log4j2.xml configuration file, so you can easily fine-tune the logging of your load test.
Please take a look at the default logging config below:
All codeless examples come with the prebuilt log4j2.xml configuration file, so you can easily fine-tune the logging of your load test.
Please take a look at the default logging config below:
<?xml version="1.0" encoding="UTF-8"?>
<Configuration
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" strict="true"
xmlns="http://logging.apache.org/log4j/2.0/config"
xsi:schemaLocation="http://logging.apache.org/log4j/2.0/config
https://raw.githubusercontent.com/apache/logging-log4j2/master/log4j-core/src/main/resources/Log4j-config.xsd"
>
<Appenders>
<Console name="Console" target="SYSTEM_OUT">
<PatternLayout pattern="[%d][%X{X-PERFORATOR-CONTEXT}][%4p] - %m%n"/>
</Console>
</Appenders>
<Loggers>
<Root level="info">
<AppenderRef ref="Console"/>
</Root>
<Logger name="org.openqa.selenium" level="error" />
</Loggers>
</Configuration>
Load generator prints actions and steps by default when you execute it locally using local.sh or local.cmd scripts.
Such behavior helps you debug and investigate browser actions when preparing your load test configuration.
Such behavior helps you debug and investigate browser actions when preparing your load test configuration.
[2022-01-30 03:38:44][INFO] - Suite '50 users for 5 mins' has started a new step 'Open landing page'
[2022-01-30 03:38:44][INFO] - Step 'Open landing page' has started a new action 'open': url = https://verifications.perforator.io/, timeout = PT30S
[2022-01-30 03:38:45][INFO] - Step 'Open landing page' has completed action 'open'
[2022-01-30 03:38:45][INFO] - Step 'Open landing page' has started a new action 'awaitElementToBeVisible': cssSelector = #async-container, timeout = PT30S
[2022-01-30 03:38:45][INFO] - Step 'Open landing page' has completed action 'awaitElementToBeVisible'
[2022-01-30 03:38:45][INFO] - Suite '50 users for 5 mins' has completed step 'Open landing page'
Additionally to the log4j2.xml configuration, the load generator supports the following options:
- logWorkerID - it adds worker ID to the logging context of the current thread.
- logSuiteInstanceID - it adds suite instance ID to the logging context of the current thread.
- logRemoteWebDriverSessionID - it adds browser session ID to the logging context of the current thread.
- logTransactionID - it adds currently active transaction ID to the logging context of the current thread.
- logSteps - it enables steps logging.
- logActions - it enables actions logging.
Note: It is not suggested to enable logSteps and logActions when you run your load test at full speed because it will log too much information, and process output might become too overloaded, so please use it at your own risk.
- logWorkerID - it adds worker ID to the logging context of the current thread.
- logSuiteInstanceID - it adds suite instance ID to the logging context of the current thread.
- logRemoteWebDriverSessionID - it adds browser session ID to the logging context of the current thread.
- logTransactionID - it adds currently active transaction ID to the logging context of the current thread.
- logSteps - it enables steps logging.
- logActions - it enables actions logging.
Note: It is not suggested to enable logSteps and logActions when you run your load test at full speed because it will log too much information, and process output might become too overloaded, so please use it at your own risk.
Running.
All prebuilt examples come with ready-to-use scripts.
There is no enforced workflow on how to prepare and execute a load test, but here is a suggested one:
1. Make necessary changes in the configuration file.
2. Execute your config.yml using local browsers, so you can verify that all settings and browser actions are valid.
3. Dry run your config.yml using a small number of remote browsers and ensure that all browser actions are not affected by network delays.
4. Execute your config.yml at full speed
Please take a look at the scripts below prepared for the suggested workflow.
There is no enforced workflow on how to prepare and execute a load test, but here is a suggested one:
1. Make necessary changes in the configuration file.
2. Execute your config.yml using local browsers, so you can verify that all settings and browser actions are valid.
3. Dry run your config.yml using a small number of remote browsers and ensure that all browser actions are not affected by network delays.
4. Execute your config.yml at full speed
Please take a look at the scripts below prepared for the suggested workflow.
Linux / macOS:
- local.sh - executes your config.yml using Chrome browsers started locally. This script is helpful to debug and visually verify that everything works as intended.
- local-headless.sh - it has the same purpose as local.sh, but valuable for CI/CD environments where a graphical environment is unavailable.
- cloud-dry-run.sh - it processes your configuration file using only 10 browsers started in the cloud. The main idea of this script is to avoid extra charges when you need to ensure that actions are executed correctly in cloud-based browsers. For example, it is not optimal to run your full-powered configuration using thousands of browsers and then, in a minute, realize that you have a simple mistake in css/xpath selector.
- cloud-full-run.sh - executes your load test at full speed as defined in config.yml.
- local.sh - executes your config.yml using Chrome browsers started locally. This script is helpful to debug and visually verify that everything works as intended.
- local-headless.sh - it has the same purpose as local.sh, but valuable for CI/CD environments where a graphical environment is unavailable.
- cloud-dry-run.sh - it processes your configuration file using only 10 browsers started in the cloud. The main idea of this script is to avoid extra charges when you need to ensure that actions are executed correctly in cloud-based browsers. For example, it is not optimal to run your full-powered configuration using thousands of browsers and then, in a minute, realize that you have a simple mistake in css/xpath selector.
- cloud-full-run.sh - executes your load test at full speed as defined in config.yml.
Windows:
- local.cmd - executes your config.yml using Chrome browsers started locally. This script is helpful to debug and visually verify that everything works as intended.
- local-headless.cmd - it has the same purpose as local.sh, but valuable for CI/CD environments where a graphical environment is unavailable.
- cloud-dry-run.cmd - it processes your configuration file using only 10 browsers started in the cloud. The main idea of this script is to avoid extra charges when you need to ensure that actions are executed correctly in cloud-based browsers. For example, it is not optimal to run your full-powered configuration using thousands of browsers and then, in a minute, realize that you have a simple mistake in css/xpath selector.
- cloud-full-run.cmd - executes your load test at full speed as defined in config.yml.
- local.cmd - executes your config.yml using Chrome browsers started locally. This script is helpful to debug and visually verify that everything works as intended.
- local-headless.cmd - it has the same purpose as local.sh, but valuable for CI/CD environments where a graphical environment is unavailable.
- cloud-dry-run.cmd - it processes your configuration file using only 10 browsers started in the cloud. The main idea of this script is to avoid extra charges when you need to ensure that actions are executed correctly in cloud-based browsers. For example, it is not optimal to run your full-powered configuration using thousands of browsers and then, in a minute, realize that you have a simple mistake in css/xpath selector.
- cloud-full-run.cmd - executes your load test at full speed as defined in config.yml.
Note: Please make sure that JDK 11+ is installed on your computer and available in your terminal PATH.
Execute `java -version` in your terminal to validate it.
Execute `java -version` in your terminal to validate it.
Limitations.
Please keep in mind all the time that the load generator runs locally on your computer or server.
While browsers generating the load on your website are running in the cloud, the load generator itself is executed locally and is only responsible for orchestrating remote browsers.
Such behavior comes with certain requirements applied on local hardware and internet connection speed.
Here are minimal requirements to orchestrate every 500 remote browsers:
- 2 CPU cores
- 2 GB RAM
- 20 Mbps Internet bandwidth
For example, you need the following resources locally to run a load test with 2000 remote browsers:
- CPU cores: 2000 ÷ 500 × 2 Cores = 8 Cores
- RAM: 2000 ÷ 500 × 2GB = 8 GB
- Internet: 2000 ÷ 500 × 20Mbps = 80 Mbps
Warning: You might see very confusing load test statistics and high errors rate if you don't meet minimal hardware and internet bandwidth requirements.
While browsers generating the load on your website are running in the cloud, the load generator itself is executed locally and is only responsible for orchestrating remote browsers.
Such behavior comes with certain requirements applied on local hardware and internet connection speed.
Here are minimal requirements to orchestrate every 500 remote browsers:
- 2 CPU cores
- 2 GB RAM
- 20 Mbps Internet bandwidth
For example, you need the following resources locally to run a load test with 2000 remote browsers:
- CPU cores: 2000 ÷ 500 × 2 Cores = 8 Cores
- RAM: 2000 ÷ 500 × 2GB = 8 GB
- Internet: 2000 ÷ 500 × 20Mbps = 80 Mbps
Warning: You might see very confusing load test statistics and high errors rate if you don't meet minimal hardware and internet bandwidth requirements.
Load Generator Lifecycle.
1.
The load generator automatically validates your configuration file.
Execution is automatically stopped in case of any validation errors.
Here are a few noticeable validations:
- API credentials, according to apiClientId and apiClientSecret
- Project access, according to projectKey
- Suite(s) settings, its associated steps, and actions
- Credits balance
- Limits
Execution is automatically stopped in case of any validation errors.
Here are a few noticeable validations:
- API credentials, according to apiClientId and apiClientSecret
- Project access, according to projectKey
- Suite(s) settings, its associated steps, and actions
- Credits balance
- Limits
2.
The load generator automatically synchronizes time with
NTP
servers.
Time synchronization is needed to correctly report timestamps of the processed transactions on the load generator side.
Transaction timestamps should be adjusted according to the world clock to avoid confusing results of the execution statistics,
so the load generator automatically calculates the offset between the local clock and the world clock.
The following NTP servers are queried to get time adjustments:
- time.cloudflare.com
- time.google.com
- time.windows.com
- time.facebook.com
Note: Load generator terminates execution with an error if it can't get time adjustment from any of the servers above.
Time synchronization is needed to correctly report timestamps of the processed transactions on the load generator side.
Transaction timestamps should be adjusted according to the world clock to avoid confusing results of the execution statistics,
so the load generator automatically calculates the offset between the local clock and the world clock.
The following NTP servers are queried to get time adjustments:
- time.cloudflare.com
- time.google.com
- time.windows.com
- time.facebook.com
Note: Load generator terminates execution with an error if it can't get time adjustment from any of the servers above.
3.
The load generator automatically calculates dimensions to request a
browser cloud.
When you execute a lot test, the actual load on your website is coming from remote browsers launched in the cloud,
while the load generator is only responsible for orchestrating such browsers.
Every browser cloud has two primary dimensions: capacity and duration.
The load generator analyzes all suites defined in the config, determines max duration, and rounds it up to the nearest hour.
The load generator sums concurrency fields of all suites to calculate the capacity of the browser cloud.
When you execute a lot test, the actual load on your website is coming from remote browsers launched in the cloud,
while the load generator is only responsible for orchestrating such browsers.
Every browser cloud has two primary dimensions: capacity and duration.
The load generator analyzes all suites defined in the config, determines max duration, and rounds it up to the nearest hour.
The load generator sums concurrency fields of all suites to calculate the capacity of the browser cloud.
4.
The load generator automatically requests and awaits the browser cloud.
The Perforator automatically starts brand new servers to back the requested browser cloud.
So, it takes some time to wait till the required hardware is available and appropriately initialized.
The load generator automatically polls the status of the requested browser cloud according to the "browserCloudStatusPollInterval" property.
The actual load test only begins when the browser cloud is operational.
The Perforator automatically starts brand new servers to back the requested browser cloud.
So, it takes some time to wait till the required hardware is available and appropriately initialized.
The load generator automatically polls the status of the requested browser cloud according to the "browserCloudStatusPollInterval" property.
The actual load test only begins when the browser cloud is operational.
5.
The load generator executes suite instances concurrently.
A dedicated thread pool is created at the beginning of the load test.
Every thread from such a pool is responsible for processing only one suite instance at a time.
The size of such a thread pool is calculated as the sum of concurrency fields across all predefined suites.
The load generator automatically maintains desired concurrency via launching a new suite instance once any instance finishes.
Please check the next section, "Suite Instance Lifecycle", to get more details about the logic behind suite instance processing.
A dedicated thread pool is created at the beginning of the load test.
Every thread from such a pool is responsible for processing only one suite instance at a time.
The size of such a thread pool is calculated as the sum of concurrency fields across all predefined suites.
The load generator automatically maintains desired concurrency via launching a new suite instance once any instance finishes.
Please check the next section, "Suite Instance Lifecycle", to get more details about the logic behind suite instance processing.
6.
The load generator automatically ramps up the load.
It is a common requirement for the load test to increase the load on a target system evenly at the beginning of the test.
Such an approach allows the target system to warm up gradually, automatically initializing and optimizing its internal resources.
Target website might go down immediately if you apply all load in a spike manner from the first second.
You can adjust the rampUp property of the suite, and the load generator automatically calculates the speed of increasing the load.
Note: You can completely disable ramp-up behavior by specifying such a setting as 0s.
It is a common requirement for the load test to increase the load on a target system evenly at the beginning of the test.
Such an approach allows the target system to warm up gradually, automatically initializing and optimizing its internal resources.
Target website might go down immediately if you apply all load in a spike manner from the first second.
You can adjust the rampUp property of the suite, and the load generator automatically calculates the speed of increasing the load.
Note: You can completely disable ramp-up behavior by specifying such a setting as 0s.
7.
The load generator automatically ramps down the load.
It is impossible to determine how much time it will take to process a suite instance without executing it.
You can specify rampDown suite property, and the load generator stops processing new suites instances once the ramp-down phase happens.
The exact moment of the ramp-down phase is calculated as duration minus rampDown.
For example, if you have duration=17m and rampDown=3m, then the calculated moment of the ramp-down stage is 14m.
Please keep in mind that the load generator doesn't launch new suite instances once the ramp-down phase activates,
but existing suites instances will continue until their natural completion.
Note: You can completely disable ramp-down behavior by specifying such a setting as 0s.
It is impossible to determine how much time it will take to process a suite instance without executing it.
You can specify rampDown suite property, and the load generator stops processing new suites instances once the ramp-down phase happens.
The exact moment of the ramp-down phase is calculated as duration minus rampDown.
For example, if you have duration=17m and rampDown=3m, then the calculated moment of the ramp-down stage is 14m.
Please keep in mind that the load generator doesn't launch new suite instances once the ramp-down phase activates,
but existing suites instances will continue until their natural completion.
Note: You can completely disable ramp-down behavior by specifying such a setting as 0s.
8.
The load generator automatically terminates the browser cloud at the end of the test.
You can override the "browserCloudTerminateAutomatically" property if you need granular control over the cloud termination behavior.
You can override the "browserCloudTerminateAutomatically" property if you need granular control over the cloud termination behavior.
Suite Instance Lifecycle.
Every suite configuration defines the behavior of the load test.
In contrast to suite configuration, suite instance is responsible for one-off processing of the logic behind such a configuration.
Suite instance takes care of executing suite's logic only one time.
Suite instance completes whenever such one-off logic completes, and the load generator launches a brand new suite instance to continue execution.
The load generator maintains active suite instances in parallel up to the defined concurrency.
For example, suppose you have a suite with concurrency = 100, duration = 5 minutes, and an average processing time of the one-off logic is 30 seconds.
In that case, you should get around 1000 suite instances by the end of the test, calculated as 100 × (5 minutes ÷ 30 seconds).
Please take a look at the flow below to understand how suite instances are processed.
In contrast to suite configuration, suite instance is responsible for one-off processing of the logic behind such a configuration.
Suite instance takes care of executing suite's logic only one time.
Suite instance completes whenever such one-off logic completes, and the load generator launches a brand new suite instance to continue execution.
The load generator maintains active suite instances in parallel up to the defined concurrency.
For example, suppose you have a suite with concurrency = 100, duration = 5 minutes, and an average processing time of the one-off logic is 30 seconds.
In that case, you should get around 1000 suite instances by the end of the test, calculated as 100 × (5 minutes ÷ 30 seconds).
Please take a look at the flow below to understand how suite instances are processed.
1.
The load generator automatically creates a brand new suite instance
from the associated suite configuration.
Such instance is automatically assigned to one of the available threads from the pool for processing.
Every suite instance is reported to the analytical system as a "Top Level" transaction:
- Transaction ID is automatically generated in UUID format.
- Transaction Name is automatically picked from the suite name.
- Transaction Type is a constant value of "Top Level".
Note: This transaction is in an active state until the suite instance processing end, and the load generator marks it as successful if there were no processing errors.
Such instance is automatically assigned to one of the available threads from the pool for processing.
Every suite instance is reported to the analytical system as a "Top Level" transaction:
- Transaction ID is automatically generated in UUID format.
- Transaction Name is automatically picked from the suite name.
- Transaction Type is a constant value of "Top Level".
Note: This transaction is in an active state until the suite instance processing end, and the load generator marks it as successful if there were no processing errors.
2.
The load generator automatically establishes a brand new browser session.
Such action is also automatically reported to the analytical system as a nested transaction, so you can understand how creating a remote browser affects suite metrics.
This transaction has the following characteristics:
- Parent Transaction ID is automatically picked from the top-level transaction.
- Parent Transaction Name is automatically picked from the top-level transaction.
- Transaction ID is automatically generated in UUID format.
- Transaction Name is a constant value of "Open browser session".
- Transaction Type is a constant value of "Nested".
- Session ID is automatically picked from the browser session, if the browser session is established successfully
Note: The load generator automatically marks this transaction as failed if it can't create a new browser session.
"Top Level" transaction is marked as failed and suite instance processing halts.
Such action is also automatically reported to the analytical system as a nested transaction, so you can understand how creating a remote browser affects suite metrics.
This transaction has the following characteristics:
- Parent Transaction ID is automatically picked from the top-level transaction.
- Parent Transaction Name is automatically picked from the top-level transaction.
- Transaction ID is automatically generated in UUID format.
- Transaction Name is a constant value of "Open browser session".
- Transaction Type is a constant value of "Nested".
- Session ID is automatically picked from the browser session, if the browser session is established successfully
Note: The load generator automatically marks this transaction as failed if it can't create a new browser session.
"Top Level" transaction is marked as failed and suite instance processing halts.
3.
The load generator automatically executes steps.
A step is a logical unit of a suite instance processing.
You can treat a step as a sequence of actions towards completing a specific goal on the target website.
For example, you can have a step named "login", and such a step might have three sequential actions: 1. enter username 2. enter password 3. press submit button.
The load generator executes all steps of the suite instance one by one using an already established browser session.
Also, every step is automatically covered with the dedicated transaction having the following characteristics:
- Parent Transaction ID is automatically picked from the top-level transaction.
- Parent Transaction Name is automatically picked from the top-level transaction.
- Transaction ID is automatically generated in UUID format.
- Transaction Name is automatically picked from the step name.
- Transaction Type is a constant value of "Nested".
- Session ID is automatically picked from the browser session.
This transaction is marked as successful at the end of the step processing if all internal actions are complete successfully.
Note: The load generator automatically marks this transaction as failed in case of any action errors.
"Top Level" transaction is marked as failed, browser session terminates, and suite instance processing halts.
A step is a logical unit of a suite instance processing.
You can treat a step as a sequence of actions towards completing a specific goal on the target website.
For example, you can have a step named "login", and such a step might have three sequential actions: 1. enter username 2. enter password 3. press submit button.
The load generator executes all steps of the suite instance one by one using an already established browser session.
Also, every step is automatically covered with the dedicated transaction having the following characteristics:
- Parent Transaction ID is automatically picked from the top-level transaction.
- Parent Transaction Name is automatically picked from the top-level transaction.
- Transaction ID is automatically generated in UUID format.
- Transaction Name is automatically picked from the step name.
- Transaction Type is a constant value of "Nested".
- Session ID is automatically picked from the browser session.
This transaction is marked as successful at the end of the step processing if all internal actions are complete successfully.
Note: The load generator automatically marks this transaction as failed in case of any action errors.
"Top Level" transaction is marked as failed, browser session terminates, and suite instance processing halts.
4.
The load generator automatically executes browser actions.
Every browser action is translated to selenium commands and such commands are executed in a remote browser using an already established session.
All such actions are executed one by one.
Please check the "Browser Actions" tab to see a list of all available actions.
Note: In contrast to steps, the load generator doesn't report browser actions as dedicated transactions to the analytical system.
But any issue with the action leads to step-level transaction failure and halting of the suite instance.
Every browser action is translated to selenium commands and such commands are executed in a remote browser using an already established session.
All such actions are executed one by one.
Please check the "Browser Actions" tab to see a list of all available actions.
Note: In contrast to steps, the load generator doesn't report browser actions as dedicated transactions to the analytical system.
But any issue with the action leads to step-level transaction failure and halting of the suite instance.
5.
The load generator automatically destroys the browser session.
It is crucial to destroy the browser session at the end of the suite instance processing,
so the next launched instance will get an available slot in the cloud to establish a new browser session.
Such action is also automatically reported to the analytical system as a nested transaction, so you can understand how destroying a remote browser affects suite metrics.
This transaction has the following characteristics:
- Parent Transaction ID is automatically picked from the top-level transaction.
- Parent Transaction Name is automatically picked from the top-level transaction.
- Transaction ID is automatically generated in UUID format.
- Transaction Name is a constant value of "Close browser session".
- Transaction Type is a constant value of "Nested".
- Session ID is automatically picked from the browser session, if the browser session is established successfully
Note: The load generator automatically marks this transaction as failed if it can't destroy the session.
"Top Level" transaction is marked as failed as well.
It is crucial to destroy the browser session at the end of the suite instance processing,
so the next launched instance will get an available slot in the cloud to establish a new browser session.
Such action is also automatically reported to the analytical system as a nested transaction, so you can understand how destroying a remote browser affects suite metrics.
This transaction has the following characteristics:
- Parent Transaction ID is automatically picked from the top-level transaction.
- Parent Transaction Name is automatically picked from the top-level transaction.
- Transaction ID is automatically generated in UUID format.
- Transaction Name is a constant value of "Close browser session".
- Transaction Type is a constant value of "Nested".
- Session ID is automatically picked from the browser session, if the browser session is established successfully
Note: The load generator automatically marks this transaction as failed if it can't destroy the session.
"Top Level" transaction is marked as failed as well.