Parallel run in MagicPod refers to running multiple patterns simultaneously within a single batch run.
This page explains how to configure parallel run and its use cases.
※This page explains parallel run in the cloud environment. For a parallel run in the local environment, please refer to the page below.
Run batch tests from the command line (Local PC environment)
Table of contents
- What is a parallel run?
- Benefits of parallel run
- How to configure a parallel run
- Adding the number of cloud devices
- Q&A
What is a parallel run?
Normally, one cloud device is used per batch run.
You can run tests in parallel by adding patterns in the Batch run settings screen.
Benefits of parallel run
Parallel run enables significant time savings by running multiple test cases simultaneously.
Additionally, by running multi-device and multi-browser tests in parallel, you can reduce test execution time and accelerate release cycles. This allows development teams to dedicate more time to developing new features, improving overall development productivity.
Early bug detection and rapid feedback loops enable efficient release of high-quality software.
How to configure a parallel run
We will explain how to configure for each parallel run use case.
When running tests in parallel across multiple types of browsers and devices
Run batch tests collectively across different types of browsers and various device versions and models.
First, click ︙ > Add new settings.
The following dialog will appear. Enter a preferred name, select Multi-browser pattern settings(Multi-device pattern settings for mobile app tests), and click OK.
The new run settings will be added. As Multiple browser pattern settings were selected, two patterns are created by default.
To add more patterns, click Add.
You can rename the patterns by clicking ︙ to the right of the pattern name.
Then, specify the browser for each newly added pattern.
Next, configure the Running method on the Common Settings page.
To run the tests, select your newly added run setting and run it.
This will execute the batch run across the specified multiple browsers.
With this method, tests are executed in a batch run across multiple browsers at once.
- In the case of Remote TestKit, a 10-minute waiting period may occur at the start of the test to process the return of the used device.
- By default, testing on the next device will not start until the test on the current device is completed. However, if you have a subscription plan with BrowserStack or Remote TestKit that allows parallel testing on multiple devices, you can set the concurrency in MagicPod, as shown above. This will enable parallel test execution across multiple devices.
When running tests in parallel to reduce test execution time
Reduce test execution time by running tests in parallel using a combination of labels and browser/device patterns.
1. Create labels
Go to the Settings tab on the project page > the Test case labels page, and click Add.
Enter any name you like and click Create.
2. Assign labels to test cases
Open the Info tab for the test case you wish to assign, check the appropriate label, and click Update.
You can see the labels assigned on the test case list page.
3. Edit batch run settings
Open Batch run settings, click Add to add patterns.
Open the Batch run target tab, select Filter by folders/labels as the Batch run target, and check the Filter by labels.
After that, click the Setup button to select the target labels.
In the pattern "Flow A", drag and drop the test case label Daily run (Flow A) into Includes.
In the pattern "Flow B", drag and drop Daily run (Flow B) as well.
Then, click Common settings and switch to Parallel run in Running method.
Then, when run in batch, Flow A and Flow B are executed in parallel, like this.
You can also see how the labels are set.
If you wish to specify the order of execution within a label, you can do so by sorting.
Reference: Run only specific test cases as a batch - 4. (To specify the order of execution within a label)
Instead of labels, test case numbers can also be specified.
Adding the number of cloud devices
For the Standard plan, you can run tests in parallel with 2 cloud devices. You can add cloud devices as options.
References: How to increase test cases and projects
If you have any questions, please feel free to contact us.
Q&A
Q. Can multiple cloud devices be used in batch runs without pattern configuration?
No. A single batch run without multiple patterns will only use one cloud device.
A single batch run without pattern configuration uses only one cloud device. If two devices are available, it is possible to run two batch runs simultaneously. When running two batch runs at the same time, you can start a new batch run as soon as one batch run completes.
You can add concurrent execution devices as options.
Reference: How to increase test cases and projects
Q. What happens if I try to run more parallel patterns than my concurrency limit?
The same number of patterns as the number set in "Concurrency" in the batch run's common settings will run in parallel, and then the remaining patterns will execute as soon as cloud devices become available.
For example, suppose you set the "Concurrency" count to 2 and configure 3 execution patterns.
In this case, any 2 of the 3 patterns will run in parallel first, and the remaining pattern will execute as soon as a cloud device becomes available. Test cases within a single pattern are always executed sequentially.
Q. What happens if there are insufficient cloud devices at the scheduled execution time?
The number of available cloud devices at test start time becomes the maximum for parallel execution.
If no execution devices are available at the time of scheduled execution start, the test will not start and will be treated as failed.
If even one execution device is available at the time of scheduled execution start, the test will begin.
When parallel execution is scheduled, the number of available execution devices at test start time becomes the maximum for parallel execution.
For example, suppose you have 3 cloud devices with 1 in use, and you want to run a batch run with 3 patterns in parallel.
In this case, since there are 2 available devices at test start time, the test will run with a maximum of 2 parallel executions. Even if cloud devices become available during test execution, the number of parallel executions will not change.