This page explains the feature that retries a finished batch run under the same conditions. By retrying the failed tests together, you can check your daily test results more quickly.
Retry from the batch run detail page
When a batch run has finished and it has one or more test cases that did not succeed (including Unresolved and Aborted), the batch run detail page shows the "Retry" menu. In the menu, choose "Retry all test cases" to run every test case in the batch run, or "Retry failed test cases" to run only the test cases that did not succeed.
The retry result is kept as a separate batch run result, and the original batch run and the retry result are linked.
If the batch run has several test patterns, two types of retry menus are shown. Use "Retry the whole batch run" to retry all patterns, and "Retry this test pattern" to retry a specific test pattern.
When you retry a specific test pattern, the pattern name is also shown in the batch run list.
Retry from the MagicPod Web API
You can also retry a batch run from the MagicPod Web API, using the endpoint below.
/v1.0/{organization_name}/{project_name}/batch-run/{batch_run_number}/retry/
targetspecifies whether to run all test cases or only the failed ones.patternsspecifies which patterns to retry when the batch run has several patterns.
How to use this feature together with mid-run retry for failed tests
In addition to the retry feature on this page, MagicPod also has mid-run retry for failed tests. This section explains the difference between the two features. Please understand what each feature is for, and combine them as needed.
| Mid-run retry for failed tests | Retry feature (this page) | |
|---|---|---|
| When the retry happens | During a batch run | After a batch run finishes |
| What is retried | One test that has just failed | All tests, or all failed tests |
| How the results before and after the retry are shown | Only the logs, inside a single test result | As separate batch run results |
What mid-run retry for failed tests is good for
-
Good use cases
- You have unstable test cases that fail often but succeed when they run again
- You want to get a result where everything succeeded in one run, without running the tests again
-
Difficulties
- If there is a real bug on top of the unstable problem, repeating the retry makes the run take longer
- You may miss a bug that fails on the first run and succeeds on the second run
What the retry feature on this page is good for
-
Good use cases
- In a daily run, you want to confirm that all test cases finally passed
- You want to confirm a fix after several test cases failed for the same reason
-
Difficulties
- You cannot retry until the batch run has finished (Aborted is also fine)
Notes and limitations
-
You cannot retry a batch run that does not specify a test settings number. Also, a retry uses the settings saved under that number as-is, so you cannot overwrite part of the settings. The concrete cases where you cannot retry are:
-
When you choose
detailed settingsfor command line execution - When you overwrite settings following Run Tests Using a Batch Run Settings Number While Overwriting Some Settings
- In the local PC environment, when you follow Specifying test settings directly section for command line execution
-
- A retry finds the test settings by number, and the test pattern by name. So you cannot retry if you have renamed the test pattern or deleted the test settings or the test pattern. (Renaming the test settings is fine.)
- This retry feature runs the batch run again using the latest test cases and settings. If you change a test case or the settings after the batch run, the change is applied. If you delete a test case, it is not run. (The rest of the test cases are run.)
- For a run with data patterns, only the failed patterns are run again. However, if you repeat the same test case with a setting like
1*10, what is retried may differ from what you expect.
Q&A
Does the status of the original batch run change to "Succeeded" when the retry succeeds?
Even when the retry succeeds, the original batch run status does not change. The retry result remains a separate batch run result, but the original batch run and the retry result are linked. See Retry from the batch run detail page section for details.