By using the Assert no visual diff command, you can check whether the display on the screen deviates from the expected image. This enables the detection of unexpected screen design issues during the test execution.
On this page, we will explain how to use the Assert no visual diff feature.
Table of contents
- Basic usage
- Add Visual diff patterns
- Skip Visual diff check
- How to check vertically long pages
- Points to note when running tests from the local PC command line
Basic usage
1. Set up expected images
First, add the Assert no visual diff command.
Ensure that the element selection mode is deactivated.
Run the test. *Do not click Edit for the command yet.
This will generate the expected image for confirmation, and the test run result will show as Unresolved. Click Details.
Once you are on the test result screen, click the message.
This will display a dialog, where you can check the content of the image and press Approve.
When approved, this image will be saved as the expected image and will be used for subsequent difference checks.
1-1. Set a diff threshold and exclusion areas
Then, return to the test editing screen and click the Edit button for the command.
In the dialog, you can use drag and drop to specify a diff threshold (the allowable mismatch rate for triggering an error) and specify areas to exclude from the diff check.
2. Run a test
Once the expected image is generated, the visual diff check will be executed in subsequent test runs.
If there is a difference, a test result status will show as Unresolved.
Clicking the error message will display a dialog, allowing you to view the details of the difference.
If there are several unresolved visual diff checks, it is recommended to check all of them from the Visual diffs tab.
In the dialog, checking the Show diff image checkbox highlights the areas where differences occurred in red.
If there are no issues with the difference, you can click Approve to use the new image as the expected image for subsequent executions. If differences are occurring as a result of issues with the site or app being tested, click Mark as an error to treat this test result as an error. Run the test again after fixing the problem with the site or app.
Add Visual diff patterns
When you want to run the same test in various environments, the expected images for the visual diff check will differ depending on the type of browser or device. In such cases, you want to prepare the correct image for each environment by adding a visual diff pattern from the test run settings dialog. You can then switch patterns for each environment and execute the test.
You can specify which pattern to use when running each test from the test run settings dialog. (The edit dialog for the expected image also has a visual diff pattern selection box, but this is used when switching the image displayed in the dialog box, rather than specifying the pattern to be used during test execution. )
Skip Visual diff check
For example, when you want to run tests that are usually executed in a CI environment on your local PC for debugging purposes, differences in the environment will almost certainly result in unexpected visual differences. In such cases, you can specify Skip for the Visual diff pattern in the test run settings dialog to avoid executing the visual diff check.
How to check vertically long pages
Currently, it is not possible to check vertically long pages with a single visual diff check. (Feature request is here)
As an alternative, you can test a vertically long page by repeating the following steps:
"Assert no visual diff" -> "Scroll (Swipe) until the specified element at the bottom of the page is visible" -> "Assert no visual diff".
Points to note when executing from the local PC command line
To run the command line on your local PC using MagicPodDesktop, you need to add a visualDiffPattern entry to your settings file. See here for details.