If we use the “Assert no visual diff” function, we can check whether the image display differs from the expected value. Based on this, it will be possible to detect unexpected screen design corruption, etc., during test execution.
On this page, we will explain how to use the "Assert no visual diff" function.
Generation of expected value images
First, add the “Assert no visual diff” command and run the test. *Don’t click "Edit."
Once done, the expected value image will be generated and the test result will be “Unresolved.”
So click "Detail."
Click the message on the test result screen.
If there are no problems with the image content, press "Approve".
And this image will be saved as an expected value image and used for the next differential check.
After approval, return to the Edit test screen, and open the dialog box from the command “Edit” button, whereupon you can specify whether to generate an error or to exclude the area when checking.
Running the test
Once the expected value image is generated once, Visual diff checks will be performed on subsequent test executions. If there is a difference, the test result will be "Unresolved." In this case, confirm the content of the difference on the test edit screen. If there is no problem in terms of content differences, you can click “Approve” and use this image as the next expected value image. If this difference is caused by a problem with the site or application being tested, please treat this test result as an error by pressing "Mark as error" and run the test again after fixing the problem with the site or application.
At this time, if you check “Show diff image,” you can display the area(s) where the differential has occurred in red.
Adding “ Visual diff confirmation patterns”
In order to run the same test in various environments, you will want to prepare the correct image for each environment. This is the case even for the same step, because the expected value image for the Visual diff check will vary depending on the type of browser or device. In such cases, please add " Visual diff confirmation patterns" for each environment from the Test Target Specification Panel, and run the test after switching the pattern for each environment.
You can specify which pattern to use when running each test from the Test Target Specification Panel. (The Edit dialog for the expected value image also has a " Visual diff Check Pattern" selection box, but this is used when switching the image displayed in the dialog box, rather than specifying the pattern to be used during test execution. )
Skipping Visual diff check
For example, for tests run in a normal CI environment run on your own local PC for debugging purposes, there will almost certainly be unexpected image differences due to environmental differences. In such a case, specify “Skip” for the “ Visual diff pattern” on the Test Target Specification Panel to omit execution of the Visual diff check.
Points to note when running from the local PC command line
To run the command line on your local PC using MagicPodDesktop, you need to add a visualDiffPattern entry to your settings file. See here for details.