By utilizing the 'Data Patterns' feature, you can conduct what is known as 'Data-driven testing,' where a single test script is executed repeatedly with varying input data patterns.
Table of contents
- Set up data patterns
- Run the test and check results
- Set data patterns via CSV format
- Set data patterns via API
- Use conditional branching commands
- Use variable-based locators
- Include a newline in patterns
- Limitations
- References
Set up data patterns
From the edit test menu on the edit test case screen, select Set Up Data Pattern (Figure 1).
Figure 1: Set up data pattern
In the dialog that appears, define the variable names and data patterns. For example, in Figure 2, two data patterns are defined for the variables UNAME and GENDER. After defining the patterns, click the OK button to save the changes.
Figure 2: Define data patterns
Next, create the script to be executed. In the script, data pattern variables can be referenced using the format ${UNAME} or ${GENDER} (Figure 3).
Figure 3: Script referencing data pattern variables
Run the test and check results
When you run the test you created, you will see that the test is run repeatedly for each data pattern. In the test results list screen, as shown in Figure 4, you can view a breakdown of the success and failure counts of all data patterns. And in the test results detail screen, as shown in Figure 5, you can check the test result for each data pattern.
Figure 4: Test results list screen
Figure 5: Test results detail screen.
If you want to execute a specific data pattern in a batch run, specify it as "3[2], 3[3]".
For example, "3[2]" specifies the second data pattern of the test case whose test case number is "3".
Set data patterns via CSV format
When there are many data patterns, it's easier to set them up using a CSV file.
To set up, click the ︙ button in the upper right corner, then select Upload CSV.
Figure 6: Set up data pattern screen
In this case, the CSV file will look as follows, with ',' separating each piece of data.
Figure 7: Sample CSV file.
Set data patterns via API
In case you want to set data patterns to many test cases or the data pattern contents frequently change and require configuration many times, you can upload data patterns efficiently via MagicPod Web API. There are two methods: using the MagicPod Web API directly or using the magicpod-api-client. Since the magicpod-api-client achieves uploading with a single command, it is the recommended way.
Upload with magicpod-api-client
Refer to this guide for the basic usage of the magicpod-api-client.
Using this command, you can upload data patterns in a CSV file for a specific test case.
./magicpod-api-client upload-data-pattern-csv -t {API_token} -o {organization_name} -p {project_name} --test_case_number 10 --csv_file_path data-pattern.csvFor details about the upload-data-pattern-csv command, run the following command to check the documentation.
./magicpod-api-client help upload-data-pattern-csvUpload by MagicPod Web API
Refer to this guide for the basic usage of the MagicPod Web API.
1. Send a POST API request
/v1.0/{organization_name}/{project_name}/test-cases/{test_case_number}/start-upload-data-patterns/Please refer to the API document for detailed parameter configurations. This command starts the process of uploading. "batch_task_id" is returned as a response, which will be used in Step 2 to monitor the status of the batch task.
2. Repeat GET API request
/v1.0/{organization_name}/{project_name}/batch-task/{batch_task_id}/Please send requests to this API until the returned status becomes "succeeded" or "failed." When you get "succeeded", the upload is completed. In case of "failed", "errors" attribute will display the error contents. Please confirm the value.
Use conditional branching commands
You can create tests that change which options to check for each data set. For example, you have the following checkbox and want to vary whether it is checked or not depending on the provided data.
Figure 8: Checkbox
In this case, set a variable 'BreakfastOption' and implement conditional branching based on the value of this variable. For more information on conditional branching commands, refer to Utilize Conditional Branching.
Figure 9: Set up data pattern
Figure 10: Example step to check the checkbox using the 'If variable value equals' command
Figure 11: Example step to uncheck the checkbox using the 'If variable value equals' command
Use variable-based locators
You can create tests that change the element’s locator depending on the variables in the data pattern. For example, suppose you have the following three checkboxes and want to change the one being checked in each data set, you would set an ‘id’ attribute to each checkbox in the source code.
Figure 12: Checkboxes
For this, set a variable ‘option_id’ and input the id name for each data.
Figure 13: Set up data pattern
Then, add a variable-based locator, and apply a click command to the element identified by this locator. This method allows you to change the click target for each data set. For more information on adding variable-based locators, refer to Add a locator.
Figure 14: Variable-based locator
Figure 15: Example step
Include a newline in patterns
It is possible to include a newline in data patterns.
▼Input
▼Output
When creating the data patterns in a CSV file format using a text editor, please include a newline character within the quotation marks("").
Limitations
The maximum limit of data patterns
The maximum limit is 300 iterations.
However, since the upper time limit for batch runs is 9.5 hours, executing 300 iterations may not be feasible depending on the execution time of each test case.
Visual diff approval
You can use the "Assert no visual diff" command in test cases with data patterns configured, but you cannot set expected images for each data pattern.
This enhancement request has been formally documented and prioritized within the MagicPod Product Portal. Should you require this feature enhancement, we encourage you to cast your vote on the corresponding feature request card to help us prioritize development efforts based on customer demand.
Check visual diff for each data pattern