Tips for initial introduction (Browser)

This section introduces the most frequently asked questions and tips for operation in the early stages of installation. If you have not yet used MagicPod, please refer to Started Guide first.

Table of Contents

• Test Creation
  • Not familiar with MagicPod-specific terms such as "UI", "UI elements", etc.
  • Cannot select a UI element
  • UI element you want to operate is off-screen
  • Not sure which locator to use
  • There are multiple operations that are performed over and over again across test cases, such as login
  • Click on a specific position of a UI element
  • Generate different strings for each execution
  • Specify a dynamically changing locator
  • Copy a locator
  • Close an alert
  • Move to a different tab or window
  • Test a site with Basic authentication enabled
  • Different options for "Open Browser" command
  • Make test steps easier to manage
  • Know what other commands are available
• Test Execution
  • Get an error saying UI element not found
  • Cannot understand the contents even if you see the failure message
  • Inquire about a test failure
  • Run regularly based on a schedule
  • Notify test results
  • Batch run takes time
  • Execute with multiple browser patterns
  • Succeed in single run but fail in batch run
• Test Management
  • Manage test cases
  • Manage test results
  • Divide and counting projects
• Other
  • Which items to start creating tests from

 

Test Creation

Not familiar with MagicPod-specific terms such as "UI", "UI elements", etc.

Please refer to  Start guide: 4 - 5. Description of terms for more information.

 

Cannot select a UI element

There is a possibility that other UI elements overlap with the UI element and cannot be selected. You can select the UI element you want to select by clicking Temporarily hide several times to hide the overlapping element.

If you still cannot select a UI element, you can send an inquiry by clicking Inquiry about this UI from the menu of the UI in UI List. You can download the XML file from the inquiry screen, and you can add a new locator based on the file.

Reference: Tested Elements Cannot be Selected From the UI / Add a Locator

 

UI element you want to operate is off-screen

With Element selection mode deactivated, scroll the cloud browser manually to display the UI element you want to manipulate on the screen. Then capture or start the element selection mode again to retrieve the element.

If you need to scroll repeatedly, it is recommended to capture the entire screen UI. With Element selection mode deactivated, check the Capture whole page checkbox, click Capture, and then retrieve the element.

When executed, even if you click elements that are off-screen, selecting the Click command for that element means that the click will be performed after it has automatically scrolled.

Reference: Test screens where you need to scroll

 

Not sure which locator to use

Basically, the suggested locators are the first ones specified and recommended from the top of the list of candidates. Add a new locator if necessary, as a locator with a value that changes from run to run or a long locator will be unstable.

Reference: Add a Locator

 

There are multiple operations that are performed over and over again across test cases, such as login

If multiple operations such as Input -> Click on the same screen appear in multiple test cases, you can define the operation as a Shared step so that it can be used as a single command from other test cases. When fixes are necessary, only a single fix of the content of the shared step is required, thus reducing the time and effort required for test case maintenance.

Reference: Utilize shared steps

 

Click on a specific position of a UI element

A UI element you want to manipulate is overlapped by another UI element.

You can click on the non-overlapping part of the UI element, such as the left edge.

A UI element not recognized by MagicPod

The UI element is intentionally placed outside of other recognizable UI elements nearby, such as at coordinates X=50,Y=200. (X=0,Y=0 for the upper left corner and X=100,Y=100 for the lower right corner)

This is only done when the "Cannot select a UI element" does not solve the problem, as it tends to be unstable.

The normal Click command operates by directly calling an event associated with a UI element, but the Click specified position command operates by clicking based on the apparent position, such as left edge or coordinates.

Reference: Click specified position

 

Generate different strings for each execution

You can use the Store unique value generated based on the current time command to create a different variable for each execution.

You can use the variable as it is or use it as Name_${Variable name}.

 

Specify a dynamically changing locator

Specify the Xth variable.

e.g.) xpath=//tr/td[3].

Specify, for example, the third of td. Select a locator from those proposed, or add a new locator based on the XML file.

Specify a partial match

e.g.) xpath=//a[text()='Category (112)'] → xpath=//a[contains(text(), 'Category')]

If the number after Category changes dynamically, you can use contains to specify only Category as a partial match, so add a new locator.

 

If you do not know what kind of locator to specify, you can inquire by sending UI information from the Inquiry about this UI of the UI's menu in UI List. You can download the xml file from the inquiry screen, and add a new locator based on that file.

Reference: Tested Elements Cannot be Selected From the UI / Add a Locator

 

Copy a locator

Press shortcut keys to copy the locator without lifting your finger from a mouse while dragging the area to be copied.

The shortcut keys are [cmd]+[C] on Mac, [ctrl]+[C] on Windows.

 

The locator is duplicated in Add a duplicated locator by pressing the icon next to the locator. You can edit this locator and add a new locator.

 

Close an alert

You can close an alert with the Close alert command.

The alert is a JavaScript alert dialog.

Other dialogs/popups/modals etc. can be recognized as UI elements and operated as usual.

 

Move to a different tab or window

You can use the Select new tab/window and Select previous tab/window commands.

Reference: Test separate tabs and separate windows

 

Test a site with Basic authentication enabled

For example, if you want to access https://example.com with user name user1 and password pass1 for Basic authentication, it can be accessed by using the URL https://user1:pass1@example.com.

Note: Test a site with basic authentication enabled

 

Different options for "Open Browser" command

Please refer to Open browser for differences in behavior.

Basically, (by first-time-only process restart), (by first-time-only process restart with clearing cookies), and (by process restart) are faster in that order, but not by much. For usage, select (by first-time-only process restart with clearing cookies) if you do not want to take over login information from the previous test when a batch run is performed, or (by process restart) if information remains in auth0 and so on.

 

Make test steps easier to manage

When you want to make a chunk of test steps easier to understand, you can use the Comment command to explain the contents, or the Blank line command to insert a blank line.

When you want to include a note, use the Comment command to highlight it with a pictogram.

 

Know what other commands are available

You can see all commands in Command List.

 

Test Execution

Get an error saying UI element not found

Put a wait time before the failed step.

Basically, it automatically waits until the screen has finished loading, but there are cases where the UI element cannot be found due to insufficient loading. In such cases, a command to wait until the UI element is visible/exists/equals, or a command to wait for a fixed value of X seconds, etc., can be placed before the failed step.

Reference: Wait command

Change the locator specified for the UI element of the failed step

When MagicPod searches for UI elements, it uses locators. For example, if the locator changes with each execution as shown below, the test will fail because the locator is different from the one used when the test was created. Therefore, it is necessary to change the locator from one of the proposed locators to another or add a new locator.

• Based on Amount/DateTime

  • e.g.) xpath=//output[text()='$70.00']

• Based on value, id, etc. that may change

  • e.g.) xpath=//input[@value='mp62130']

Reference: How to maintain a test when it fails - Fix an unstable test case

 

Cannot understand the contents even if you see the failure message

You may find a solution by searching for that failure message on the help page. If you cannot find it, or if it is difficult to identify the cause even after reading the help page, please contact us by clicking Inquiry about the test failure.

Reference: Inquiries About Test Failures

 

Inquire about a test failure

If you do not know the cause of the error, click the Inquiry about the test failure to inquire about it.

Reference: Inquiries About Test Failures

 

Run regularly based on a schedule

Use MagicPod's scheduled run feature to set up periodic execution.

Reference: Start guide: 3 - 1-1. Create MagicPod batch run settings

You can also call MagicPod's WebAPI with CI tools such as CircleCI, Bitrise, or Jenkins. In this case, you can also execute it every time it is pushed to Git.

Reference: Run tests regularly with CI services

 

Notify test results

You can be notified via email and Slack.

Reference: Start guide: 3 - 1-2. Set up test results email notification / 1-3. Set up test results Slack notification

Alternatively, you can use MagicPod's WebAPI to send notifications to Teams, etc.

 

Batch run takes a long time

Parallel run

If there are two flows, label the test cases Flow A and Flow B, and set each pattern in Batch runs settings to include the test case label Flow A on one side and Flow B on the other side.

Then, switch Running method to Parallel run in Common settings. The standard plan allows 2 parallel runs at the same time, but you can add options to allow up to 10 parallel runs.

Reference: To shorten the time required for test execution - Parallel run

Reference: How to increase test cases and projects

You can also specify test case numbers instead of labels, and you can also specify the order in which test cases are executed.

Note that there should be no dependencies between test cases.

For example,

• When creating a test case
  • Add pre- and post-processing steps for adding/deleting data.
  • When manipulating an element in a screen where data is listed, specify the locator based on a string, not the Xth locator, so that it can be manipulated even after new data has been registered.
• When setting up batch run
  • Test cases affected by another test case are not included in parallel executiorun.

Capture only specific steps

By default, all steps are captured, but you can save time by capturing only specific steps.

Select Every UI transit or Failure capture only from Every step in the Runtime Screenshots field on the batch runs settings screen. If there is a specific step you want to capture, add the Take screenshot command after that step.

 

Execute with multiple browser patterns

On the batch runs settings screen, click Add to add patterns, and set the browser type you wish to run for each. In this case, parallel run is possible as described above.

 

Succeed in single run but fail in batch run

It is difficult to isolate the cause because there are multiple patterns, but the most common causes are as follows.

• Failure due to a timing error
  
  • Adding a wait command such as Wait until UI element is visible/exists before that step will stabilize it.
• Failure due to data created by a previously executed test
  
  • Add a step to delete data at the end of the previously run test. You can also delete data from the screen or by calling API with Web API call if API is available.
  • Change the locator to one that is based on a string rather than Xth. If the string changes with each execution, you can put a variable containing the string in the locator.

    • e.g.) xpath=//tr/td[3] → xpath=//tr/td[text()='${Variable name}']

Reference: How to maintain a test when it fails - Fix an unstable test case

 

Test Management

Manage test cases

Since it is difficult to keep track of a large number of test cases, we can manage them by labels and test case names. If you also include descriptions of test cases,  they will be displayed in the test case list for easy understanding.

Reference: Create labels

 

Manage test results

• Paste result captures and result URLs into Spreadsheet/Excel.
  • You can download all result captures from the test results of a batch run. You can also selectively download by adding the Take screenshot command.
• Use cloud tools to manage test results (Testrail, QualityForward, etc.)
  • By retrieving test results by MagicPod's WebAPI, results data can be automatically accumulated.

Reference: Manage test results

 

Divide and counting projects

It is recommended to assign one project to one product.

Shared UI and shared steps can be used between test cases in a project.

For example, since the development environment and the production environment will have only one target product, we recommend that you do not separate projects, but share test cases. The execution environment can be changed by changing the base URL from the settings.

Reference: Switching Test Execution Environments (Switch Domain to Use Test Case)

 

Other

Which items to start creating tests from

First, create tests for basic features such as login, and payments, which pose the greatest risk for quality assurance.

Once you have several test cases, run them daily. When you run the tests every day, you will encounter various problems that you were not aware of until you started running the tests, such as test stability. As you solve these problems, you gradually increase the number of tests.

After that, you will cover a wide and shallow range of features and screens. Rather than automating the manual test cases at hand in order from the top to the bottom, it is more efficient to prioritize and automate only the basic test cases first.

Reference: Start guide: 5. Establish test automation habits as quickly as possible (Browser)

Another method of prioritization is to weight test items based on their suitability for automation and compare the scores.